Yuuyuuyuuyuu/Swift-Qwen3.8-27B-OrcaRouter-NVFP4-DFlash2-ninfer

Swift-Qwen3.8-27B-OrcaRouter-NVFP4-DFlash2

An uncensored community release based on Swift Qwen3.8-27B, packaged in native NInfer format with mixed NVFP4/FP8 weights and an embedded DFlash2 K=7 speculative-decoding head.

The target model uses an OrcaRouter-style single-direction transformation derived from the Swift checkpoint. This is a third-party release, not an official UkisAI or OrcaRouter checkpoint. The bundled DFlash2 head comes from the upstream Qwen3.8 DFlash2 companion and was not retrained specifically for Swift, so speculative acceptance depends on workload.

Property	Value
Model family	Qwen3.8-27B
Source	ukisai/Swift-Qwen3.8-27b
Format	.ninfer
Quantization	Mixed NVFP4/FP8
Speculative decoding	DFlash2, embedded in the model file
Recommended draft window	K=7
File size	23.72 GB (22.09 GiB)
SHA-256	b57bb82f6c43fa4bba3bcb883f45ccff12b81fd461502a37c79d78c3ba7a979b

Download

hf download Yuuyuuyuuyuu/Swift-Qwen3.8-27B-OrcaRouter-NVFP4-DFlash2-ninfer \
  --local-dir ./swift-ninfer

Model file: Swift-Qwen3.8-27B-OrcaRouter-NVFP4-DFlash2.ninfer

Run with NInfer

Runtime compatibility: This download is a NInfer container v2 artifact. The example was validated with NInfer revision d4929686; the corpus benchmark used 03177b91. Use a v2-compatible build with Qwen3.8-27B NVFP4 and DFlash2 support. Current upstream requires v3 artifacts; a v3-only build is not a drop-in loader for this file. These benchmark results refer to the unchanged v2 download, not a later format conversion.

Use a compatible NInfer build with Qwen3.8-27B NVFP4 and DFlash2 support. This is not GGUF and cannot be loaded directly with Transformers or llama.cpp.

ninfer-serve ./swift-ninfer/Swift-Qwen3.8-27B-OrcaRouter-NVFP4-DFlash2.ninfer \
  --host 127.0.0.1 --port 8080 \
  --max-context 8192 --kv-dtype fp8 \
  --spec dflash2 --draft-tokens 7 --lm-head-draft

Add --vision for image input. Increase --max-context as VRAM allows. DFlash2 is already embedded; no separate draft model download is required.

Benchmarks

PCIe disclosure: These benchmarks used an RTX 5090 in a PCIe 5.0 x8 configuration (not x16). The x8 host configuration was identified retrospectively; these original runs did not record per-sample PCIe link-width telemetry. The published measurements are retained unchanged as x8-host results, with no x16 correction or estimated speedup applied.

Benchmarks below were run on an NVIDIA RTX 5090. They measure serving performance, not model quality. Results can vary with NInfer revision, driver/toolkit, clocks, thermals, context length, sampling, and prompt mix.

Native ninfer_bench throughput

NInfer revision: d49296868dcc17bd478ec185f0d3a801bcc0bf56
Configuration: DFlash2 K=7, 128 generated tokens, 3 measured runs after 1 warm-up, BF16 KV, 16,384 context, 512-token prefill chunk.

Prompt tokens	Generation	Prefill	First token*	Total request
512	475.58 tok/s	9,756.06 tok/s	0.0525 s	0.3198 s
4,096	468.34 tok/s	9,099.31 tok/s	0.4502 s	0.7216 s

* ninfer_bench first-token time here is native preparation + prefill, not HTTP streaming TTFT.

NInfer upstream 75-request corpus-makespan

This uses NInfer's own fixed 75-request corpus-makespan benchmark with the published Qwen3.8 DFlash2 K=7 methodology.

NInfer revision: 03177b910e70f783b00c4f980ce0d1896a6b8592
Configuration: concurrency 1, 131,072 context, INT8-group64 KV, prefill chunk 1,024, stochastic sampling, temperature 0.6, top-p 0.95, top-k 20, min-p 0, presence penalty 1.0, DFlash2 K=7 with optimized proposal head.

Metric	Result
Requests completed	75 / 75
Failed requests	0
Corpus decode throughput	217.0 tok/s
DFlash2 acceptance	40.9%
Completion tokens	782,783
Total makespan	3,607.69 s

Workload breakdown

Workload	Samples	Decode throughput	DFlash2 acceptance
Code	15	277.4 tok/s	53.9%
Story	15	126.9 tok/s	16.7%
Translation	15	258.8 tok/s	48.9%
Structured	15	387.9 tok/s	81.2%

Long-reasoning fixtures

Fixture	Samples	Avg. completion tokens	Decode throughput	DFlash2 acceptance	Finish outcome
AIME 2026 #1	5	1,262.6	330.2 tok/s	66.4%	5/5 stop
AIME 2026 #15	5	65,536.0	206.9 tok/s	39.1%	5/5 output limit
AIME 2026 #30	5	53,927.8	232.0 tok/s	45.0%	4/5 stop, 1/5 output limit

These AIME fixtures are used here as long-generation throughput workloads. No AIME answer-accuracy score is claimed.

Upstream NInfer stock reference

For context, NInfer's published stock Qwen3.8-27B NVFP4 + DFlash2 K=7 ND run on its RTX 5090 serving profile reports 192.5 tok/s corpus decode throughput and 37.0% DFlash2 acceptance on the same 75-request methodology. This upload measured 217.0 tok/s and 40.9%, respectively.

75-request corpus-makespan	This upload	NInfer published stock Qwen3.8 NVFP4 reference
Corpus decode throughput	217.0 tok/s	192.5 tok/s
DFlash2 acceptance	40.9%	37.0%
Completion tokens	782,783	695,432
Makespan	3,607.69 s	3,611.52 s

The stock reference is an external published result, not a rerun performed for this upload. Because the target weights differ and sampling is stochastic, continuations and total generated-token counts differ. Treat the table as a serving-performance reference, not a controlled quality comparison. In particular, makespan should not be compared without also considering completion-token totals.

Benchmark notes

• The DFlash2 head is the upstream Qwen3.8 companion rather than a Swift-specific retrain.
• Aggregate corpus acceptance is summed accepted tokens divided by summed drafted tokens. Per-workload acceptance is the mean of per-request ratios, matching the upstream methodology.
• Output-limit requests count as completed requests. Some long-generation fixtures intentionally exercise sustained decode for tens of thousands of tokens.
• Throughput results are hardware- and runtime-specific and should not be interpreted as general quality scores.

Credits and license

Thanks to UkisAI for Swift, Qwen for the base model, OrcaRouter for the published decensoring approach, and the DFlash2 and NInfer projects.

The Swift contribution is covered by the Swift Open License v1.0. Qwen and DFlash2 components are covered by Apache-2.0. See LICENSE, LICENSE-APACHE-2.0, and NOTICE. The Swift license includes commercial-use conditions; this release does not change them.

A Swift 1.5 version is also available.