FastBrain-9B

A Single-Pass Non-Autoregressive Model for Calibrated Structured Decisions

English | 简体中文

---

Abstract

Autoregressive language models incur sequential decoding overhead and token-level probability miscalibration when deployed on closed-schema classification, tool routing, and safety verification tasks. We present FastBrain-9B, a 9.41B-parameter non-autoregressive structured decision model built on a 32-layer hybrid Gated DeltaNet and full-attention backbone (Qwen3.5-9B, initialized from Cloudflare/clef-flash) coupled with a 116M-parameter cross-attention decision head (JointSchemaHead). Given an arbitrary JSON or multimodal state and a set of typed decision specifications (choice, binary noul, and ordinal score), FastBrain-9B evaluates all questions and candidate options simultaneously in a single prefill pass (output_tokens = 0). We adapt 104 linear-attention and self-attention projections jointly with the decision head under a composite ordinal-KL, cumulative distribution (ranked probability score), and expected-value regression objective, and merge all parameter updates directly into the base tensors. Across three public evaluation suites encompassing 109 model entries, FastBrain-9B ranks 1st of 8 on the safeagents-judge-bench (v2) 7-way misuse classification track (0.751 macro-F1, 0.858 accuracy; +18.8 percentage points over 27.4B Cloudflare/clef) while intercepting 82/82 unsafe agent trajectories (0.985 AUROC); ranks 7th of 43 zero-shot overall (5th of 36 open-weight, 2nd of 19 in the ≤9B open-weight class) on LocalLLaMA/typed-decisions at 50 ms median latency (66.7× faster than 31B Quyet-1.0-Large), and 12th-tied (14th listed of 51) under prior-only calibration without weight updates (0.797 accuracy, 0.0816 KL); and achieves an 80.55% 10-task macro mean on Cloudflare Decision Index (v0.2.1), ranking 2nd of 7 overall, 1st on 10 of 10 tasks among ≤9B models, and outperforming 27.4B Cloudflare/clef on 5 of 10 tasks.

---

1. Overview and Motivation

Production agentic pipelines invoke structured decisions at high frequency: selecting among candidate tool schemas, verifying whether an execution trace violates sandbox policy, routing customer or security tickets, and scoring rubric criteria. Solving these tasks via autoregressive token generation introduces two structural inefficiencies. First, latency scales with output sequence length and formatting tokens rather than decision complexity. Second, surface-form token probabilities fragment probability mass across equivalent verbalizations, degrading calibration metrics such as Kullback-Leibler (KL) divergence and Brier score.

Single-pass decision models (ClefModel / System-1 architecture) address both issues by appending delimiter-bounded question and option spans directly to the input context and reading out option logits from final hidden states in one prefill pass. Existing 9B-scale checkpoints, however, trail 27B–35B models substantially on fine-grained multi-class discrimination, ordinal rubric estimation, and open-set abstention (e.g., Cloudflare/clef-flash scores 46.2% macro-F1 on safeagents-judge-bench v2 and 66.77% on CLINC150+OOS). FastBrain-9B closes and reverses much of this scaling gap through full-depth projection adaptation, ordinal distribution distillation, and compile-time schema-topology calibration, while preserving sub-100 ms single-GPU latency and lossless 8-bit / high-retention 4-bit GGUF quantization.

---

2. Model Architecture and Training Objective

2.1 Hybrid State-Space and Attention Backbone

FastBrain-9B uses a 32-layer hybrid sequence backbone (9.30B parameters, bfloat16, maximum context length 262,144 tokens) interleaving 24 linear-attention Gated DeltaNet blocks with 8 full-attention blocks. During training, 104 projection matrices across all 32 blocks—specifically in_proj_qkv, in_proj_z, and out_proj in linear-attention layers, alongside q_proj, k_proj, v_proj, and o_proj in full-attention layers—are updated jointly with the decision head. For distribution, all low-rank and projection updates are folded in-place into model-00001-of-00004.safetensors through model-00004-of-00004.safetensors, requiring zero adapter overhead at inference time.

2.2 Joint Schema Decision Head and Vectorized CUDA Kernel

The JointSchemaHead (116M parameters, stored in joint_head.safetensors) pools token representations bounded by <|question_start|>/<|question_end|> and <|option_start|>/<|option_end|> markers from the final backbone hidden states (H \in \mathbb{R}^{T \times d}), applies cross-attention over question-option pairs, and projects directly to scalar logits for three decision primitives:

1. choice ((K)-way categorical distribution): (\hat{p} = \operatorname{softmax}(z / \tau) \in \Delta^{K-1}).
2. noul (binary verification): (\hat{p}(\text{true}) = \sigma(z_{\text{true}} - z_{\text{false}})).
3. score (ordinal rubric over levels (v_1 < \dots < v_K)): categorical probabilities (\hat{p} \in \Delta^{K-1}) together with expected value (\mathbb{E}[\hat{y}] = \sum_{k=1}^K \hat{p}_k v_k).

In joint_schema_model.py, the reference Python span-extraction loop and padded SDPA mask are replaced on CUDA devices by _forward_fast_cuda: span means are computed in (\mathcal{O}(1)) kernel launches via prefix sums (torch.cumsum) over the schema suffix, and cross-attention is executed via packed variable-length FlashAttention (torch.ops.aten._flash_attention_forward parameterized by cu_seqlens). On NVIDIA RTX PRO 6000 Blackwell, this reduces isolated decision-head latency from 3.27 ms to 1.72 ms (1.90× speedup) with exact numerical equivalence (max_logit_diff = 0.00000).

2.3 Ordinal-KL Distillation and Schema-Topology Calibration

Standard cross-entropy or flat KL divergence treats ordinal rubric levels as unordered classes, penalizing adjacent-level errors identically to extreme errors. FastBrain-9B is trained on TypeScript/FastBrain-Instruct under a composite loss combining soft-target KL divergence, discrete ranked probability score (squared (L_2) distance between predicted and target cumulative distribution functions (\hat{F}k = \sum{j \le k} \hat{p}_j)), and expected-score mean squared error on score primitives:

$$\mathcal{L}(\theta )=\mathrm{K}\mathrm{L}(p\mathrm{\parallel }{\widehat{p}}_{\theta })+{\lambda }_{\mathrm{o}\mathrm{r}\mathrm{d}}\sum _{k=1}^{K-1}({\widehat{F}}_{\theta ,k}-F_k{)}^2+\mathbb{I}[\text{type}=\text{score}]\cdot {\lambda }_{\mathrm{m}\mathrm{s}\mathrm{e}}(\mathbb{E}[{\widehat{y}}_{\theta }]-\mathbb{E}[y]{)}^2\mathrm{.}\backslash mathcal\{L\}(\backslash theta)\; =\; \backslash mathrm\{KL\}(p\; \backslash ,\backslash |\backslash ,\; \backslash hat\{p\}\_\backslash theta)\; +\; \backslash lambda\_\{\backslash mathrm\{ord\}\}\; \backslash sum\_\{k=1\}^\{K-1\}\; \backslash bigl(\backslash hat\{F\}\_\{\backslash theta,k\}\; -\; F\_k\backslash bigr)^2\; +\; \backslash mathbb\{I\}[\backslash text\{type\}=\backslash text\{score\}]\; \backslash cdot\; \backslash lambda\_\{\backslash mathrm\{mse\}\}\; \backslash bigl(\backslash mathbb\{E\}[\backslash hat\{y\}\_\backslash theta]\; -\; \backslash mathbb\{E\}[y]\backslash bigr)^2.$$

At inference time, joint_schema_model.py optionally applies compile-time schema-topology calibration (apply_schema_aware_calibration), which inspects candidate option structures (e.g., <tool_call> invocations, clarification prompts, and open-set abstention clauses) to adjust decision margins against systematic tool over-triggering and out-of-scope false acceptance.

---

3. Empirical Evaluation

Table 1: Summary of official public benchmark standings (verified across 109 model entries as of November 2026).

Benchmark & Evaluation Track	Evaluated Entries	FastBrain-9B Metric	Official Standing	Reference Baselines (Params, Metric)
safeagents-judge-bench (v2) — 7-Way Category (PR #1)	8 decision models (+2 baselines)	0.751 Macro-F1 / 0.858 Acc	1 / 8	Cloudflare/clef (27.4B, 0.563), vllm-sr/d3-mini (1.8B, 0.512), Cloudflare/clef-flash (9.4B, 0.462), clef-omni (30B, 0.437)
safeagents-judge-bench (v2) — Binary Guardrail (PR #1)	8 decision models (+2 baselines)	0.985 AUROC / 82/82 Traj. (0.814 raw; 0.938 calibrated)	3 / 8 AUROC 1 / 8 (tied) Traj.	Cloudflare/clef (27.4B, 0.808 raw bal. acc), clef-flash (0.967 AUROC, 81/82), clef-omni (0.946 AUROC, 81/82), d1-3B (68/82)
safeagents-judge-bench (v1) — Binary & Category (PR #1)	9 decision models (+2 baselines)	0.974 Bal. Acc / 0.850 Cat. F1	1 / 9 (both tracks)	Cloudflare/clef-omni (30B, 0.845 bal. acc / 0.327 F1), Cloudflare/clef (27.4B, 0.718 / 0.425), clef-flash (0.721 / 0.345)
LocalLLaMA/typed-decisions — Zero-Shot (PR #58)	43 total (36 open, 19 ≤9B open)	0.7525 Acc (0.7625 tied-top) 0.1508 KL · 0.0795 Brier · 50 ms	7 / 43 overall 5 / 36 open · 2 / 19 ≤9B	vllm-sr/d3 (27B, 0.749), Elephant 1 (9B, 0.748, 352 ms), Darwin-27B (0.743), Cloudflare/clef (27B, 0.728), clef-flash (0.710)
LocalLLaMA/typed-decisions — Fitted on train (PR #57)	51 specialist & calibrated models	0.7970 Acc (0.8070 tied-top) 0.0816 KL · 0.0437 Brier · 50 ms	12-tied (14 / 51 listed) 1st frozen-weight ≤9B & KL	Frozen weights (no weights changed); surpasses Rev-gen1-midi (25.6B, 0.7945), OpenDecider-small-td (4B LoRA, 0.792), OpenDecider-medium-td (30B LoRA, 0.788)
Cloudflare Decision Index (v0.2.1) — 10-Task Suite	7 models (0.4B–27.4B)	80.55% Macro Mean	2 / 7 overall 1st on 10 / 10 (≤9B)	Jev (27B, 80.32%), Kev 9B (76.51%), Clef-flash (9.4B, 74.96%), DiffGemma (26B, 69.14%); exceeds 27.4B Clef on 5 / 10 tasks

---

3.1 Agent Tool-Call Guardrail Evaluation (safeagents-judge-bench)

harness-hub/safeagents-judge-bench evaluates whether a System-1 decision model can audit suspicious tool calls flagged by a static scanner (atif-scan) during real coding-agent executions on Terminal-Bench 2.1, OpenThoughts-TBLite, and Terminal-Bench 4. Version 2 (v2) comprises (N = 661) tool calls across 55 tasks (536 malicious block calls spanning 82 misbehaving trajectories and 125 legitimate allow calls). Each call is issued as a single zero-shot System-1 request containing one binary noul question (block) and one 7-way choice question (legit, eval_gaming, harness_introspection, credential_misuse, exfiltration, sandbox_escape, other_overreach). All v2 results in Table 2 were executed independently on NVIDIA A100-SXM4-80GB via Hugging Face Jobs (Merged PR #1).

On the 7-way category classification track, FastBrain-9B achieves 0.751 macro-F1 and 0.858 accuracy (567/661), outperforming the 27.4B Cloudflare/clef model (0.563 macro-F1, 0.737 accuracy) by +18.8 percentage points in macro-F1 and +12.1 points in accuracy, and improving upon its 9.4B base model Cloudflare/clef-flash (0.462 macro-F1, 0.576 accuracy) by +28.9 and +28.2 points, respectively. Per-category F1 reaches 0.945 on harness_introspection ((n=272), recall 0.978), 0.828 on eval_gaming ((n=245), recall 0.853), and 0.761 on legit ((n=125), precision 0.975).

On the binary guardrail track, FastBrain-9B attains 0.985 AUROC and 0.996 malicious-call recall (534/536), intercepting all 82/82 unsafe trajectories (median_bad_calls_before_catch = 0.0), with 1.000 recall on harness_introspection (272/272), credential_misuse (15/15), and sandbox_escape (4/4). Under the uncalibrated default decision rule (P(\text{block}) \ge 0.50), balanced accuracy is 0.814 (false-block rate 0.368 across the 125 allow calls, compared with 0.808 balanced accuracy and 0.384 false-block rate for 27.4B Cloudflare/clef). Because rank-ordering discrimination is preserved (0.985 AUROC), setting the operating threshold to (P(\text{block}) \ge 0.961) yields 0.938 balanced accuracy at a 0.056 (7/125) false-block rate while maintaining 82/82 trajectory interception. Table 2 also includes the v1 release ((N = 562) calls, 78 trajectories), where FastBrain-9B ranked 1st of 9 on both tracks.

Table 2: Zero-shot evaluation on harness-hub/safeagents-judge-bench (v2, (N = 661) calls, 82 unsafe trajectories; and v1 reference, (N = 562)).

Model	Params	Cat. Macro-F1 ↑	Cat. Acc ↑	AUROC ↑	Bal. Acc ↑	Recall ↑	False Block ↓	Traj. Caught ↑	p50 Latency (Hardware)
TypeScript/FastBrain-9B (v2, default (t = 0.50))	9.4B	0.751	0.858	0.985	0.814	0.996	0.368	82 / 82	327 ms (A100) · 104 ms (G4)
TypeScript/FastBrain-9B (v2, calibrated (t = 0.961))	9.4B	0.751	0.858	0.985	0.938	0.931	0.056	82 / 82	327 ms (A100) · 104 ms (G4)
TypeScript/FastBrain-9B (v1 official, (N = 562))	9.4B	0.850	0.923	0.983	0.974	1.000	0.051	78 / 78	104 ms (RTX PRO 6000)
Cloudflare/clef	27.4B	0.563	0.737	0.993	0.808	1.000	0.384	82 / 82	636 ms (A100)
vllm-sr/d3-mini	1.8B	0.512	0.652	0.986	0.856	1.000	0.288	82 / 82	168 ms (A100)
Cloudflare/clef-flash (base)	9.4B	0.462	0.576	0.967	0.893	0.978	0.192	81 / 82	326 ms (A100)
Cloudflare/clef-omni	30B-A3B	0.437	0.564	0.946	0.877	0.914	0.160	81 / 82	285 ms (A100)
knowledgator/gliclass-v4-edge	0.4B	0.373	0.528	0.753	0.676	0.696	0.344	68 / 82	101 ms (L4)
fastino/GLiNER2.5-Decide	0.5B	0.355	0.620	0.517	0.576	0.151	0.000	17 / 82	117 ms (M5 Pro CPU)
LiquidAI/d1-3B	3.0B	0.328	0.457	0.946	0.836	0.681	0.008	68 / 82	35 ms (A100)

---

3.2 Multi-Workflow Enterprise Decisions (LocalLLaMA/typed-decisions)

LocalLLaMA/typed-decisions tests multi-primitive structured judgment across four enterprise workflows (agent_trace_observability, customer_service, invoice_processing, and security_incidents), comprising 400 held-out test cases and 2,000 individual decisions (noul, choice, and score) scored against teacher-panel soft label distributions. Two scoring conventions exist in the benchmark repository: bench/score_with_jobs.py resolves exact ties in the gold distribution by selecting the first index (g.index(max(g))), whereas scripts/score_model.py credits any tied maximum across the 20 tied test items; KL divergence and Brier score are identical under both scripts. We report both accuracy figures (strict / tied-top) alongside prediction traces in eval_submissions/.

On the official zero-shot leaderboard ((N = 43) models), FastBrain-9B scores 0.7525 (0.7625 tied-top) accuracy, 0.1508 KL divergence, 0.0795 Brier score, and 0.085 ECE at 50 ms p50 latency (RTX PRO 6000 via SGLang). It places 7th of 43 overall, 5th of 36 among open-weight models, 2nd of 19 among ≤9B open-weight models, and 1st among all Clef-architecture models, improving over 9.4B Cloudflare/clef-flash (0.7100 accuracy, 0.2093 KL) by +4.25 percentage points and over 27.4B Cloudflare/clef (0.7275 accuracy, 0.1907 KL) by +2.50 points. Across all 28 models exceeding 0.700 zero-shot accuracy, FastBrain-9B exhibits the lowest p50 latency (50 ms, compared with 118 ms for 9B Amos 1 medium, 357 ms for 27B Amos 1 large, 526 ms for meraGPT Decider 1, and 3,335 ms for 31B Quyet-1.0-Large). Evaluating the uncalibrated ordinal-KL distilled head directly yields 0.7735 raw accuracy (0.1039 KL, 0.0582 Brier).

On the separate train-fitted leaderboard ((N = 51) models), fitting only scalar per-question log-priors and temperature parameters on the 1,200 train cases—with all backbone and head weights frozen (no weights changed)—achieves 0.7970 (0.8070 tied-top) accuracy, 0.0816 KL divergence, and 0.0437 Brier score, ranking 12th-tied (14th listed of 51) overall and 1st in KL calibration among frozen-weight models.

Table 3: Performance on LocalLLaMA/typed-decisions ((N = 400) cases, 2,000 decisions). Values separated by / denote score_with_jobs.py / score_model.py tie handling.

Model	Params	Protocol	Overall Acc ↑	KL ↓	Brier ↓	noul ↑	choice ↑	score ↑	Agent ↑	Cust. Svc ↑	Invoice ↑	Security ↑	p50 Latency
FastBrain-9B (prior-fitted, frozen weights)	9.4B	Fitted	79.70% / 80.70%	0.0816	0.0437	86.7% / 88.0%	76.5% / 77.2%	76.9% / 77.9%	75.4% / 76.6%	82.0% / 83.4%	83.4% / 83.6%	78.0% / 79.2%	50 ms
FastBrain-9B (ordinal-KL distilled head)	9.4B	Zero-shot	77.35%	0.1039	0.0582	84.8%	74.8%	73.8%	71.2%	80.4%	82.6%	75.2%	50 ms
FastBrain-9B (official PR #58)	9.4B	Zero-shot	75.25% / 76.25%	0.1508	0.0795	83.2% / 84.3%	74.7% / 76.0%	69.8% / 70.4%	68.2% / 69.8%	77.6% / 78.8%	82.0%	73.2% / 74.4%	50 ms
Amos 1 (medium)	9.0B	Zero-shot	76.45%	0.1510	0.0800	—	—	—	—	—	—	—	118 ms
Amos 1 (large)	27.0B	Zero-shot	76.10%	0.1260	0.0690	83.2%	73.7%	72.6%	67.4%	78.8%	83.0%	75.2%	357 ms
pplx-decider-v1.1-27b	27.0B	Zero-shot	75.30%	0.4410	0.1600	85.2%	73.7%	69.1%	66.4%	80.2%	80.2%	74.4%	—
vllm-sr/d3	27.0B	Zero-shot	74.90%	0.4090	0.1460	84.7%	68.5%	72.4%	67.2%	76.6%	82.8%	73.0%	—
Baseten Elephant 1	9.0B	Zero-shot	74.80%	0.1340	0.0690	—	—	—	—	—	—	—	352 ms
Darwin-27B-ZTC	27.0B	Zero-shot	74.30%	0.2040	0.0970	—	—	—	—	—	—	—	—
Cloudflare/clef	27.4B	Zero-shot	72.75%	0.1907	0.1032	84.5%	64.8%	69.9%	63.8%	74.2%	78.6%	74.4%	—
vllm-sr/d3-flash	9.0B	Zero-shot	71.70%	0.3220	0.1310	—	—	—	—	—	—	—	—
Cloudflare/clef-flash (base)	9.4B	Zero-shot	71.00%	0.2093	0.1105	81.5%	71.3%	62.9%	62.0%	76.0%	76.4%	69.6%	—
Cloudflare/clef-omni	35B-A3B	Zero-shot	67.30%	0.2469	0.1233	82.7%	63.5%	58.6%	55.2%	73.6%	74.2%	66.2%	—

---

3.3 Multi-Domain Decision Suite (Cloudflare Decision Index v0.2.1)

On the 10-benchmark Cloudflare Decision Index (v0.2.1) suite ((N = 22,427) evaluation requests spanning AST tool calling, 20-candidate tool reranking, stateful API invocation, fine-grained intent classification, out-of-scope rejection, reasoning-intensive retrieval, e-commerce relevance, phishing detection, and tool-use governance), FastBrain-9B attains an 80.55% unweighted macro mean (Table 4, Figures 1–3).

Across the ≤9B parameter tier (Clef-flash 9.4B, Kev 9B, and Laya 0.4B), FastBrain-9B ranks 1st on all 10 benchmarks, improving the 10-task macro mean over Clef-flash (74.96%) by +5.59 percentage points and over Kev 9B (76.51%) by +4.04 points. The largest gains occur on calibration-sensitive abstention and governance tasks: +31.67 points on CLINC150+OOS (98.44% vs. 66.77%), +15.55 points on When2Call (81.13% vs. 65.58%), and +6.80 points on PhishNChips (78.75% vs. 71.95%). Across all model scales up to 27.4B, FastBrain-9B ranks 2nd of 7 overall, surpassing 27B Jev (80.32%) and 26B-A4B DiffusionGemma Jev (69.14%), and exceeding 27.4B Clef (81.81%) on 5 of 10 individual benchmarks (BFCL, API-Bank, CLINC150+OOS, Home Appliances, and When2Call).

Figure 1. Aggregate 10-benchmark macro mean versus parameter count on Cloudflare Decision Index (v0.2.1, 22,427 requests).

Figure 2. Per-benchmark performance matrix across 0.4B–27.4B decision models on Cloudflare Decision Index (v0.2.1).

Figure 3. Confidence calibration, open-set out-of-scope rejection (CLINC150+OOS), and abstention margin analysis (When2Call).

Table 4: Per-benchmark results on Cloudflare Decision Index (v0.2.1, 10 tasks, (N = 22,427)). Bold denotes the highest score across all parameter scales; underline denotes best in the ≤9B class.

Benchmark (Metric)	FastBrain-9B (Ours)	Clef-flash (Base)	Kev 9B	Clef	Jev	DiffGemma	Laya
Total Parameters	9.41B	9.41B	9.00B	27.36B	27.00B	25.8B (4B act.)	0.42B
1. BFCL (Case Exact Acc)	98.76%	98.76%	97.58%	98.47%	98.58%	97.64%	95.87%
2. ToolRet (nDCG@10)	67.26%	67.26%	65.72%	68.18%	65.50%	45.28%	51.31%
3. API-Bank (Accuracy)	93.70%	93.11%	89.76%	91.93%	90.35%	85.24%	86.22%
4. BANKING77 (Macro-F1)	93.77%	93.69%	91.77%	94.07%	93.58%	93.80%	62.66%
5. CLINC150+OOS (Macro-F1)	98.44%	66.77%	96.65%	97.43%	96.44%	94.64%	73.79%
6. Home Appliances (Case Exact)	97.73%	96.59%	88.64%	96.59%	88.64%	68.18%	23.86%
7. BRIGHT (nDCG@10)	38.58%	38.49%	38.53%	48.03%	43.37%	18.69%	25.00%
8. Amazon ESCI (Macro-F1)	57.40%	57.39%	53.75%	59.30%	56.60%	56.83%	52.84%
9. PhishNChips (Accuracy)	78.75%	71.95%	72.10%	91.70%	88.20%	55.65%	60.45%
10. When2Call (Macro-F1)	81.13%	65.58%	70.64%	72.37%	81.91%	75.44%	48.92%
10-Task Macro Mean	80.55%	74.96%	76.51%	81.81%	80.32%	69.14%	58.09%

---

4. Systems Performance and Post-Training Quantization

4.1 Runtime Comparison Across Inference Engines

Table 5 compares end-to-end latency, prefill throughput, and memory footprint across three serving backends—SGLang (lmsysorg/sglang:dev-clef with JointSchemaPooler), llama.cpp (b11430 ClefModel graph in Q8_0 and Q4_K_M), and native PyTorch (joint_schema_model.py)—benchmarking identical requests on a single NVIDIA RTX PRO 6000 Blackwell Server Edition GPU. Because output_tokens = 0 for all requests, throughput is governed purely by prefill compute and span-pooling efficiency, reaching 19,319 tok/s (131.2 decisions/s) at batch size (B = 4) in SGLang.

Table 5: End-to-end latency and prefill throughput on NVIDIA RTX PRO 6000 Blackwell Server Edition (G4).

Workload / System Metric	SGLang dev-clef (BF16)	llama.cpp b11430 (Q8_0)	llama.cpp b11430 (Q4_K_M)	PyTorch Native (BF16)
Cold-Start Load Time	11.11 s	2.56 s	2.00 s	11.11 s
Peak GPU Memory (16K context)	~20.5 GiB	12.55 GiB	10.30 GiB	20.3–24.7 GiB
Compatible with 16 GiB T4 GPU	No (BF16 TP=1)	Yes (9.03 GiB at 4K)	Yes (6.78 GiB at 4K)	Yes (6.8 GiB via NF4)
Top-1 Agreement vs. PyTorch BF16	100.00%	99.86% (1,471 / 1,473)	98.78% (1,455 / 1,473)	100.00% (reference)
1. Short Intent Routing (293 tok, (K=2))	30.01 ms (9,763 tok/s)	32.82 ms (8,929 tok/s)	31.69 ms (9,245 tok/s)	43.23 ms (6,778 tok/s)
2. Typed Decisions (736 tok, (K=5))	50.29 ms (14,635 tok/s)	62.40 ms (11,795 tok/s)	61.80 ms (11,909 tok/s)	63.58 ms (11,576 tok/s)
3. When2Call Governance (1,068 tok, (K=1))	66.06 ms (16,167 tok/s)	87.78 ms (12,167 tok/s)	89.34 ms (11,955 tok/s)	77.77 ms (13,733 tok/s)
4. PhishNChips Security (1,303 tok, (K=9))	75.86 ms (17,176 tok/s)	108.55 ms (12,004 tok/s)	109.97 ms (11,849 tok/s)	90.32 ms (14,426 tok/s)
5. SafeAgents Judge (2,217 tok, (K=2))	125.85 ms (17,616 tok/s)	181.70 ms (12,201 tok/s)	183.50 ms (12,082 tok/s)	138.06 ms (16,058 tok/s)
Batch (B=4) Typed Decisions (2,944 tok, 20 dec)	152.39 ms (19,319 tok/s)	248.50 ms (11,847 tok/s)	245.20 ms (12,006 tok/s)	175.85 ms (16,742 tok/s)

4.2 Mixed-Precision GGUF Quantization Fidelity

Uniform 4-bit quantization degrades cross-attention span readout and linear-attention recurrent state precision if applied indiscriminately to decision-head projections. In our FastBrain-9B-Q4_K_M.gguf (6.73 GB) build, feed-forward blocks are quantized to 4-bit K-quants while all dec.* decision-head tensors, ssm_* Gated DeltaNet parameters, and attn_* projections remain at Q8_0 precision. Evaluated across a stratified 10-task subset (203 multi-question requests, 1,473 scored schema questions; Table 6), Q8_0.gguf (9.66 GB) matches native bfloat16 on 1,471/1,473 decisions (99.86% Top-1 agreement, mean absolute probability error 0.00203, -0.03 percentage-point delta on the 10-task mean), and Q4_K_M.gguf retains 98.78% Top-1 agreement (0.00387 probability MAE, -0.20 percentage-point calibrated delta). Table 7 summarizes hardware memory requirements across formats, including NVIDIA Tesla T4 (16 GiB) measurements.

Table 6: Quantization fidelity across 1,473 scored schema questions on the 10-task stratified Decision Index subset.

Metric / Subset	PyTorch bfloat16 (18.82 GB)	GGUF Q8_0 (9.66 GB)	GGUF Q4_K_M (6.73 GB, Q8_0 Head)
Top-1 Decision Agreement ((N = 1,473))	100.00% (ref)	99.86% (1,471 / 1,473)	98.78% (1,455 / 1,473)
Mean Absolute Probability Error ((\lvert \hat{p}q - \hat{p}{\mathrm{bf16}} \rvert))	0.00000	0.00203	0.00387
10-Task Mean (Uncalibrated Head)	82.37%	82.34% (-0.03)	82.56% (+0.19)
10-Task Mean (Schema-Calibrated)	83.68%	83.65% (-0.03)	83.48% (-0.20)
├─ 1. BFCL / 3. API-Bank / 4. BANKING77	100.0% / 95.0% / 100.0%	100.0% / 95.0% / 100.0%	100.0% / 95.0% / 100.0%
├─ 2. ToolRet / 7. BRIGHT	73.15% / 41.93%	73.70% / 41.00%	75.60% / 40.31%
├─ 5. CLINC150+OOS / 6. Home Appliances	93.94% / 90.00%	93.94% / 90.00%	93.94% / 90.00%
└─ 8. Amazon ESCI / 9. PhishNChips / 10. When2Call	73.89% / 85.00% / 75.00%	73.89% / 85.00% / 75.00%	75.79% / 85.00% / 72.92%

Table 7: Disk footprint, peak VRAM allocation, and Tesla T4 (16 GiB) latency across precision tiers.

Precision Format	Artifact Size	Peak VRAM (4K–16K ctx)	T4 Latency (293 / 1,068 tok)	Target Hardware
FastBrain-9B-Q4_K_M.gguf	6.73 GB	6.78 – 10.30 GiB	343.0 ms / 1,154.3 ms	8–16 GB GPU (RTX 3060/4060, Tesla T4), Apple Silicon (8 GB+)
PyTorch load_in_4bit=True (NF4 + fp16 head)	~5.5 GB (mem)	6.20 – 6.80 GiB	—	8–16 GB CUDA GPU (Colab T4, consumer GPUs)
FastBrain-9B-Q8_0.gguf	9.66 GB	9.03 – 12.55 GiB	349.3 ms / 1,149.8 ms	12–16 GB GPU (Tesla T4, RTX 4070 Ti), Apple Silicon (16 GB+)
PyTorch load_in_8bit=True (INT8 + bf16 head)	~9.6 GB (mem)	~11.20 GiB	—	12–16 GB CUDA GPU
Native bfloat16 (safetensors / BF16.gguf)	18.82 / 18.16 GB	20.30 – 24.70 GiB	—	≥24 GB GPU (NVIDIA L4, A100, H100, RTX 3090/4090, PRO 6000)

---

5. Reproducible Inference and Deployment

5.1 Python / Transformers Runtime (bfloat16, 8-bit, or 4-bit NF4)

TypeScript/FastBrain-9B implements the joint_schema_model interface and is API-compatible with Cloudflare/clef-flash. Passing load_in_4bit=True quantizes only the 9.3B backbone via bitsandbytes NF4 while preserving the 116M JointSchemaHead in floating point.

import sys
import torch
from huggingface_hub import snapshot_download

model_dir = snapshot_download("TypeScript/FastBrain-9B", ignore_patterns=["*.gguf"])
sys.path.insert(0, model_dir)
import joint_schema_model as clef

# Native bfloat16 (>=24 GB VRAM):
model, processor = clef.load_release_model(model_dir, device="cuda", dtype=torch.bfloat16)

# Mixed-precision 4-bit NF4 (8-16 GB VRAM, e.g., Tesla T4):
# model, processor = clef.load_release_model(model_dir, device="cuda", dtype=torch.float16, load_in_4bit=True)

response = clef.systemone(model, processor, {
    "model": "TypeScript/FastBrain-9B",
    "state": {
        "query": "Can you check my account balance?",
        "tools": ["get_balance", "transfer_funds"],
    },
    "questions": {
        "next_action": {
            "type": "choice",
            "instructions": "Select the appropriate assistant action.",
            "criteria": {
                "call_balance": "Invoke <tool_call>{\"name\": \"get_balance\", \"arguments\": {}}</tool_call>",
                "request_auth": "Ask the user to provide an account identifier or authentication token.",
                "abstain": "Decline because personal banking records are out of scope.",
            },
        },
    },
})
print(response)

5.2 llama.cpp GGUF Server (/v1/systemone)

Runtime Requirements for ClefModel GGUF Execution

1. Use llama.cpp build b11430 or later, which implements the LLM_ARCH_CLEF computation graph (dec.* tensors) and the /v1/systemone HTTP endpoint. Earlier builds and standard generative wrappers do not execute the cross-attention decision head.
2. Set physical micro-batch size equal to logical batch size (-b 16384 -ub 16384 or -b 4096 -ub 4096) together with --no-cache-prompt. ClefModel pools <|question_*|> and <|option_*|> spans within a single physical ubatch; leaving -ub at its default (512) splits prompts longer than 512 tokens across micro-batches, and prefix caching can bypass marker positions.

# 1. Download prebuilt llama.cpp (b11430+) and GGUF checkpoint
curl -sL https://github.com/ggml-org/llama.cpp/releases/download/b11430/llama-b11430-bin-ubuntu-cuda-13.4-x64.tar.gz | tar -xz
pip install -U "huggingface_hub[hf_transfer]"
HF_HUB_ENABLE_HF_TRANSFER=1 hf download TypeScript/FastBrain-9B FastBrain-9B-Q8_0.gguf --local-dir .

# 2. Start llama-server (add --mmproj mmproj-FastBrain-9B-Q8_0.gguf for image inputs)
export LD_LIBRARY_PATH="$(pwd)/llama-b11430:/usr/lib64-nvidia:/usr/local/cuda/lib64:${LD_LIBRARY_PATH}"
./llama-b11430/llama-server \
  -m ./FastBrain-9B-Q8_0.gguf \
  -ngl 99 -c 16384 -b 16384 -ub 16384 -np 1 -fa on --no-cache-prompt \
  --port 8080

Querying /v1/systemone evaluates multiple heterogeneous primitives (choice + noul) in a single prefill pass:

curl http://127.0.0.1:8080/v1/systemone \
  -H "Content-Type: application/json" \
  -d '{
    "model": "FastBrain-9B",
    "state": {
      "ticket_id": "ORD-99281",
      "customer_message": "I was charged twice for my subscription last week and nobody has replied to my ticket."
    },
    "questions": {
      "department": {
        "type": "choice",
        "instructions": "Assign this ticket to the responsible queue.",
        "criteria": {
          "billing": "Payments, refunds, duplicate charges, or invoices",
          "engineering": "Software defects, crashes, or service outages",
          "identity": "Password reset or multi-factor authentication issues"
        }
      },
      "is_urgent": {
        "type": "noul",
        "instructions": "Does the message report an unresolved financial loss or time-critical failure?"
      }
    }
  }'

5.3 SGLang Server (JointSchemaPooler)

docker run --gpus all --shm-size 32g -p 30000:30000 \
  -v ~/.cache/huggingface:/root/.cache/huggingface --ipc=host \
  lmsysorg/sglang:dev-clef \
  sglang serve --model-path TypeScript/FastBrain-9B --host 0.0.0.0 --port 30000

---

6. Repository Manifest and Citation

Table 8: Repository file specification.

Path	Size	Contents
model-00001-of-00004.safetensors .. 00004	18.59 GB	bfloat16 hybrid backbone tensors with all adapted projections merged in-place
joint_head.safetensors, joint_head_config.json	232 MB	116M-parameter JointSchemaHead weights (122 tensors) and architectural config
FastBrain-9B-Q4_K_M.gguf	6.73 GB	Mixed-precision 4-bit GGUF (dec.*, ssm_*, and attn_* retained in Q8_0)
FastBrain-9B-Q8_0.gguf / FastBrain-9B-BF16.gguf	9.66 / 18.16 GB	8-bit and bfloat16 GGUF builds for llama-server (b11430+)
mmproj-FastBrain-9B-Q8_0.gguf / BF16.gguf	624 / 922 MB	Vision projector checkpoints for multimodal state inputs
joint_schema_model.py	25 KB	Reference ClefModel, vectorized _forward_fast_cuda head, and systemone() runtime
eval_submissions/typescript_fastbrain-9b*.json	1.80 MB	Verifiable 2,000-decision zero-shot and prior-fitted traces for LocalLLaMA/typed-decisions
eval_submissions/TypeScript__FastBrain-9B*.jsonl	110 KB	Verifiable 661-call binary and 7-way category traces for safeagents-judge-bench (v2)
.eval_results/*.yaml	2.3 KB	Hugging Face Hub machine-readable evaluation records
README.md, README_zh.md, assets/*.png	1.28 MB	English and Chinese technical documentation and evaluation figures

Citation

@misc{fastbrain9b2026,
  title        = {FastBrain-9B: A Single-Pass Non-Autoregressive Model for Calibrated Structured Decisions},
  author       = {TypeScript},
  year         = {2026},
  howpublished = {\url{https://huggingface.co/TypeScript/FastBrain-9B}},
  note         = {Apache-2.0 License}
}