---
license: apache-2.0
language:
- en
- zh
library_name: transformers
pipeline_tag: text-generation
base_model: Cloudflare/clef-flash
datasets:
- TypeScript/FastBrain-Instruct
tags:
- fastbrain
- clef
- gguf
- llama-cpp
- sglang
- systemone
- decision-model
- image-text-to-typed-output
- decision-index
- tool-calling
- function-calling
- intent-classification
- structured-decision
---
# FastBrain-9B
**A Single-Pass Non-Autoregressive Model for Calibrated Structured Decisions**
[-3b82f6?style=flat-square)](#2-model-architecture-and-training-objective)
[-10b981?style=flat-square)](#2-model-architecture-and-training-objective)
[-ec4899?style=flat-square)](#31-agent-tool-call-guardrail-evaluation-safeagents-judge-bench)
[_%C2%B7_50ms-8b5cf6?style=flat-square)](#32-multi-workflow-enterprise-decisions-localllamatyped-decisions)
[-f59e0b?style=flat-square)](#33-multi-domain-decision-suite-cloudflare-decision-index-v021)
[](./LICENSE)
**English** | **[简体中文](./README_zh.md)**
---
## Abstract
Autoregressive language models incur sequential decoding overhead and token-level probability miscalibration when deployed on closed-schema classification, tool routing, and safety verification tasks. We present **FastBrain-9B**, a 9.41B-parameter non-autoregressive structured decision model built on a 32-layer hybrid Gated DeltaNet and full-attention backbone (`Qwen3.5-9B`, initialized from `Cloudflare/clef-flash`) coupled with a 116M-parameter cross-attention decision head (`JointSchemaHead`). Given an arbitrary JSON or multimodal state and a set of typed decision specifications (`choice`, binary `noul`, and ordinal `score`), FastBrain-9B evaluates all questions and candidate options simultaneously in a single prefill pass (`output_tokens = 0`). We adapt 104 linear-attention and self-attention projections jointly with the decision head under a composite ordinal-KL, cumulative distribution (ranked probability score), and expected-value regression objective, and merge all parameter updates directly into the base tensors. Across three public evaluation suites encompassing 109 model entries, FastBrain-9B ranks **1st of 8** on the `safeagents-judge-bench` (v2) 7-way misuse classification track (`0.751` macro-F1, `0.858` accuracy; +18.8 percentage points over 27.4B `Cloudflare/clef`) while intercepting `82/82` unsafe agent trajectories (`0.985` AUROC); ranks **7th of 43** zero-shot overall (**5th of 36** open-weight, **2nd of 19** in the ≤9B open-weight class) on `LocalLLaMA/typed-decisions` at `50 ms` median latency (66.7× faster than 31B `Quyet-1.0-Large`), and **12th-tied (14th listed of 51)** under prior-only calibration without weight updates (`0.797` accuracy, `0.0816` KL); and achieves an **80.55%** 10-task macro mean on Cloudflare Decision Index (v0.2.1), ranking **2nd of 7** overall, **1st on 10 of 10** tasks among ≤9B models, and outperforming 27.4B `Cloudflare/clef` on 5 of 10 tasks.
---
## 1. Overview and Motivation
Production agentic pipelines invoke structured decisions at high frequency: selecting among candidate tool schemas, verifying whether an execution trace violates sandbox policy, routing customer or security tickets, and scoring rubric criteria. Solving these tasks via autoregressive token generation introduces two structural inefficiencies. First, latency scales with output sequence length and formatting tokens rather than decision complexity. Second, surface-form token probabilities fragment probability mass across equivalent verbalizations, degrading calibration metrics such as Kullback-Leibler (KL) divergence and Brier score.
Single-pass decision models (`ClefModel` / System-1 architecture) address both issues by appending delimiter-bounded question and option spans directly to the input context and reading out option logits from final hidden states in one prefill pass. Existing 9B-scale checkpoints, however, trail 27B–35B models substantially on fine-grained multi-class discrimination, ordinal rubric estimation, and open-set abstention (e.g., `Cloudflare/clef-flash` scores `46.2%` macro-F1 on `safeagents-judge-bench` v2 and `66.77%` on `CLINC150+OOS`). FastBrain-9B closes and reverses much of this scaling gap through full-depth projection adaptation, ordinal distribution distillation, and compile-time schema-topology calibration, while preserving sub-100 ms single-GPU latency and lossless 8-bit / high-retention 4-bit GGUF quantization.
---
## 2. Model Architecture and Training Objective
### 2.1 Hybrid State-Space and Attention Backbone
FastBrain-9B uses a 32-layer hybrid sequence backbone (`9.30B` parameters, `bfloat16`, maximum context length `262,144` tokens) interleaving 24 linear-attention Gated DeltaNet blocks with 8 full-attention blocks. During training, 104 projection matrices across all 32 blocks—specifically `in_proj_qkv`, `in_proj_z`, and `out_proj` in linear-attention layers, alongside `q_proj`, `k_proj`, `v_proj`, and `o_proj` in full-attention layers—are updated jointly with the decision head. For distribution, all low-rank and projection updates are folded in-place into `model-00001-of-00004.safetensors` through `model-00004-of-00004.safetensors`, requiring zero adapter overhead at inference time.
### 2.2 Joint Schema Decision Head and Vectorized CUDA Kernel
The `JointSchemaHead` (`116M` parameters, stored in `joint_head.safetensors`) pools token representations bounded by `<|question_start|>`/`<|question_end|>` and `<|option_start|>`/`<|option_end|>` markers from the final backbone hidden states \\(H \in \mathbb{R}^{T \times d}\\), applies cross-attention over question-option pairs, and projects directly to scalar logits for three decision primitives:
1. **`choice`** (*K*-way categorical distribution): \\(\hat{p} = \operatorname{softmax}(z / \tau) \in \Delta^{K-1}\\).
2. **`noul`** (binary verification): \\(\hat{p}(\text{true}) = \sigma(z_{\text{true}} - z_{\text{false}})\\).
3. **`score`** (ordinal rubric over levels \\(v_1 < \dots < v_K\\) ): categorical probabilities \\(\hat{p} \in \Delta^{K-1}\\) together with expected value \\(\mathbb{E}[\hat{y}] = \sum_{k=1}^K \hat{p}_k v_k\\).
In `joint_schema_model.py`, the reference Python span-extraction loop and padded SDPA mask are replaced on CUDA devices by `_forward_fast_cuda`: span means are computed in \\(\mathcal{O}(1)\\) kernel launches via prefix sums (`torch.cumsum`) over the schema suffix, and cross-attention is executed via packed variable-length FlashAttention (`torch.ops.aten._flash_attention_forward` parameterized by `cu_seqlens`). On NVIDIA RTX PRO 6000 Blackwell, this reduces isolated decision-head latency from `3.27 ms` to `1.72 ms` (**1.90× speedup**) with exact numerical equivalence (`max_logit_diff = 0.00000`).
### 2.3 Ordinal-KL Distillation and Schema-Topology Calibration
Standard cross-entropy or flat KL divergence treats ordinal rubric levels as unordered classes, penalizing adjacent-level errors identically to extreme errors. FastBrain-9B is trained on `TypeScript/FastBrain-Instruct` under a composite loss combining soft-target KL divergence, discrete ranked probability score (squared \\(L_2\\) distance between predicted and target cumulative distribution functions \\(\hat{F}_k = \sum_{j \le k} \hat{p}_j\\) ), and expected-score mean squared error on `score` primitives:
$$
\mathcal{L}(\theta) = \mathrm{KL}(p \,\|\, \hat{p}_\theta) + \lambda_{\mathrm{ord}} \sum_{k=1}^{K-1} \bigl(\hat{F}_{\theta,k} - F_k\bigr)^2 + \mathbb{I}[\text{type}=\text{score}] \cdot \lambda_{\mathrm{mse}} \bigl(\mathbb{E}[\hat{y}_\theta] - \mathbb{E}[y]\bigr)^2.
$$
At inference time, `joint_schema_model.py` optionally applies compile-time schema-topology calibration (`apply_schema_aware_calibration`), which inspects candidate option structures (e.g., `` invocations, clarification prompts, and open-set abstention clauses) to adjust decision margins against systematic tool over-triggering and out-of-scope false acceptance.
---
## 3. Empirical Evaluation
**Table 1: Summary of official public benchmark standings (verified across 109 model entries as of November 2026).**
| Benchmark & Evaluation Track | Evaluated Entries | FastBrain-9B Metric | Official Standing | Reference Baselines (`Params`, Metric) |
| :--- | :---: | :---: | :---: | :--- |
| [`safeagents-judge-bench` (v2)](https://huggingface.co/datasets/harness-hub/safeagents-judge-bench/tree/v2) — 7-Way Category ([PR #1](https://huggingface.co/TypeScript/FastBrain-9B/discussions/1)) | 8 decision models (+2 baselines) | **0.751** Macro-F1 / **0.858** Acc | **1 / 8** | `Cloudflare/clef` (27.4B, `0.563`), `vllm-sr/d3-mini` (1.8B, `0.512`), `Cloudflare/clef-flash` (9.4B, `0.462`), `clef-omni` (30B, `0.437`) |
| [`safeagents-judge-bench` (v2)](https://huggingface.co/datasets/harness-hub/safeagents-judge-bench/tree/v2) — Binary Guardrail ([PR #1](https://huggingface.co/TypeScript/FastBrain-9B/discussions/1)) | 8 decision models (+2 baselines) | **0.985** AUROC / **82/82** Traj.
(`0.814` raw; **0.938** calibrated) | **3 / 8** AUROC
**1 / 8** (tied) Traj. | `Cloudflare/clef` (27.4B, `0.808` raw bal. acc), `clef-flash` (`0.967` AUROC, `81/82`), `clef-omni` (`0.946` AUROC, `81/82`), `d1-3B` (`68/82`) |
| [`safeagents-judge-bench` (v1)](https://huggingface.co/datasets/harness-hub/safeagents-judge-bench/tree/v1) — Binary & Category ([PR #1](https://huggingface.co/datasets/harness-hub/safeagents-judge-bench/discussions/1)) | 9 decision models (+2 baselines) | **0.974** Bal. Acc / **0.850** Cat. F1 | **1 / 9** (both tracks) | `Cloudflare/clef-omni` (30B, `0.845` bal. acc / `0.327` F1), `Cloudflare/clef` (27.4B, `0.718` / `0.425`), `clef-flash` (`0.721` / `0.345`) |
| [`LocalLLaMA/typed-decisions`](https://huggingface.co/datasets/LocalLLaMA/typed-decisions) — Zero-Shot ([PR #58](https://huggingface.co/datasets/LocalLLaMA/typed-decisions/discussions/58)) | 43 total (36 open, 19 ≤9B open) | **0.7525** Acc (`0.7625` tied-top)
`0.1508` KL · `0.0795` Brier · **50 ms** | **7 / 43** overall
**5 / 36** open · **2 / 19** ≤9B | `vllm-sr/d3` (27B, `0.749`), `Elephant 1` (9B, `0.748`, `352 ms`), `Darwin-27B` (`0.743`), `Cloudflare/clef` (27B, `0.728`), `clef-flash` (`0.710`) |
| [`LocalLLaMA/typed-decisions`](https://huggingface.co/datasets/LocalLLaMA/typed-decisions) — Fitted on `train` ([PR #57](https://huggingface.co/datasets/LocalLLaMA/typed-decisions/discussions/57)) | 51 specialist & calibrated models | **0.7970** Acc (`0.8070` tied-top)
**0.0816** KL · **0.0437** Brier · **50 ms** | **12-tied** (14 / 51 listed)
**1st** frozen-weight ≤9B & KL | Frozen weights (`no weights changed`); surpasses `Rev-gen1-midi` (25.6B, `0.7945`), `OpenDecider-small-td` (4B LoRA, `0.792`), `OpenDecider-medium-td` (30B LoRA, `0.788`) |
| Cloudflare Decision Index (`v0.2.1`) — 10-Task Suite | 7 models (0.4B–27.4B) | **80.55%** Macro Mean | **2 / 7** overall
**1st** on **10 / 10** (≤9B) | `Jev` (27B, `80.32%`), `Kev 9B` (`76.51%`), `Clef-flash` (9.4B, `74.96%`), `DiffGemma` (26B, `69.14%`); exceeds `27.4B Clef` on 5 / 10 tasks |
---
### 3.1 Agent Tool-Call Guardrail Evaluation (`safeagents-judge-bench`)
[`harness-hub/safeagents-judge-bench`](https://huggingface.co/datasets/harness-hub/safeagents-judge-bench/tree/v2) evaluates whether a System-1 decision model can audit suspicious tool calls flagged by a static scanner (`atif-scan`) during real coding-agent executions on Terminal-Bench 2.1, OpenThoughts-TBLite, and Terminal-Bench 4. Version 2 (`v2`) comprises *N* = 661 tool calls across 55 tasks (`536` malicious `block` calls spanning `82` misbehaving trajectories and `125` legitimate `allow` calls). Each call is issued as a single zero-shot System-1 request containing one binary `noul` question (`block`) and one 7-way `choice` question (`legit`, `eval_gaming`, `harness_introspection`, `credential_misuse`, `exfiltration`, `sandbox_escape`, `other_overreach`). All v2 results in Table 2 were executed independently on NVIDIA A100-SXM4-80GB via Hugging Face Jobs ([Merged PR #1](https://huggingface.co/TypeScript/FastBrain-9B/discussions/1)).
On the 7-way category classification track, FastBrain-9B achieves `0.751` macro-F1 and `0.858` accuracy (`567/661`), outperforming the 27.4B `Cloudflare/clef` model (`0.563` macro-F1, `0.737` accuracy) by `+18.8` percentage points in macro-F1 and `+12.1` points in accuracy, and improving upon its 9.4B base model `Cloudflare/clef-flash` (`0.462` macro-F1, `0.576` accuracy) by `+28.9` and `+28.2` points, respectively. Per-category F1 reaches `0.945` on `harness_introspection` (*n* = 272, recall `0.978`), `0.828` on `eval_gaming` (*n* = 245, recall `0.853`), and `0.761` on `legit` (*n* = 125, precision `0.975`).
On the binary guardrail track, FastBrain-9B attains `0.985` AUROC and `0.996` malicious-call recall (`534/536`), intercepting all `82/82` unsafe trajectories (`median_bad_calls_before_catch = 0.0`), with `1.000` recall on `harness_introspection` (`272/272`), `credential_misuse` (`15/15`), and `sandbox_escape` (`4/4`). Under the uncalibrated default decision rule \\(P(\text{block}) \ge 0.50\\), balanced accuracy is `0.814` (false-block rate `0.368` across the 125 `allow` calls, compared with `0.808` balanced accuracy and `0.384` false-block rate for 27.4B `Cloudflare/clef`). Because rank-ordering discrimination is preserved (`0.985` AUROC), setting the operating threshold to \\(P(\text{block}) \ge 0.961\\) yields `0.938` balanced accuracy at a `0.056` (`7/125`) false-block rate while maintaining `82/82` trajectory interception. Table 2 also includes the `v1` release (*N* = 562 calls, `78` trajectories), where FastBrain-9B ranked 1st of 9 on both tracks.
**Table 2: Zero-shot evaluation on `harness-hub/safeagents-judge-bench` (`v2`, *N* = 661 calls, 82 unsafe trajectories; and `v1` reference, *N* = 562).**
| Model | Params | Cat. Macro-F1 ↑ | Cat. Acc ↑ | AUROC ↑ | Bal. Acc ↑ | Recall ↑ | False Block ↓ | Traj. Caught ↑ | p50 Latency (Hardware) |
| :--- | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
| **TypeScript/FastBrain-9B** (`v2`, default *t* = 0.50) | 9.4B | **0.751** | **0.858** | 0.985 | 0.814 | 0.996 | 0.368 | **82 / 82** | 327 ms (A100) · 104 ms (G4) |
| **TypeScript/FastBrain-9B** (`v2`, calibrated *t* = 0.961) | 9.4B | **0.751** | **0.858** | 0.985 | **0.938** | 0.931 | 0.056 | **82 / 82** | 327 ms (A100) · 104 ms (G4) |
| **TypeScript/FastBrain-9B** (`v1` official, *N* = 562) | 9.4B | **0.850** | **0.923** | 0.983 | **0.974** | **1.000** | 0.051 | **78 / 78** | 104 ms (RTX PRO 6000) |
| Cloudflare/clef | 27.4B | 0.563 | 0.737 | **0.993** | 0.808 | **1.000** | 0.384 | **82 / 82** | 636 ms (A100) |
| vllm-sr/d3-mini | 1.8B | 0.512 | 0.652 | 0.986 | 0.856 | **1.000** | 0.288 | **82 / 82** | 168 ms (A100) |
| Cloudflare/clef-flash (base) | 9.4B | 0.462 | 0.576 | 0.967 | 0.893 | 0.978 | 0.192 | 81 / 82 | 326 ms (A100) |
| Cloudflare/clef-omni | 30B-A3B | 0.437 | 0.564 | 0.946 | 0.877 | 0.914 | 0.160 | 81 / 82 | 285 ms (A100) |
| knowledgator/gliclass-v4-edge | 0.4B | 0.373 | 0.528 | 0.753 | 0.676 | 0.696 | 0.344 | 68 / 82 | 101 ms (L4) |
| fastino/GLiNER2.5-Decide | 0.5B | 0.355 | 0.620 | 0.517 | 0.576 | 0.151 | **0.000** | 17 / 82 | 117 ms (M5 Pro CPU) |
| LiquidAI/d1-3B | 3.0B | 0.328 | 0.457 | 0.946 | 0.836 | 0.681 | 0.008 | 68 / 82 | **35 ms** (A100) |
---
### 3.2 Multi-Workflow Enterprise Decisions (`LocalLLaMA/typed-decisions`)
[`LocalLLaMA/typed-decisions`](https://huggingface.co/datasets/LocalLLaMA/typed-decisions) tests multi-primitive structured judgment across four enterprise workflows (`agent_trace_observability`, `customer_service`, `invoice_processing`, and `security_incidents`), comprising 400 held-out test cases and 2,000 individual decisions (`noul`, `choice`, and `score`) scored against teacher-panel soft label distributions. Two scoring conventions exist in the benchmark repository: `bench/score_with_jobs.py` resolves exact ties in the gold distribution by selecting the first index (`g.index(max(g))`), whereas `scripts/score_model.py` credits any tied maximum across the 20 tied test items; KL divergence and Brier score are identical under both scripts. We report both accuracy figures (`strict / tied-top`) alongside prediction traces in [`eval_submissions/`](./eval_submissions/).
On the official zero-shot leaderboard (*N* = 43 models), FastBrain-9B scores `0.7525` (`0.7625` tied-top) accuracy, `0.1508` KL divergence, `0.0795` Brier score, and `0.085` ECE at `50 ms` p50 latency (RTX PRO 6000 via SGLang). It places **7th of 43** overall, **5th of 36** among open-weight models, **2nd of 19** among ≤9B open-weight models, and **1st** among all `Clef`-architecture models, improving over 9.4B `Cloudflare/clef-flash` (`0.7100` accuracy, `0.2093` KL) by `+4.25` percentage points and over 27.4B `Cloudflare/clef` (`0.7275` accuracy, `0.1907` KL) by `+2.50` points. Across all 28 models exceeding `0.700` zero-shot accuracy, FastBrain-9B exhibits the lowest p50 latency (`50 ms`, compared with `118 ms` for 9B `Amos 1 medium`, `357 ms` for 27B `Amos 1 large`, `526 ms` for `meraGPT Decider 1`, and `3,335 ms` for 31B `Quyet-1.0-Large`). Evaluating the uncalibrated ordinal-KL distilled head directly yields `0.7735` raw accuracy (`0.1039` KL, `0.0582` Brier).
On the separate `train`-fitted leaderboard (*N* = 51 models), fitting only scalar per-question log-priors and temperature parameters on the 1,200 `train` cases—with all backbone and head weights frozen (`no weights changed`)—achieves `0.7970` (`0.8070` tied-top) accuracy, `0.0816` KL divergence, and `0.0437` Brier score, ranking **12th-tied (14th listed of 51)** overall and **1st in KL calibration** among frozen-weight models.
**Table 3: Performance on `LocalLLaMA/typed-decisions` (*N* = 400 cases, 2,000 decisions). Values separated by `/` denote `score_with_jobs.py` / `score_model.py` tie handling.**
| Model | Params | Protocol | Overall Acc ↑ | KL ↓ | Brier ↓ | `noul` ↑ | `choice` ↑ | `score` ↑ | Agent ↑ | Cust. Svc ↑ | Invoice ↑ | Security ↑ | p50 Latency |
| :--- | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
| **FastBrain-9B** (prior-fitted, frozen weights) | 9.4B | Fitted | **79.70% / 80.70%** | **0.0816** | **0.0437** | **86.7% / 88.0%** | **76.5% / 77.2%** | **76.9% / 77.9%** | **75.4% / 76.6%** | **82.0% / 83.4%** | **83.4% / 83.6%** | **78.0% / 79.2%** | **50 ms** |
| **FastBrain-9B** (ordinal-KL distilled head) | 9.4B | Zero-shot | 77.35% | 0.1039 | 0.0582 | 84.8% | 74.8% | 73.8% | 71.2% | 80.4% | 82.6% | 75.2% | **50 ms** |
| **FastBrain-9B** (official PR #58) | 9.4B | Zero-shot | 75.25% / 76.25% | 0.1508 | 0.0795 | 83.2% / 84.3% | 74.7% / 76.0% | 69.8% / 70.4% | 68.2% / 69.8% | 77.6% / 78.8% | 82.0% | 73.2% / 74.4% | **50 ms** |
| Amos 1 (medium) | 9.0B | Zero-shot | 76.45% | 0.1510 | 0.0800 | — | — | — | — | — | — | — | 118 ms |
| Amos 1 (large) | 27.0B | Zero-shot | 76.10% | 0.1260 | 0.0690 | 83.2% | 73.7% | 72.6% | 67.4% | 78.8% | 83.0% | 75.2% | 357 ms |
| pplx-decider-v1.1-27b | 27.0B | Zero-shot | 75.30% | 0.4410 | 0.1600 | 85.2% | 73.7% | 69.1% | 66.4% | 80.2% | 80.2% | 74.4% | — |
| vllm-sr/d3 | 27.0B | Zero-shot | 74.90% | 0.4090 | 0.1460 | 84.7% | 68.5% | 72.4% | 67.2% | 76.6% | 82.8% | 73.0% | — |
| Baseten Elephant 1 | 9.0B | Zero-shot | 74.80% | 0.1340 | 0.0690 | — | — | — | — | — | — | — | 352 ms |
| Darwin-27B-ZTC | 27.0B | Zero-shot | 74.30% | 0.2040 | 0.0970 | — | — | — | — | — | — | — | — |
| Cloudflare/clef | 27.4B | Zero-shot | 72.75% | 0.1907 | 0.1032 | 84.5% | 64.8% | 69.9% | 63.8% | 74.2% | 78.6% | 74.4% | — |
| vllm-sr/d3-flash | 9.0B | Zero-shot | 71.70% | 0.3220 | 0.1310 | — | — | — | — | — | — | — | — |
| Cloudflare/clef-flash (base) | 9.4B | Zero-shot | 71.00% | 0.2093 | 0.1105 | 81.5% | 71.3% | 62.9% | 62.0% | 76.0% | 76.4% | 69.6% | — |
| Cloudflare/clef-omni | 35B-A3B | Zero-shot | 67.30% | 0.2469 | 0.1233 | 82.7% | 63.5% | 58.6% | 55.2% | 73.6% | 74.2% | 66.2% | — |
---
### 3.3 Multi-Domain Decision Suite (Cloudflare Decision Index `v0.2.1`)
On the 10-benchmark **Cloudflare Decision Index (`v0.2.1`)** suite (*N* = 22,427 evaluation requests spanning AST tool calling, 20-candidate tool reranking, stateful API invocation, fine-grained intent classification, out-of-scope rejection, reasoning-intensive retrieval, e-commerce relevance, phishing detection, and tool-use governance), FastBrain-9B attains an **80.55%** unweighted macro mean (Table 4, Figures 1–3).
Across the ≤9B parameter tier (`Clef-flash` 9.4B, `Kev 9B`, and `Laya` 0.4B), FastBrain-9B ranks **1st on all 10 benchmarks**, improving the 10-task macro mean over `Clef-flash` (`74.96%`) by `+5.59` percentage points and over `Kev 9B` (`76.51%`) by `+4.04` points. The largest gains occur on calibration-sensitive abstention and governance tasks: `+31.67` points on `CLINC150+OOS` (`98.44%` vs. `66.77%`), `+15.55` points on `When2Call` (`81.13%` vs. `65.58%`), and `+6.80` points on `PhishNChips` (`78.75%` vs. `71.95%`). Across all model scales up to 27.4B, FastBrain-9B ranks **2nd of 7** overall, surpassing `27B Jev` (`80.32%`) and `26B-A4B DiffusionGemma Jev` (`69.14%`), and exceeding `27.4B Clef` (`81.81%`) on 5 of 10 individual benchmarks (`BFCL`, `API-Bank`, `CLINC150+OOS`, `Home Appliances`, and `When2Call`).
*Figure 1. Aggregate 10-benchmark macro mean versus parameter count on Cloudflare Decision Index (v0.2.1, 22,427 requests).*
*Figure 2. Per-benchmark performance matrix across 0.4B–27.4B decision models on Cloudflare Decision Index (v0.2.1).*
*Figure 3. Confidence calibration, open-set out-of-scope rejection (`CLINC150+OOS`), and abstention margin analysis (`When2Call`).*
**Table 4: Per-benchmark results on Cloudflare Decision Index (`v0.2.1`, 10 tasks, *N* = 22,427). Bold denotes the highest score across all parameter scales; underline denotes best in the ≤9B class.**
| Benchmark (Metric) | **FastBrain-9B** (Ours) | Clef-flash (Base) | Kev 9B | Clef | Jev | DiffGemma | Laya |
| :--- | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
| Total Parameters | 9.41B | 9.41B | 9.00B | 27.36B | 27.00B | 25.8B (4B act.) | 0.42B |
| 1. BFCL (Case Exact Acc) | **98.76%** | **98.76%** | 97.58% | 98.47% | 98.58% | 97.64% | 95.87% |
| 2. ToolRet (nDCG@10) | 67.26% | 67.26% | 65.72% | **68.18%** | 65.50% | 45.28% | 51.31% |
| 3. API-Bank (Accuracy) | **93.70%** | 93.11% | 89.76% | 91.93% | 90.35% | 85.24% | 86.22% |
| 4. BANKING77 (Macro-F1) | 93.77% | 93.69% | 91.77% | **94.07%** | 93.58% | 93.80% | 62.66% |
| 5. CLINC150+OOS (Macro-F1) | **98.44%** | 66.77% | 96.65% | 97.43% | 96.44% | 94.64% | 73.79% |
| 6. Home Appliances (Case Exact) | **97.73%** | 96.59% | 88.64% | 96.59% | 88.64% | 68.18% | 23.86% |
| 7. BRIGHT (nDCG@10) | 38.58% | 38.49% | 38.53% | **48.03%** | 43.37% | 18.69% | 25.00% |
| 8. Amazon ESCI (Macro-F1) | 57.40% | 57.39% | 53.75% | **59.30%** | 56.60% | 56.83% | 52.84% |
| 9. PhishNChips (Accuracy) | 78.75% | 71.95% | 72.10% | **91.70%** | 88.20% | 55.65% | 60.45% |
| 10. When2Call (Macro-F1) | 81.13% | 65.58% | 70.64% | 72.37% | **81.91%** | 75.44% | 48.92% |
| **10-Task Macro Mean** | 80.55% | 74.96% | 76.51% | **81.81%** | 80.32% | 69.14% | 58.09% |
---
## 4. Systems Performance and Post-Training Quantization
### 4.1 Runtime Comparison Across Inference Engines
Table 5 compares end-to-end latency, prefill throughput, and memory footprint across three serving backends—SGLang (`lmsysorg/sglang:dev-clef` with `JointSchemaPooler`), `llama.cpp` (`b11430` `ClefModel` graph in `Q8_0` and `Q4_K_M`), and native PyTorch (`joint_schema_model.py`)—benchmarking identical requests on a single NVIDIA RTX PRO 6000 Blackwell Server Edition GPU. Because `output_tokens = 0` for all requests, throughput is governed purely by prefill compute and span-pooling efficiency, reaching `19,319 tok/s` (`131.2` decisions/s) at batch size *B* = 4 in SGLang.
**Table 5: End-to-end latency and prefill throughput on NVIDIA RTX PRO 6000 Blackwell Server Edition (G4).**
| Workload / System Metric | SGLang `dev-clef` (BF16) | `llama.cpp b11430` (`Q8_0`) | `llama.cpp b11430` (`Q4_K_M`) | PyTorch Native (BF16) |
| :--- | :---: | :---: | :---: | :---: |
| Cold-Start Load Time | 11.11 s | 2.56 s | **2.00 s** | 11.11 s |
| Peak GPU Memory (16K context) | ~20.5 GiB | 12.55 GiB | **10.30 GiB** | 20.3–24.7 GiB |
| Compatible with 16 GiB T4 GPU | No (BF16 TP=1) | Yes (`9.03 GiB` at 4K) | **Yes (`6.78 GiB` at 4K)** | Yes (`6.8 GiB` via NF4) |
| Top-1 Agreement vs. PyTorch BF16 | **100.00%** | 99.86% (1,471 / 1,473) | 98.78% (1,455 / 1,473) | 100.00% (reference) |
| 1. Short Intent Routing (293 tok, *K* = 2) | **30.01 ms** (9,763 tok/s) | 32.82 ms (8,929 tok/s) | 31.69 ms (9,245 tok/s) | 43.23 ms (6,778 tok/s) |
| 2. Typed Decisions (736 tok, *K* = 5) | **50.29 ms** (14,635 tok/s) | 62.40 ms (11,795 tok/s) | 61.80 ms (11,909 tok/s) | 63.58 ms (11,576 tok/s) |
| 3. When2Call Governance (1,068 tok, *K* = 1) | **66.06 ms** (16,167 tok/s) | 87.78 ms (12,167 tok/s) | 89.34 ms (11,955 tok/s) | 77.77 ms (13,733 tok/s) |
| 4. PhishNChips Security (1,303 tok, *K* = 9) | **75.86 ms** (17,176 tok/s) | 108.55 ms (12,004 tok/s) | 109.97 ms (11,849 tok/s) | 90.32 ms (14,426 tok/s) |
| 5. SafeAgents Judge (2,217 tok, *K* = 2) | **125.85 ms** (17,616 tok/s) | 181.70 ms (12,201 tok/s) | 183.50 ms (12,082 tok/s) | 138.06 ms (16,058 tok/s) |
| Batch *B* = 4 Typed Decisions (2,944 tok, 20 dec) | **152.39 ms** (19,319 tok/s) | 248.50 ms (11,847 tok/s) | 245.20 ms (12,006 tok/s) | 175.85 ms (16,742 tok/s) |
### 4.2 Mixed-Precision GGUF Quantization Fidelity
Uniform 4-bit quantization degrades cross-attention span readout and linear-attention recurrent state precision if applied indiscriminately to decision-head projections. In our `FastBrain-9B-Q4_K_M.gguf` (`6.73 GB`) build, feed-forward blocks are quantized to 4-bit K-quants while all `dec.*` decision-head tensors, `ssm_*` Gated DeltaNet parameters, and `attn_*` projections remain at `Q8_0` precision. Evaluated across a stratified 10-task subset (`203` multi-question requests, `1,473` scored schema questions; Table 6), `Q8_0.gguf` (`9.66 GB`) matches native `bfloat16` on `1,471/1,473` decisions (`99.86%` Top-1 agreement, mean absolute probability error `0.00203`, `-0.03` percentage-point delta on the 10-task mean), and `Q4_K_M.gguf` retains `98.78%` Top-1 agreement (`0.00387` probability MAE, `-0.20` percentage-point calibrated delta). Table 7 summarizes hardware memory requirements across formats, including NVIDIA Tesla T4 (`16 GiB`) measurements.
**Table 6: Quantization fidelity across 1,473 scored schema questions on the 10-task stratified Decision Index subset.**
| Metric / Subset | PyTorch `bfloat16` (18.82 GB) | GGUF `Q8_0` (9.66 GB) | GGUF `Q4_K_M` (6.73 GB, `Q8_0` Head) |
| :--- | :---: | :---: | :---: |
| Top-1 Decision Agreement (*N* = 1,473) | 100.00% (ref) | 99.86% (1,471 / 1,473) | 98.78% (1,455 / 1,473) |
| Mean Absolute Probability Error — \\(\lvert \hat{p}_q - \hat{p}_{\mathrm{bf16}} \rvert\\) | 0.00000 | 0.00203 | 0.00387 |
| 10-Task Mean (Uncalibrated Head) | 82.37% | 82.34% (-0.03) | 82.56% (+0.19) |
| 10-Task Mean (Schema-Calibrated) | 83.68% | 83.65% (-0.03) | 83.48% (-0.20) |
| ├─ 1. BFCL / 3. API-Bank / 4. BANKING77 | 100.0% / 95.0% / 100.0% | 100.0% / 95.0% / 100.0% | 100.0% / 95.0% / 100.0% |
| ├─ 2. ToolRet / 7. BRIGHT | 73.15% / 41.93% | 73.70% / 41.00% | 75.60% / 40.31% |
| ├─ 5. CLINC150+OOS / 6. Home Appliances | 93.94% / 90.00% | 93.94% / 90.00% | 93.94% / 90.00% |
| └─ 8. Amazon ESCI / 9. PhishNChips / 10. When2Call | 73.89% / 85.00% / 75.00% | 73.89% / 85.00% / 75.00% | 75.79% / 85.00% / 72.92% |
**Table 7: Disk footprint, peak VRAM allocation, and Tesla T4 (`16 GiB`) latency across precision tiers.**
| Precision Format | Artifact Size | Peak VRAM (4K–16K ctx) | T4 Latency (293 / 1,068 tok) | Target Hardware |
| :--- | :---: | :---: | :---: | :--- |
| `FastBrain-9B-Q4_K_M.gguf` | 6.73 GB | 6.78 – 10.30 GiB | 343.0 ms / 1,154.3 ms | 8–16 GB GPU (RTX 3060/4060, Tesla T4), Apple Silicon (8 GB+) |
| PyTorch `load_in_4bit=True` (NF4 + `fp16` head) | ~5.5 GB (mem) | 6.20 – 6.80 GiB | — | 8–16 GB CUDA GPU (Colab T4, consumer GPUs) |
| `FastBrain-9B-Q8_0.gguf` | 9.66 GB | 9.03 – 12.55 GiB | 349.3 ms / 1,149.8 ms | 12–16 GB GPU (Tesla T4, RTX 4070 Ti), Apple Silicon (16 GB+) |
| PyTorch `load_in_8bit=True` (INT8 + `bf16` head) | ~9.6 GB (mem) | ~11.20 GiB | — | 12–16 GB CUDA GPU |
| Native `bfloat16` (`safetensors` / `BF16.gguf`) | 18.82 / 18.16 GB | 20.30 – 24.70 GiB | — | ≥24 GB GPU (NVIDIA L4, A100, H100, RTX 3090/4090, PRO 6000) |
---
## 5. Reproducible Inference and Deployment
### 5.1 Python / Transformers Runtime (`bfloat16`, `8-bit`, or `4-bit` NF4)
`TypeScript/FastBrain-9B` implements the `joint_schema_model` interface and is API-compatible with `Cloudflare/clef-flash`. Passing `load_in_4bit=True` quantizes only the 9.3B backbone via `bitsandbytes` NF4 while preserving the 116M `JointSchemaHead` in floating point.
```python
import sys
import torch
from huggingface_hub import snapshot_download
model_dir = snapshot_download("TypeScript/FastBrain-9B", ignore_patterns=["*.gguf"])
sys.path.insert(0, model_dir)
import joint_schema_model as clef
# Native bfloat16 (>=24 GB VRAM):
model, processor = clef.load_release_model(model_dir, device="cuda", dtype=torch.bfloat16)
# Mixed-precision 4-bit NF4 (8-16 GB VRAM, e.g., Tesla T4):
# model, processor = clef.load_release_model(model_dir, device="cuda", dtype=torch.float16, load_in_4bit=True)
response = clef.systemone(model, processor, {
"model": "TypeScript/FastBrain-9B",
"state": {
"query": "Can you check my account balance?",
"tools": ["get_balance", "transfer_funds"],
},
"questions": {
"next_action": {
"type": "choice",
"instructions": "Select the appropriate assistant action.",
"criteria": {
"call_balance": "Invoke {\"name\": \"get_balance\", \"arguments\": {}}",
"request_auth": "Ask the user to provide an account identifier or authentication token.",
"abstain": "Decline because personal banking records are out of scope.",
},
},
},
})
print(response)
```
### 5.2 `llama.cpp` GGUF Server (`/v1/systemone`)
> [!IMPORTANT]
> **Runtime Requirements for `ClefModel` GGUF Execution**
> 1. Use **`llama.cpp` build `b11430` or later**, which implements the `LLM_ARCH_CLEF` computation graph (`dec.*` tensors) and the `/v1/systemone` HTTP endpoint. Earlier builds and standard generative wrappers do not execute the cross-attention decision head.
> 2. Set physical micro-batch size equal to logical batch size (`-b 16384 -ub 16384` or `-b 4096 -ub 4096`) together with `--no-cache-prompt`. `ClefModel` pools `<|question_*|>` and `<|option_*|>` spans within a single physical `ubatch`; leaving `-ub` at its default (`512`) splits prompts longer than 512 tokens across micro-batches, and prefix caching can bypass marker positions.
```bash
# 1. Download prebuilt llama.cpp (b11430+) and GGUF checkpoint
curl -sL https://github.com/ggml-org/llama.cpp/releases/download/b11430/llama-b11430-bin-ubuntu-cuda-13.4-x64.tar.gz | tar -xz
pip install -U "huggingface_hub[hf_transfer]"
HF_HUB_ENABLE_HF_TRANSFER=1 hf download TypeScript/FastBrain-9B FastBrain-9B-Q8_0.gguf --local-dir .
# 2. Start llama-server (add --mmproj mmproj-FastBrain-9B-Q8_0.gguf for image inputs)
export LD_LIBRARY_PATH="$(pwd)/llama-b11430:/usr/lib64-nvidia:/usr/local/cuda/lib64:${LD_LIBRARY_PATH}"
./llama-b11430/llama-server \
-m ./FastBrain-9B-Q8_0.gguf \
-ngl 99 -c 16384 -b 16384 -ub 16384 -np 1 -fa on --no-cache-prompt \
--port 8080
```
Querying `/v1/systemone` evaluates multiple heterogeneous primitives (`choice` + `noul`) in a single prefill pass:
```bash
curl http://127.0.0.1:8080/v1/systemone \
-H "Content-Type: application/json" \
-d '{
"model": "FastBrain-9B",
"state": {
"ticket_id": "ORD-99281",
"customer_message": "I was charged twice for my subscription last week and nobody has replied to my ticket."
},
"questions": {
"department": {
"type": "choice",
"instructions": "Assign this ticket to the responsible queue.",
"criteria": {
"billing": "Payments, refunds, duplicate charges, or invoices",
"engineering": "Software defects, crashes, or service outages",
"identity": "Password reset or multi-factor authentication issues"
}
},
"is_urgent": {
"type": "noul",
"instructions": "Does the message report an unresolved financial loss or time-critical failure?"
}
}
}'
```
### 5.3 SGLang Server (`JointSchemaPooler`)
```bash
docker run --gpus all --shm-size 32g -p 30000:30000 \
-v ~/.cache/huggingface:/root/.cache/huggingface --ipc=host \
lmsysorg/sglang:dev-clef \
sglang serve --model-path TypeScript/FastBrain-9B --host 0.0.0.0 --port 30000
```
---
## 6. Repository Manifest and Citation
**Table 8: Repository file specification.**
| Path | Size | Contents |
| :--- | :---: | :--- |
| `model-00001-of-00004.safetensors` .. `00004` | 18.59 GB | `bfloat16` hybrid backbone tensors with all adapted projections merged in-place |
| `joint_head.safetensors`, `joint_head_config.json` | 232 MB | 116M-parameter `JointSchemaHead` weights (`122` tensors) and architectural config |
| `FastBrain-9B-Q4_K_M.gguf` | 6.73 GB | Mixed-precision 4-bit GGUF (`dec.*`, `ssm_*`, and `attn_*` retained in `Q8_0`) |
| `FastBrain-9B-Q8_0.gguf` / `FastBrain-9B-BF16.gguf` | 9.66 / 18.16 GB | 8-bit and `bfloat16` GGUF builds for `llama-server` (`b11430`+) |
| `mmproj-FastBrain-9B-Q8_0.gguf` / `BF16.gguf` | 624 / 922 MB | Vision projector checkpoints for multimodal state inputs |
| `joint_schema_model.py` | 25 KB | Reference `ClefModel`, vectorized `_forward_fast_cuda` head, and `systemone()` runtime |
| `eval_submissions/typescript_fastbrain-9b*.json` | 1.80 MB | Verifiable 2,000-decision zero-shot and prior-fitted traces for `LocalLLaMA/typed-decisions` |
| `eval_submissions/TypeScript__FastBrain-9B*.jsonl` | 110 KB | Verifiable 661-call binary and 7-way category traces for `safeagents-judge-bench` (v2) |
| `.eval_results/*.yaml` | 2.3 KB | Hugging Face Hub machine-readable evaluation records |
| `README.md`, `README_zh.md`, `assets/*.png` | 1.28 MB | English and Chinese technical documentation and evaluation figures |
### Citation
```bibtex
@misc{fastbrain9b2026,
title = {FastBrain-9B: A Single-Pass Non-Autoregressive Model for Calibrated Structured Decisions},
author = {TypeScript},
year = {2026},
howpublished = {\url{https://huggingface.co/TypeScript/FastBrain-9B}},
note = {Apache-2.0 License}
}
```