Text Generation
Transformers
Safetensors
GGUF
English
Chinese
qwen3_5
image-text-to-text
fastbrain
clef
llama-cpp
sglang
systemone
decision-model
image-text-to-typed-output
decision-index
tool-calling
function-calling
intent-classification
structured-decision
conversational
Eval Results
Instructions to use TypeScript/FastBrain-9B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use TypeScript/FastBrain-9B with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="TypeScript/FastBrain-9B") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] pipe(text=messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoProcessor, AutoModelForMultimodalLM processor = AutoProcessor.from_pretrained("TypeScript/FastBrain-9B") model = AutoModelForMultimodalLM.from_pretrained("TypeScript/FastBrain-9B", device_map="auto") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] inputs = processor.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=256) print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use TypeScript/FastBrain-9B with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf TypeScript/FastBrain-9B:Q4_K_M # Run inference directly in the terminal: llama cli -hf TypeScript/FastBrain-9B:Q4_K_M
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf TypeScript/FastBrain-9B:Q4_K_M # Run inference directly in the terminal: llama cli -hf TypeScript/FastBrain-9B:Q4_K_M
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf TypeScript/FastBrain-9B:Q4_K_M # Run inference directly in the terminal: ./build/bin/llama cli -hf TypeScript/FastBrain-9B:Q4_K_M
Use Docker
docker model run hf.co/TypeScript/FastBrain-9B:Q4_K_M
- Llama new
- LM Studio
- Jan
- vLLM
How to use TypeScript/FastBrain-9B with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "TypeScript/FastBrain-9B" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "TypeScript/FastBrain-9B", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/TypeScript/FastBrain-9B:Q4_K_M
- SGLang
How to use TypeScript/FastBrain-9B with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "TypeScript/FastBrain-9B" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "TypeScript/FastBrain-9B", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "TypeScript/FastBrain-9B" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "TypeScript/FastBrain-9B", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Ollama
How to use TypeScript/FastBrain-9B with Ollama:
ollama run hf.co/TypeScript/FastBrain-9B:Q4_K_M
- Unsloth Desktop
- Pi
How to use TypeScript/FastBrain-9B with Pi:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf TypeScript/FastBrain-9B:Q4_K_M
Configure the model in Pi
# Install Pi: npm install -g @earendil-works/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "llama-cpp": { "baseUrl": "http://localhost:8080/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "TypeScript/FastBrain-9B:Q4_K_M" } ] } } }Run Pi
# Start Pi in your project directory: pi
- Docker Model Runner
How to use TypeScript/FastBrain-9B with Docker Model Runner:
docker model run hf.co/TypeScript/FastBrain-9B:Q4_K_M
- Lemonade
How to use TypeScript/FastBrain-9B with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull TypeScript/FastBrain-9B:Q4_K_M
Run and chat with the model
lemonade run user.FastBrain-9B-Q4_K_M
List all available models
lemonade list
- Hermes Agent
How to use TypeScript/FastBrain-9B with Hermes Agent:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf TypeScript/FastBrain-9B:Q4_K_M
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:8080/v1 hermes config set model.default TypeScript/FastBrain-9B:Q4_K_M
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use TypeScript/FastBrain-9B with OpenClaw:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf TypeScript/FastBrain-9B:Q4_K_M
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:8080/v1 \ --custom-model-id "TypeScript/FastBrain-9B:Q4_K_M" \ --custom-provider-id llama-cpp \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
Fix escaped double-backslash inline KaTeX delimiters
Browse files- README.md +8 -8
- README_zh.md +14 -14
README.md
CHANGED
|
@@ -65,16 +65,16 @@ FastBrain-9B uses a 32-layer hybrid sequence backbone (`9.30B` parameters, `bflo
|
|
| 65 |
|
| 66 |
### 2.2 Joint Schema Decision Head and Vectorized CUDA Kernel
|
| 67 |
|
| 68 |
-
The `JointSchemaHead` (`116M` parameters, stored in `joint_head.safetensors`) pools token representations bounded by `<|question_start|>`/`<|question_end|>` and `<|option_start|>`/`<|option_end|>` markers from the final backbone hidden states \
|
| 69 |
-
1. **`choice`** (*K*-way categorical distribution): \
|
| 70 |
-
2. **`noul`** (binary verification): \
|
| 71 |
-
3. **`score`** (ordinal rubric over levels \
|
| 72 |
|
| 73 |
-
In `joint_schema_model.py`, the reference Python span-extraction loop and padded SDPA mask are replaced on CUDA devices by `_forward_fast_cuda`: span means are computed in \
|
| 74 |
|
| 75 |
### 2.3 Ordinal-KL Distillation and Schema-Topology Calibration
|
| 76 |
|
| 77 |
-
Standard cross-entropy or flat KL divergence treats ordinal rubric levels as unordered classes, penalizing adjacent-level errors identically to extreme errors. FastBrain-9B is trained on `TypeScript/FastBrain-Instruct` under a composite loss combining soft-target KL divergence, discrete ranked probability score (squared \
|
| 78 |
|
| 79 |
$$
|
| 80 |
\mathcal{L}(\theta) = \mathrm{KL}(p \,\|\, \hat{p}_\theta) + \lambda_{\mathrm{ord}} \sum_{k=1}^{K-1} \bigl(\hat{F}_{\theta,k} - F_k\bigr)^2 + \mathbb{I}[\text{type}=\text{score}] \cdot \lambda_{\mathrm{mse}} \bigl(\mathbb{E}[\hat{y}_\theta] - \mathbb{E}[y]\bigr)^2.
|
|
@@ -105,7 +105,7 @@ At inference time, `joint_schema_model.py` optionally applies compile-time schem
|
|
| 105 |
|
| 106 |
On the 7-way category classification track, FastBrain-9B achieves `0.751` macro-F1 and `0.858` accuracy (`567/661`), outperforming the 27.4B `Cloudflare/clef` model (`0.563` macro-F1, `0.737` accuracy) by `+18.8` percentage points in macro-F1 and `+12.1` points in accuracy, and improving upon its 9.4B base model `Cloudflare/clef-flash` (`0.462` macro-F1, `0.576` accuracy) by `+28.9` and `+28.2` points, respectively. Per-category F1 reaches `0.945` on `harness_introspection` (*n* = 272, recall `0.978`), `0.828` on `eval_gaming` (*n* = 245, recall `0.853`), and `0.761` on `legit` (*n* = 125, precision `0.975`).
|
| 107 |
|
| 108 |
-
On the binary guardrail track, FastBrain-9B attains `0.985` AUROC and `0.996` malicious-call recall (`534/536`), intercepting all `82/82` unsafe trajectories (`median_bad_calls_before_catch = 0.0`), with `1.000` recall on `harness_introspection` (`272/272`), `credential_misuse` (`15/15`), and `sandbox_escape` (`4/4`). Under the uncalibrated default decision rule \
|
| 109 |
|
| 110 |
**Table 2: Zero-shot evaluation on `harness-hub/safeagents-judge-bench` (`v2`, *N* = 661 calls, 82 unsafe trajectories; and `v1` reference, *N* = 562).**
|
| 111 |
|
|
@@ -216,7 +216,7 @@ Uniform 4-bit quantization degrades cross-attention span readout and linear-atte
|
|
| 216 |
| Metric / Subset | PyTorch `bfloat16` (18.82 GB) | GGUF `Q8_0` (9.66 GB) | GGUF `Q4_K_M` (6.73 GB, `Q8_0` Head) |
|
| 217 |
| :--- | :---: | :---: | :---: |
|
| 218 |
| Top-1 Decision Agreement (*N* = 1,473) | 100.00% (ref) | 99.86% (1,471 / 1,473) | 98.78% (1,455 / 1,473) |
|
| 219 |
-
| Mean Absolute Probability Error — \
|
| 220 |
| 10-Task Mean (Uncalibrated Head) | 82.37% | 82.34% (-0.03) | 82.56% (+0.19) |
|
| 221 |
| 10-Task Mean (Schema-Calibrated) | 83.68% | 83.65% (-0.03) | 83.48% (-0.20) |
|
| 222 |
| ├─ 1. BFCL / 3. API-Bank / 4. BANKING77 | 100.0% / 95.0% / 100.0% | 100.0% / 95.0% / 100.0% | 100.0% / 95.0% / 100.0% |
|
|
|
|
| 65 |
|
| 66 |
### 2.2 Joint Schema Decision Head and Vectorized CUDA Kernel
|
| 67 |
|
| 68 |
+
The `JointSchemaHead` (`116M` parameters, stored in `joint_head.safetensors`) pools token representations bounded by `<|question_start|>`/`<|question_end|>` and `<|option_start|>`/`<|option_end|>` markers from the final backbone hidden states \\(H \in \mathbb{R}^{T \times d}\\), applies cross-attention over question-option pairs, and projects directly to scalar logits for three decision primitives:
|
| 69 |
+
1. **`choice`** (*K*-way categorical distribution): \\(\hat{p} = \operatorname{softmax}(z / \tau) \in \Delta^{K-1}\\).
|
| 70 |
+
2. **`noul`** (binary verification): \\(\hat{p}(\text{true}) = \sigma(z_{\text{true}} - z_{\text{false}})\\).
|
| 71 |
+
3. **`score`** (ordinal rubric over levels \\(v_1 < \dots < v_K\\) ): categorical probabilities \\(\hat{p} \in \Delta^{K-1}\\) together with expected value \\(\mathbb{E}[\hat{y}] = \sum_{k=1}^K \hat{p}_k v_k\\).
|
| 72 |
|
| 73 |
+
In `joint_schema_model.py`, the reference Python span-extraction loop and padded SDPA mask are replaced on CUDA devices by `_forward_fast_cuda`: span means are computed in \\(\mathcal{O}(1)\\) kernel launches via prefix sums (`torch.cumsum`) over the schema suffix, and cross-attention is executed via packed variable-length FlashAttention (`torch.ops.aten._flash_attention_forward` parameterized by `cu_seqlens`). On NVIDIA RTX PRO 6000 Blackwell, this reduces isolated decision-head latency from `3.27 ms` to `1.72 ms` (**1.90× speedup**) with exact numerical equivalence (`max_logit_diff = 0.00000`).
|
| 74 |
|
| 75 |
### 2.3 Ordinal-KL Distillation and Schema-Topology Calibration
|
| 76 |
|
| 77 |
+
Standard cross-entropy or flat KL divergence treats ordinal rubric levels as unordered classes, penalizing adjacent-level errors identically to extreme errors. FastBrain-9B is trained on `TypeScript/FastBrain-Instruct` under a composite loss combining soft-target KL divergence, discrete ranked probability score (squared \\(L_2\\) distance between predicted and target cumulative distribution functions \\(\hat{F}_k = \sum_{j \le k} \hat{p}_j\\) ), and expected-score mean squared error on `score` primitives:
|
| 78 |
|
| 79 |
$$
|
| 80 |
\mathcal{L}(\theta) = \mathrm{KL}(p \,\|\, \hat{p}_\theta) + \lambda_{\mathrm{ord}} \sum_{k=1}^{K-1} \bigl(\hat{F}_{\theta,k} - F_k\bigr)^2 + \mathbb{I}[\text{type}=\text{score}] \cdot \lambda_{\mathrm{mse}} \bigl(\mathbb{E}[\hat{y}_\theta] - \mathbb{E}[y]\bigr)^2.
|
|
|
|
| 105 |
|
| 106 |
On the 7-way category classification track, FastBrain-9B achieves `0.751` macro-F1 and `0.858` accuracy (`567/661`), outperforming the 27.4B `Cloudflare/clef` model (`0.563` macro-F1, `0.737` accuracy) by `+18.8` percentage points in macro-F1 and `+12.1` points in accuracy, and improving upon its 9.4B base model `Cloudflare/clef-flash` (`0.462` macro-F1, `0.576` accuracy) by `+28.9` and `+28.2` points, respectively. Per-category F1 reaches `0.945` on `harness_introspection` (*n* = 272, recall `0.978`), `0.828` on `eval_gaming` (*n* = 245, recall `0.853`), and `0.761` on `legit` (*n* = 125, precision `0.975`).
|
| 107 |
|
| 108 |
+
On the binary guardrail track, FastBrain-9B attains `0.985` AUROC and `0.996` malicious-call recall (`534/536`), intercepting all `82/82` unsafe trajectories (`median_bad_calls_before_catch = 0.0`), with `1.000` recall on `harness_introspection` (`272/272`), `credential_misuse` (`15/15`), and `sandbox_escape` (`4/4`). Under the uncalibrated default decision rule \\(P(\text{block}) \ge 0.50\\), balanced accuracy is `0.814` (false-block rate `0.368` across the 125 `allow` calls, compared with `0.808` balanced accuracy and `0.384` false-block rate for 27.4B `Cloudflare/clef`). Because rank-ordering discrimination is preserved (`0.985` AUROC), setting the operating threshold to \\(P(\text{block}) \ge 0.961\\) yields `0.938` balanced accuracy at a `0.056` (`7/125`) false-block rate while maintaining `82/82` trajectory interception. Table 2 also includes the `v1` release (*N* = 562 calls, `78` trajectories), where FastBrain-9B ranked 1st of 9 on both tracks.
|
| 109 |
|
| 110 |
**Table 2: Zero-shot evaluation on `harness-hub/safeagents-judge-bench` (`v2`, *N* = 661 calls, 82 unsafe trajectories; and `v1` reference, *N* = 562).**
|
| 111 |
|
|
|
|
| 216 |
| Metric / Subset | PyTorch `bfloat16` (18.82 GB) | GGUF `Q8_0` (9.66 GB) | GGUF `Q4_K_M` (6.73 GB, `Q8_0` Head) |
|
| 217 |
| :--- | :---: | :---: | :---: |
|
| 218 |
| Top-1 Decision Agreement (*N* = 1,473) | 100.00% (ref) | 99.86% (1,471 / 1,473) | 98.78% (1,455 / 1,473) |
|
| 219 |
+
| Mean Absolute Probability Error — \\(\lvert \hat{p}_q - \hat{p}_{\mathrm{bf16}} \rvert\\) | 0.00000 | 0.00203 | 0.00387 |
|
| 220 |
| 10-Task Mean (Uncalibrated Head) | 82.37% | 82.34% (-0.03) | 82.56% (+0.19) |
|
| 221 |
| 10-Task Mean (Schema-Calibrated) | 83.68% | 83.65% (-0.03) | 83.48% (-0.20) |
|
| 222 |
| ├─ 1. BFCL / 3. API-Bank / 4. BANKING77 | 100.0% / 95.0% / 100.0% | 100.0% / 95.0% / 100.0% | 100.0% / 95.0% / 100.0% |
|
README_zh.md
CHANGED
|
@@ -19,7 +19,7 @@
|
|
| 19 |
|
| 20 |
## 摘要
|
| 21 |
|
| 22 |
-
将自回归语言模型用于封闭 Schema 分类、工具路由与安全合规校验等结构化决策任务时,逐 Token 解码机制不仅引入与决策复杂度无关的序列生成延迟,还会因表面词形切分导致类别概率分布失准。本文发布 **FastBrain-9B**,一个总参数量为 9.41B 的非自回归结构化决策模型,由 32 层混合 Gated DeltaNet 线性注意力与全注意力主干网络(基于 `Cloudflare/clef-flash` 初始化的 `Qwen3.5-9B` 架构)及 116M 参数的交叉注意力联合决策头(`JointSchemaHead`)组成。给定任意 JSON 或多模态状态以及一组类型化决策定义(多选分类 `choice`、二元真值判别 `noul` 与序数量表评分 `score`),模型仅通过单次 Prefill 前向传播(`output_tokens = 0`)即可同步输出全部问题与候选项的校准概率分布。训练阶段,我们在复合序数 KL 散度、累积分布距离(离散分级概率评分 RPS)与期望值均方误差目标下,联合优化全部 32 层中的 104 个注意力投影矩阵及决策头参数,并将增量权重原地合并至主干张量。在涵盖 109 个参评模型条目的三项公开评测基准中:FastBrain-9B 在 `safeagents-judge-bench`(v2)7 类违规判别赛道位列 **第 1 / 8**(Macro-F1 `0.751`,准确率 `0.858`,较 27.4B 参数的 `Cloudflare/clef` 高出 18.8 个百分点),并在二元拦截赛道实现 `82/82` 越界轨迹全检出(AUROC `0.985`);在 `LocalLLaMA/typed-decisions` 零样本总榜位列 **第 7 / 43**(开源权重模型 **第 5 / 36**,≤9B 开源模型 **第 2 / 19**),中位推理延迟为 `50 ms`(较 31B 模型 `Quyet-1.0-Large` 快 66.7 倍),在冻结全部模型权重仅拟合先验与温度参数的设定下位列 `train` 校准榜 **并列第 12(列表第 14 / 51)**(准确率 `0.7970`,KL 散度 `0.0816`);在 Cloudflare Decision Index(v0.2.1)10 任务套件上取得 **80.55%** 的宏平均分,位列全量级 **第 2 / 7**、≤9B 量级 **10 / 10 任务单项第 1**,并在其中 5 项任务上超越 27.4B 的 `Cloudflare/clef`。
|
| 23 |
|
| 24 |
---
|
| 25 |
|
|
@@ -39,16 +39,16 @@ FastBrain-9B 主干网络包含 32 层混合序列建模模块(`9.30B` 参数
|
|
| 39 |
|
| 40 |
### 2.2 联合 Schema 决策头与向量化 CUDA 算子
|
| 41 |
|
| 42 |
-
`JointSchemaHead`(`116M` 参数,独立存储于 `joint_head.safetensors`)从主干末层隐藏状态 \
|
| 43 |
-
1. **`choice`** (*K* 分类离散分布): 输出归一化概率向量 \
|
| 44 |
-
2. **`noul`** (二元真值判别): 输出命题成立概率 \
|
| 45 |
-
3. **`score`** (定义在有序分值 \
|
| 46 |
|
| 47 |
-
在参考运行时 `joint_schema_model.py` 中,针对 CUDA 设备实现了融合算子路径 `_forward_fast_cuda`:利用基于 `torch.cumsum` 的 \
|
| 48 |
|
| 49 |
### 2.3 序数 KL 蒸馏与编译期 Schema 拓扑校准
|
| 50 |
|
| 51 |
-
标准交叉熵或普通多类 KL 散度将有序评定量表视为无序独立类别,对相邻等级误差与跨端极端误差施加同等惩罚。FastBrain-9B 在 `TypeScript/FastBrain-Instruct` 数据集上采用复合分布匹配损失进行训练,联合优化软标签 KL 散度、离散分级概率评分(预测累积分布 \
|
| 52 |
|
| 53 |
$$
|
| 54 |
\mathcal{L}(\theta) = \mathrm{KL}(p \,\|\, \hat{p}_\theta) + \lambda_{\mathrm{ord}} \sum_{k=1}^{K-1} \bigl(\hat{F}_{\theta,k} - F_k\bigr)^2 + \mathbb{I}[\text{type}=\text{score}] \cdot \lambda_{\mathrm{mse}} \bigl(\mathbb{E}[\hat{y}_\theta] - \mathbb{E}[y]\bigr)^2.
|
|
@@ -79,7 +79,7 @@ $$
|
|
| 79 |
|
| 80 |
在 7 类违规归因赛道上,FastBrain-9B 取得 `0.751` 的 Macro-F1 与 `0.858` 的分类准确率(`567/661`),位列全部参评模型第 1:相较于参数量约为其 3 倍的 27.4B 模型 `Cloudflare/clef`(Macro-F1 `0.563`,准确率 `0.737`),分别高出 `+18.8` 与 `+12.1` 个百分点;相较于同尺寸基座 `Cloudflare/clef-flash`(Macro-F1 `0.462`,准确率 `0.576`),分别提升 `+28.9` 与 `+28.2` 个百分点。分项指标上,`harness_introspection`(*n* = 272)F1 达 `0.945`(召回率 `0.978`),`eval_gaming`(*n* = 245)F1 达 `0.828`(召回率 `0.853`),合法调用 `legit`(*n* = 125)精确率达 `0.975`。
|
| 81 |
|
| 82 |
-
在二元安全拦截赛道上,FastBrain-9B 的 AUROC 为 `0.985`,恶意调用总召回率为 `0.996`(`534/536`),全部 `82/82` 条越界轨迹均在首次违规调用处即被捕获(`median_bad_calls_before_catch = 0.0`),其中 `harness_introspection`(`272/272`)、`credential_misuse`(`15/15`)与 `sandbox_escape`(`4/4`)三类召回率均为 `1.000`。在未经验证集调整的默认判别阈值 \
|
| 83 |
|
| 84 |
**表 2:`harness-hub/safeagents-judge-bench` 零样本评测结果对比(`v2` 测试集 *N* = 661,82 条越界轨迹;附 `v1` 归档对照 *N* = 562)。**
|
| 85 |
|
|
@@ -102,9 +102,9 @@ $$
|
|
| 102 |
|
| 103 |
[`LocalLLaMA/typed-decisions`](https://huggingface.co/datasets/LocalLLaMA/typed-decisions) 评测集覆盖智能体轨迹可观测性(`agent_trace_observability`)、客户服务(`customer_service`)、供应商发票核验(`invoice_processing`)与安全事件响应(`security_incidents`)四类企业工作流,测试集包含 400 个独立案例、共 2,000 个类型化子问题(涵盖 `noul`、`choice` 与 `score`),并以多教师模型集成的软标签分布作为真值基准。官方仓库存在两种并列真值处理脚本:`bench/score_with_jobs.py` 对真值分布中概率并列最高的选项按首个索引判对(`g.index(max(g))`),而 `scripts/score_model.py` 将 20 道并列题中的任一最高分选项均计为正确;两种脚本下的 KL 散度与 Brier 评分完全一致。我们在表 3 中同时报告两种口径的准确率,并在 [`eval_submissions/`](./eval_submissions/) 中公开原始预测文件。
|
| 104 |
|
| 105 |
-
在零样本总榜(*N* = 43 个参评模型)中,FastBrain-9B 取得 `0.7525`(并列口径 `0.7625`)的准确率、`0.1508` 的 KL 散度、`0.0795` 的 Brier 评分与 `0.085` 的期望校准误差(ECE),在 RTX PRO 6000(SGLang)上的 p50 延迟为 `50 ms`。该成绩位列全量零样本榜 **第 7 / 43**、开源权重模型 **第 5 / 36**、≤9B 开源模型 **第 2 / 19** 以及全部 `Clef` 架构模型 **第 1**,较 9.4B 基座 `Cloudflare/clef-flash`(准确率 `0.7100`,KL `0.2093`)提升 `+4.25` 个百分点,较 27.4B `Cloudflare/clef`(准确率 `0.7275`,KL `0.1907`)提升 `+2.50` 个百分点。在零样本准确率高于 `0.700` 的全部 28 个模型中,FastBrain-9B 具有最低的 p50 推理延迟(`50 ms`,对比 9B `Amos 1 medium` 的 `118 ms`、27B `Amos 1 large` 的 `357 ms`、`meraGPT Decider 1` 的 `526 ms` 以及 31B `Quyet-1.0-Large` 的 `3,335 ms`)。若直接评测未经后验规则调整的序数 KL 蒸馏原生头,零样本准确率为 `0.7735`(KL `0.1039`,Brier `0.0582`)。
|
| 106 |
|
| 107 |
-
在使用 `train` 划分(1,200 个样本)的独立榜单(*N* = 51 个模型)上,在 **完全冻结主干与决策头权重 (`no weights changed`)** 、仅拟合逐问题对数先验偏置与标量温度参数的前提下,FastBrain-9B 取得 `0.7970`(并列口径 `0.8070`)的准确率、`0.0816` 的 KL 散度与 `0.0437` 的 Brier 评分,位列 **并列第 12(列表第 14 / 51)**,并在冻结权重模型中取得最优的 KL 分布校准度。
|
| 108 |
|
| 109 |
**表 3:`LocalLLaMA/typed-decisions` 测试集(*N* = 400 案例,2,000 次决策)评测结果。`/` 前后分别对应 `score_with_jobs.py` 与 `score_model.py` 口径。**
|
| 110 |
|
|
@@ -128,9 +128,9 @@ $$
|
|
| 128 |
|
| 129 |
### 3.3 多领域结构化决策基准 (Cloudflare Decision Index `v0.2.1`)
|
| 130 |
|
| 131 |
-
在涵盖 10 个评测数据集、共 *N* = 22,427 条请求的 **Cloudflare Decision Index (`v0.2.1`)** 套件(覆盖 AST 函数调用、20 候选工具检索重排、多步状态 API 调用、细粒度金融意图识别、开放集越界拒识、推理密集型检索、电商相关性分级、多探针钓鱼网站鉴别与工具调用触发治理)上,FastBrain-9B 的无加权 10 任务宏平均得分为 **80.55%**
|
| 132 |
|
| 133 |
-
在 ≤9B 参数量级(对比 `Clef-flash` 9.4B、`Kev 9B` 与 `Laya` 0.4B)中,FastBrain-9B 在全部 **10 / 10 个子基准上均取得第 1**,宏平均分较同架构基座 `Clef-flash`(`74.96%`)提升 `+5.59` 个百分点,较 `Kev 9B`(`76.51%`)提升 `+4.04` 个百分点。增幅主要集中于对边界校准敏感的开放集拒识与安全治理任务:`CLINC150+OOS` 提升 `+31.67` 个百分点(`98.44%` 对比 `66.77%`),`When2Call` 提升 `+15.55` 个百分点(`81.13%` 对比 `65.58%`),`PhishNChips` 提升 `+6.80` 个百分点(`78.75%` 对比 `71.95%`)。跨全部参数规模(最高 27.4B)比较,FastBrain-9B 总宏平均位列 **第 2 / 7**,高于 `27B Jev`(`80.32%`)与 `26B-A4B DiffusionGemma Jev`(`69.14%`),并在 `BFCL`、`API-Bank`、`CLINC150+OOS`、`Home Appliances` 与 `When2Call` 五个子基准上高于 `27.4B Clef`(`81.81%`)。
|
| 134 |
|
| 135 |

|
| 136 |
*图 1. Cloudflare Decision Index(v0.2.1,22,427 条请求)10 任务宏平均分与模型参数规模关系。*
|
|
@@ -190,7 +190,7 @@ $$
|
|
| 190 |
| 评测指标 / 子任务 | PyTorch `bfloat16` (18.82 GB) | GGUF `Q8_0` (9.66 GB) | GGUF `Q4_K_M` (6.73 GB, `Q8_0` 决策头) |
|
| 191 |
| :--- | :---: | :---: | :---: |
|
| 192 |
| Top-1 决策一致率 (*N* = 1,473) | 100.00% (基准) | 99.86% (1,471 / 1,473) | 98.78% (1,455 / 1,473) |
|
| 193 |
-
| 预测概率平均绝对误差 — \
|
| 194 |
| 10 任务宏平均(未校准原生输出) | 82.37% | 82.34% (-0.03) | 82.56% (+0.19) |
|
| 195 |
| 10 任务宏平均(Schema 拓扑校准) | 83.68% | 83.65% (-0.03) | 83.48% (-0.20) |
|
| 196 |
| ├─ 1. BFCL / 3. API-Bank / 4. BANKING77 | 100.0% / 95.0% / 100.0% | 100.0% / 95.0% / 100.0% | 100.0% / 95.0% / 100.0% |
|
|
@@ -256,7 +256,7 @@ print(response)
|
|
| 256 |
|
| 257 |
> [!IMPORTANT]
|
| 258 |
> **`ClefModel` 架构 GGUF 运行参数要求**
|
| 259 |
-
> 1. 需使用 **`llama.cpp` `b11430` 或更高版本**。该版本包含 `LLM_ARCH_CLEF` 计算图定义(支持 `dec.*` 决策头张量)与 `/v1/systemone` HTTP 路由;旧版构建或纯生成式封装无法执行交叉注意力决策头。
|
| 260 |
> 2. 启动时需将物理微批次设为���逻辑批次一致(如 `-b 16384 -ub 16384` 或 `-b 4096 -ub 4096`),并开启 `--no-cache-prompt`。`ClefModel` 在单个物理 `ubatch` 内对 `<|question_*|>` 与 `<|option_*|>` 区间执行池化,默认的 `-ub 512` 会截断超过 512 tokens 的输入序列,而前缀缓存可能跳过边界标记计算。
|
| 261 |
|
| 262 |
```bash
|
|
|
|
| 19 |
|
| 20 |
## 摘要
|
| 21 |
|
| 22 |
+
将自回归语言模型用于封闭 Schema 分类、工具路由与安全合规校验等结构化决策任务时,逐 Token 解码机制不仅引入与决策复杂度无关的序列生成延迟,还会因表面词形切分导致类别概率分布失准。本文发布 **FastBrain-9B** ,一个总参数量为 9.41B 的非自回归结构化决策模型,由 32 层混合 Gated DeltaNet 线性注意力与全注意力主干网络(基于 `Cloudflare/clef-flash` 初始化的 `Qwen3.5-9B` 架构)及 116M 参数的交叉注意力联合决策头(`JointSchemaHead`)组成。给定任意 JSON 或多模态状态以及一组类型化决策定义(多选分类 `choice`、二元真值判别 `noul` 与序数量表评分 `score`),模型仅通过单次 Prefill 前向传播(`output_tokens = 0`)即可同步输出全部问题与候选项的校准概率分布。训练阶段,我们在复合序数 KL 散度、累积分布距离(离散分级概率评分 RPS)与期望值均方误差目标下,联合优化全部 32 层中的 104 个注意力投影矩阵及决策头参数,并将增量权重原地合并至主干张量。在涵盖 109 个参评模型条目的三项公开评测基准中:FastBrain-9B 在 `safeagents-judge-bench`(v2)7 类违规判别赛道位列 **第 1 / 8** (Macro-F1 `0.751`,准确率 `0.858`,较 27.4B 参数的 `Cloudflare/clef` 高出 18.8 个百分点),并在二元拦截赛道实现 `82/82` 越界轨迹全检出(AUROC `0.985`);在 `LocalLLaMA/typed-decisions` 零样本总榜位列 **第 7 / 43** (开源权重模型 **第 5 / 36** ,≤9B 开源模型 **第 2 / 19**),中位推理延迟为 `50 ms`(较 31B 模型 `Quyet-1.0-Large` 快 66.7 倍),在冻结全部模型权重仅拟合先验与温度参数的设定下位列 `train` 校准榜 **并列第 12(列表第 14 / 51)** (准确率 `0.7970`,KL 散度 `0.0816`);在 Cloudflare Decision Index(v0.2.1)10 任务套件上取得 **80.55%** 的宏平均分,位列全量级 **第 2 / 7** 、≤9B 量级 **10 / 10 任务单项第 1** ,并在其中 5 项任务上超越 27.4B 的 `Cloudflare/clef`。
|
| 23 |
|
| 24 |
---
|
| 25 |
|
|
|
|
| 39 |
|
| 40 |
### 2.2 联合 Schema 决策头与向量化 CUDA 算子
|
| 41 |
|
| 42 |
+
`JointSchemaHead`(`116M` 参数,独立存储于 `joint_head.safetensors`)从主干末层隐藏状态 \\(H \in \mathbb{R}^{T \times d}\\) 中提取由 `<|question_start|>`/`<|question_end|>` 与 `<|option_start|>`/`<|option_end|>` 界定的片段表征,经问题-选项交叉注意力交互后映射为三类基础决策原语的标量对数几率:
|
| 43 |
+
1. **`choice`** (*K* 分类离散分布): 输出归一化概率向量 \\(\hat{p} = \operatorname{softmax}(z / \tau) \in \Delta^{K-1}\\) 。
|
| 44 |
+
2. **`noul`** (二元真值判别): 输出命题成立概率 \\(\hat{p}(\text{true}) = \sigma(z_{\text{true}} - z_{\text{false}})\\) 。
|
| 45 |
+
3. **`score`** (定义在有序分值 \\(v_1 < \dots < v_K\\) 上的量表评估): 同时输出各等级概率 \\(\hat{p} \in \Delta^{K-1}\\) 与数学期望得分 \\(\mathbb{E}[\hat{y}] = \sum_{k=1}^K \hat{p}_k v_k\\) 。
|
| 46 |
|
| 47 |
+
在参考运行时 `joint_schema_model.py` 中,针对 CUDA 设备实现了融合算子路径 `_forward_fast_cuda`:利用基于 `torch.cumsum` 的 \\(\mathcal{O}(1)\\) 前缀和差分替代逐片段 Python 循环完成变长片段均值池化,并通过 `cu_seqlens` 紧凑打包调用变长 FlashAttention(`torch.ops.aten._flash_attention_forward`)替代稠密掩码注意力。在 NVIDIA RTX PRO 6000 Blackwell 上,该实现将决策头单次前向耗时由 `3.27 ms` 降至 `1.72 ms`(**1.90 倍加速**),且保持严格数值等价(最大 Logit 绝对偏差 `max_logit_diff = 0.00000`)。
|
| 48 |
|
| 49 |
### 2.3 序数 KL 蒸馏与编译期 Schema 拓扑校准
|
| 50 |
|
| 51 |
+
标准交叉熵或普通多类 KL 散度将有序评定量表视为无序独立类别,对相邻等级误差与跨端极端误差施加同等惩罚。FastBrain-9B 在 `TypeScript/FastBrain-Instruct` 数据集上采用复合分布匹配损失进行训练,联合优化软标签 KL 散度、离散分级概率评分(预测累积分布 \\(\hat{F}_k = \sum_{j \le k} \hat{p}_j\\) 与目标累积分布 \\(F_k\\) 之间的二阶距离)以及 `score` 题型的期望值均方误差:
|
| 52 |
|
| 53 |
$$
|
| 54 |
\mathcal{L}(\theta) = \mathrm{KL}(p \,\|\, \hat{p}_\theta) + \lambda_{\mathrm{ord}} \sum_{k=1}^{K-1} \bigl(\hat{F}_{\theta,k} - F_k\bigr)^2 + \mathbb{I}[\text{type}=\text{score}] \cdot \lambda_{\mathrm{mse}} \bigl(\mathbb{E}[\hat{y}_\theta] - \mathbb{E}[y]\bigr)^2.
|
|
|
|
| 79 |
|
| 80 |
在 7 类违规归因赛道上,FastBrain-9B 取得 `0.751` 的 Macro-F1 与 `0.858` 的分类准确率(`567/661`),位列全部参评模型第 1:相较于参数量约为其 3 倍的 27.4B 模型 `Cloudflare/clef`(Macro-F1 `0.563`,准确率 `0.737`),分别高出 `+18.8` 与 `+12.1` 个百分点;相较于同尺寸基座 `Cloudflare/clef-flash`(Macro-F1 `0.462`,准确率 `0.576`),分别提升 `+28.9` 与 `+28.2` 个百分点。分项指标上,`harness_introspection`(*n* = 272)F1 达 `0.945`(召回率 `0.978`),`eval_gaming`(*n* = 245)F1 达 `0.828`(召回率 `0.853`),合法调用 `legit`(*n* = 125)精确率达 `0.975`。
|
| 81 |
|
| 82 |
+
在二元安全拦截赛道上,FastBrain-9B 的 AUROC 为 `0.985`,恶意调用总召回率为 `0.996`(`534/536`),全部 `82/82` 条越界轨迹均在首次违规调用处即被捕获(`median_bad_calls_before_catch = 0.0`),其中 `harness_introspection`(`272/272`)、`credential_misuse`(`15/15`)与 `sandbox_escape`(`4/4`)三类召回率均为 `1.000`。在未经验证集调整的默认判别阈值 \\(P(\text{block}) \ge 0.50\\) 下,平衡准确率为 `0.814`(误拦率 `0.368`,优于 27.4B `Cloudflare/clef` 的平衡准确率 `0.808` 与误拦率 `0.384`)。由于连续概率排序区分度保持在 `0.985` AUROC,将工作点阈值设定为 \\(P(\text{block}) \ge 0.961\\) 时,平衡准确率提升至 `0.938`,误拦率降至 `0.056`(`7/125`),同时仍保持 `82/82` 越界轨迹全检出。表 2 同时列出了 `v1` 版本(*N* = 562,78 条轨迹)的官方归档结果,FastBrain-9B 在该版本二元与多分类双赛道均位居第 1 / 9。
|
| 83 |
|
| 84 |
**表 2:`harness-hub/safeagents-judge-bench` 零样本评测结果对比(`v2` 测试集 *N* = 661,82 条越界轨迹;附 `v1` 归档对照 *N* = 562)。**
|
| 85 |
|
|
|
|
| 102 |
|
| 103 |
[`LocalLLaMA/typed-decisions`](https://huggingface.co/datasets/LocalLLaMA/typed-decisions) 评测集覆盖智能体轨迹可观测性(`agent_trace_observability`)、客户服务(`customer_service`)、供应商发票核验(`invoice_processing`)与安全事件响应(`security_incidents`)四类企业工作流,测试集包含 400 个独立案例、共 2,000 个类型化子问题(涵盖 `noul`、`choice` 与 `score`),并以多教师模型集成的软标签分布作为真值基准。官方仓库存在两种并列真值处理脚本:`bench/score_with_jobs.py` 对真值分布中概率并列最高的选项按首个索引判对(`g.index(max(g))`),而 `scripts/score_model.py` 将 20 道并列题中的任一最高分选项均计为正确;两种脚本下的 KL 散度与 Brier 评分完全一致。我们在表 3 中同时报告两种口径的准确率,并在 [`eval_submissions/`](./eval_submissions/) 中公开原始预测文件。
|
| 104 |
|
| 105 |
+
在零样本总榜(*N* = 43 个参评模型)中,FastBrain-9B 取得 `0.7525`(并列口径 `0.7625`)的准确率、`0.1508` 的 KL 散度、`0.0795` 的 Brier 评分与 `0.085` 的期望校准误差(ECE),在 RTX PRO 6000(SGLang)上的 p50 延迟为 `50 ms`。该成绩位列全量零样本榜 **第 7 / 43** 、开源权重模型 **第 5 / 36** 、≤9B 开源模型 **第 2 / 19** 以及全部 `Clef` 架构模型 **第 1** ,较 9.4B 基座 `Cloudflare/clef-flash`(准确率 `0.7100`,KL `0.2093`)提升 `+4.25` 个百分点,较 27.4B `Cloudflare/clef`(准确率 `0.7275`,KL `0.1907`)提升 `+2.50` 个百分点。在零样本准确率高于 `0.700` 的全部 28 个模型中,FastBrain-9B 具有最低的 p50 推理延迟(`50 ms`,对比 9B `Amos 1 medium` 的 `118 ms`、27B `Amos 1 large` 的 `357 ms`、`meraGPT Decider 1` 的 `526 ms` 以及 31B `Quyet-1.0-Large` 的 `3,335 ms`)。若直接评测未经后验规则调整的序数 KL 蒸馏原生头,零样本准确率为 `0.7735`(KL `0.1039`,Brier `0.0582`)。
|
| 106 |
|
| 107 |
+
在使用 `train` 划分(1,200 个样本)的独立榜单(*N* = 51 个模型)上,在 **完全冻结主干与决策头权重 (`no weights changed`)** 、仅拟合逐问题对数先验偏置与标量温度参数的前提下,FastBrain-9B 取得 `0.7970`(并列口径 `0.8070`)的准确率、`0.0816` 的 KL 散度与 `0.0437` 的 Brier 评分,位列 **并列第 12(列表第 14 / 51)** ,并在冻结权重模型中取得最优的 KL 分布校准度。
|
| 108 |
|
| 109 |
**表 3:`LocalLLaMA/typed-decisions` 测试集(*N* = 400 案例,2,000 次决策)评测结果。`/` 前后分别对应 `score_with_jobs.py` 与 `score_model.py` 口径。**
|
| 110 |
|
|
|
|
| 128 |
|
| 129 |
### 3.3 多领域结构化决策基准 (Cloudflare Decision Index `v0.2.1`)
|
| 130 |
|
| 131 |
+
在涵盖 10 个评测数据集、共 *N* = 22,427 条请求的 **Cloudflare Decision Index (`v0.2.1`)** 套件(覆盖 AST 函数调用、20 候选工具检索重排、多步状态 API 调用、细粒度金融意图识别、开放集越界拒识、推理密集型检索、电商相关性分级、多探针钓鱼网站鉴别与工具调用触发治理)上,FastBrain-9B 的无加权 10 任务宏平均得分为 **80.55%** (表 4,图 1–3)。
|
| 132 |
|
| 133 |
+
在 ≤9B 参数量级(对比 `Clef-flash` 9.4B、`Kev 9B` 与 `Laya` 0.4B)中,FastBrain-9B 在全部 **10 / 10 个子基准上均取得第 1** ,宏平均分较同架构基座 `Clef-flash`(`74.96%`)提升 `+5.59` 个百分点,较 `Kev 9B`(`76.51%`)提升 `+4.04` 个百分点。增幅主要集中于对边界校准敏感的开放集拒识与安全治理任务:`CLINC150+OOS` 提升 `+31.67` 个百分点(`98.44%` 对比 `66.77%`),`When2Call` 提升 `+15.55` 个百分点(`81.13%` 对比 `65.58%`),`PhishNChips` 提升 `+6.80` 个百分点(`78.75%` 对比 `71.95%`)。跨全部参数规模(最高 27.4B)比较,FastBrain-9B 总宏平均位列 **第 2 / 7** ,高于 `27B Jev`(`80.32%`)与 `26B-A4B DiffusionGemma Jev`(`69.14%`),并在 `BFCL`、`API-Bank`、`CLINC150+OOS`、`Home Appliances` 与 `When2Call` 五个子基准上高于 `27.4B Clef`(`81.81%`)。
|
| 134 |
|
| 135 |

|
| 136 |
*图 1. Cloudflare Decision Index(v0.2.1,22,427 条请求)10 任务宏平均分与模型参数规模关系。*
|
|
|
|
| 190 |
| 评测指标 / 子任务 | PyTorch `bfloat16` (18.82 GB) | GGUF `Q8_0` (9.66 GB) | GGUF `Q4_K_M` (6.73 GB, `Q8_0` 决策头) |
|
| 191 |
| :--- | :---: | :---: | :---: |
|
| 192 |
| Top-1 决策一致率 (*N* = 1,473) | 100.00% (基准) | 99.86% (1,471 / 1,473) | 98.78% (1,455 / 1,473) |
|
| 193 |
+
| 预测概率平均绝对误差 — \\(\lvert \hat{p}_q - \hat{p}_{\mathrm{bf16}} \rvert\\) | 0.00000 | 0.00203 | 0.00387 |
|
| 194 |
| 10 任务宏平均(未校准原生输出) | 82.37% | 82.34% (-0.03) | 82.56% (+0.19) |
|
| 195 |
| 10 任务宏平均(Schema 拓扑校准) | 83.68% | 83.65% (-0.03) | 83.48% (-0.20) |
|
| 196 |
| ├─ 1. BFCL / 3. API-Bank / 4. BANKING77 | 100.0% / 95.0% / 100.0% | 100.0% / 95.0% / 100.0% | 100.0% / 95.0% / 100.0% |
|
|
|
|
| 256 |
|
| 257 |
> [!IMPORTANT]
|
| 258 |
> **`ClefModel` 架构 GGUF 运行参数要求**
|
| 259 |
+
> 1. 需使用 **`llama.cpp` `b11430` 或更高版本** 。该版本包含 `LLM_ARCH_CLEF` 计算图定义(支持 `dec.*` 决策头张量)与 `/v1/systemone` HTTP 路由;旧版构建或纯生成式封装无法执行交叉注意力决策头。
|
| 260 |
> 2. 启动时需将物理微批次设为���逻辑批次一致(如 `-b 16384 -ub 16384` 或 `-b 4096 -ub 4096`),并开启 `--no-cache-prompt`。`ClefModel` 在单个物理 `ubatch` 内对 `<|question_*|>` 与 `<|option_*|>` 区间执行池化,默认的 `-ub 512` 会截断超过 512 tokens 的输入序列,而前缀缓存可能跳过边界标记计算。
|
| 261 |
|
| 262 |
```bash
|