KnowLine-26B-A4B-Gen2

The second release of PelaAI's System One decision model on a Gemma 4 base: a 26B-A4B mixture of experts.

Inference guide · 中文说明 · weights Apache-2.0 · also: KnowLine-26B-A4B-Gen1, KnowLine-35B-A3B-Gen1, KnowLine-4B-Gen4

Highlights:

• Decision Index 0.3 public suite: 66.94 (self-run), up from 65.76 for KnowLine-26B-A4B-Gen1. KnowLine-35B-A3B-Gen1 scores 69.49 on the same suite and 62.92 on the board's full score; see Comparison.
• Calibrated probabilities: a temperature is folded into the weights. Computed with the board's method, ECE is about 0.01 (board median: 0.078) and the Brier score about 0.26-0.28 (board median: 0.46); see Calibration.
• Other public benchmarks: C-Eval 74.8 (Gen1: 74.3), JevBench / JevBench-hard 88.3 / 76.6 (89.6 / 80.2). It is still clearly lower than the 35B-A3B on Chinese (C-Eval 87.6); see Evaluation.
• Images without any image training: trained on text only, it keeps the base model's vision tower. On the official Decision Index vision public suite (12,210 rows, self-run with the official kit) it scores 63.3 (Gen1: 63.4); see Images.

KnowLine is an independent model. It is not affiliated with or endorsed by TypeSafe, Jev or Google, and it is not derived from TypeSafe or Jev.

What it does

You send a state (text or a chat) and up to 64 typed questions: yes/no, choose one of k, or score on a rubric. The model answers every question with a probability distribution over its options:

• one forward pass per question, with no text generation;
• the output is the probability of each option's label token;
• existing Jev clients only need to change the base URL, and users of other KnowLine models only need to change the model name, because the interface is the same.

Base model	google/gemma-4-26B-A4B-it (Apache-2.0): mixture of experts, about 26B parameters in total and about 4B active per token
Training	LoRA SFT (rank 32, alpha 64, language model only, including the routed experts), merged into bf16 weights (see How Gen2 was made)
Release format	bf16 weights (about 52 GB), with a readout temperature T = 1.25 folded into the final norm weight and the final logit soft-cap (see Calibration). The vision tower is unchanged from the base model; the tokenizer is Gemma 4's, not the one of the Qwen-based KnowLine models.
Serving	serve_knowline.sh: SGLang in bf16, as for Gen1. We do not convert the weights to FP8 at load for this model: in our tests two FP8 runs of the same questions agreed on only 95.7% of the decisions, against 99.4% for bf16. SGLang reserves 85% of the GPU's memory by default (MEM; about 82-88 GB on a 96 GB card); the weights take about 52 GB, so one 80 GB GPU is enough. The transformers backend peaks at about 58 GB. We ran every evaluation with one server on one 96 GB H20.
Languages	English, Simplified Chinese, Traditional Chinese

Quickstart

pip install "sglang==0.5.21" "transformers==5.12.1" requests
bash serve_knowline.sh PelaAI/KnowLine-26B-A4B-Gen2 0 8080     # SGLang (bf16) on :9080 + /v1/systemone on :8080
curl -s http://127.0.0.1:8080/v1/systemone -H 'Content-Type: application/json' -d '{
  "model": "m",
  "state": "Customer: my order arrived broken, I want my money back.",
  "questions": {"refund": {"type": "noul", "instructions": "Should the agent offer a refund?"},
                "tone": {"type": "choice", "instructions": "Customer tone?",
                         "criteria": {"angry": "Angry", "neutral": "Neutral", "happy": "Happy"}}}}'

• Front end: knowline_server.py is a single file that depends only on transformers and requests. It is the same code as in the other KnowLine releases; it renders each question with the model's own chat template.
• Full settings: the exact settings used for our Decision Index run are in INFERENCE.md.

How Gen2 was made

• Base model: google/gemma-4-26B-A4B-it.
• Training: LoRA SFT (rank 32, alpha 64, language model only, including the routed experts), merged into bf16 weights; Gen2 continues from KnowLine-26B-A4B-Gen1 with one further epoch. Training examples include decision tasks written and labelled by LLM teachers.
• Readout temperature: T = 1.25, fitted on held-out data (the same as for Gen1) and folded into the final norm weight and the final logit soft-cap. It changes the size of the probabilities, but not which option ranks first.

The training data is not included in this repository.

Comparison

Scores on the Decision Index 0.3 board. The full score also includes private tests that only the maintainers run (weighted 0.5 same-skill and 0.3 new-domain); the full score of this model is not available yet.

model	size	DI 0.3 public	DI 0.3 full	source
KnowLine-26B-A4B-Gen2	26B-A4B	66.94	not yet scored	self-run, official kit
KnowLine-26B-A4B-Gen1	26B-A4B	65.76	not yet scored	self-run, official kit
KnowLine-35B-A3B-Gen1	35B-A3B	69.49	62.92	board
StartLux-Decision v1.1 27B	27B	72.88	69.12	board
linne-oak 1.0	31B	63.94	65.61	board
span-02	26B-A4B	63.62	62.56	board
Shisa DE-2 (repeat)	26B-A4B	57.57	59.77	board
Jev 1.14	(API)	60.70	61.53	board
AJev lora1	26B-A4B	57.67	56.43	board
GEV-26B-Decide	26B-A4B	58.03	56.17	board

Board rows as of the board's latest update (2026-10-16).

Public-suite scores by area (DI 0.3):

area	KnowLine-26B-A4B-Gen2	KnowLine-26B-A4B-Gen1	KnowLine-35B-A3B-Gen1
Knowledge & reasoning	51.9	50.4	52.5
Language	69.7	69.2	71.2
Retrieval & routing	70.4	69.7	75.2
Tools & agents	84.4	82.2	88.1
Arts & taste	60.0	58.6	63.4

Releases

The model is trained in a self-evolving loop, and new versions will follow.

model	date	DI 0.3 public	notes
KnowLine-26B-A4B-Gen2 (this model)	2026-10-19	66.94	second 26B-A4B release
KnowLine-26B-A4B-Gen1	2026-10-19	65.76	first release on a Gemma 4 base
KnowLine-35B-A3B-Gen1	2026-10-12	69.49	first 35B-A3B release; DI 0.3 full 62.92
KnowLine-4B-Gen4	2026-10-10	64.90	4B, fourth release
KnowLine-4B-Gen3	2026-10-09	63.11	4B, third release
KnowLine-4B-Gen2	2026-10-08	62.54	4B, second release
KnowLine-4B-Gen1	2026-10-07	60.47	4B, first release

Evaluation

All results are self-run and have not been verified by a third party.

Other public benchmarks

suite	26B-A4B-Gen2	26B-A4B-Gen1	35B-A3B-Gen1
C-Eval (4 categories, macro)	74.8	74.3	87.6
JevBench / JevBench-hard	88.3 / 76.6	89.6 / 80.2	85.7 / 70.3

In a paired sign test against Gen1, the difference on JevBench-hard (80.2 → 76.6, p = 0.34) is not significant. Against KnowLine-35B-A3B-Gen1, it remains lower on C-Eval.

Web operation (Mind2Web official test splits, evaluation only)

split	26B-A4B-Gen2 element selection	26B-A4B-Gen2 operation (balanced)	35B-A3B-Gen1 element selection	35B-A3B-Gen1 operation (balanced)
test_task (Cross-Task)	89.5	91.6	94.4	97.4
test_website (Cross-Website)	88.1	94.1	92.4	97.5
test_domain (Cross-Domain)	88.8	93.4	93.1	98.0

• Element selection is 1.1-2.4 points higher than Gen1 on all three splits (87.1 / 87.0 / 87.0); operation is within 0.6 points of Gen1. Still lower than 35B-A3B-Gen1 on all three splits.
• The task is to pick the target element from a candidate set: the target plus up to 5 other elements sampled from the page. This is easier than the original Mind2Web protocol, so do not compare these results with the Mind2Web leaderboard.

Images (zero-shot)

• No image training: Gen2 keeps the base model's vision tower unchanged and saw no images in training. knowline_server.py accepts OpenAI-style image / image_url content parts in a chat state and passes them to SGLang; each question is still one prefill.
• How we measured it: the official vision public suite (12,210 rows, complete), run with the official kit through the release server, 0 failed requests: weighted public index 63.3 (Gen1: 63.4).

Calibration

Computed on the 0.3 public suite using the method described by the Decision Index board: each field is scored right or wrong, confidence is the probability of the chosen option, benchmarks are weighted equally, and 10 equal-width bins are used.

metric	26B-A4B-Gen2 (self-run)	26B-A4B-Gen1 (self-run)	35B-A3B-Gen1 (board)	board median (173 models)
ECE (lower is better)	0.008-0.012	0.02-0.03	0.048	0.078
Brier score (lower is better)	0.26-0.28	0.27-0.29	0.261	0.46
Confidence ≥95% but wrong	0.8-0.9%	1.1-1.2%	1.4%
Mean confidence / accuracy	0.80-0.81 / 0.79-0.81	0.82 / 0.78-0.80	0.85 / 0.80

• Checked against the board: for KnowLine-35B-A3B-Gen1, the board's own figures (ECE 0.048, Brier 0.261) fall inside the ranges our computation gave, so we use the same computation here.
• Temperature in the weights: the final norm weight and the final logit soft-cap include a readout temperature T = 1.25 (fitted on held-out data), the same as for Gen1, so every final logit is divided by 1.25 before the softmax over the option labels. The top-ranked option does not change; the probabilities are just less extreme.
• Why ranges: the board does not publish which 32 benchmarks it uses, so we report ranges over the plausible benchmark sets, as for the earlier releases.
• Recommendation: if your use depends on probability thresholds, check calibration on your own data.

Disclosures

• Decontamination: all training data was checked against every Decision Index 0.3 row and the public benchmarks we report (Intern-Decision, S1MB, JevBench, MMDM). Rows that nearly duplicated a benchmark item, repeated its question or claim, or contained its whole state were removed (751 in the data added for Gen2; 491 for Gen1).
• Teacher-labelled synthetic data: LLM teachers wrote and labelled our synthetic tasks. The labels were filtered, but not all of them were checked by a human.
• Public suite: the Decision Index public suite is public; the board's private tests may show a lower score.

Limitations

• Chinese: clearly lower than KnowLine-35B-A3B-Gen1 on C-Eval (74.8 against 87.6). For Chinese-heavy use, prefer the 35B-A3B.
• JevBench-hard: 76.6 against 80.2 for Gen1; the drop is not significant, but check it if your use is close to JevBench-hard.
• Size and speed: about 52 GB of bf16 weights, so it needs one GPU with 80 GB or more. One request at a time on one H20, requests take 43.1 ms median but 237.6 ms at the 95th percentile, a longer tail than KnowLine-35B-A3B-Gen1 with FP8 serving (174 ms); see INFERENCE.md.
• Web operation and images: below KnowLine-35B-A3B-Gen1 on Mind2Web; there was no image training, so check image use on your own data.
• Prompt injection: planted instructions can change the answer. Keep untrusted text clearly delimited.
• Private tests: the board's private tests may show a lower score than the public suite.

Citation

@misc{knowline26ba4bgen2,
  title  = {KnowLine-26B-A4B-Gen2: a 26B-A4B decision model},
  author = {PelaAI},
  year   = {2026},
  url    = {https://huggingface.co/PelaAI/KnowLine-26B-A4B-Gen2}
}

License

• Weights: Apache-2.0, the same as the base model.
• Code: knowline_server.py is MIT (see the file header).