BunGPT-DeepEdu-Decision-4B-Open
A decision model. It reads a state (a message, a conversation, a document) and one or more typed questions about it, and scores every option of every question in a single pass, without generating text. Use it for the small decisions inside a product: routing a turn, detecting intent, flagging unsafe messages, checking a claim.
- Backbone: Qwen3.5-4B, causal attention, with a small scoring head that gives each option its own score.
- Questions:
choice(pick one of named options) andnoul(yes/no). - Output: per question the chosen option and a probability for each option, plus the raw option scores.
- Serving: a Python API and an HTTP server compatible with the Jev Decision Index
POST /v1/systemoneformat.
Quickstart
huggingface-cli download DeepEduVN/bungpt-open --include "source/*" --local-dir bungpt-open
pip install ./bungpt-open/source
pip install --no-build-isolation -r bungpt-open/source/requirements-fast.txt # optional: faster kernels
If the repository is private, set HF_TOKEN to a token with read access first. The weights are downloaded on the first
from_pretrained call.
from deepedu_decision import DecisionModel
model = DecisionModel.from_pretrained("DeepEduVN/bungpt-open", device="cuda:0")
state = "Hแปc sinh: Em khรดng hiแปu vรฌ sao ฤแบกo hร m cแปงa x^2 lร 2x, thแบงy giแบฃi thรญch lแบกi ฤฦฐแปฃc khรดng แบก?"
questions = {
"intent": {
"type": "choice",
"instructions": "What does the student want?",
"criteria": {
"explain": "An explanation of a concept",
"solve": "A worked solution to a specific exercise",
"chitchat": "Small talk, nothing to answer",
},
},
"unsafe": {"type": "noul", "instructions": "Is this message harmful or inappropriate for a school setting?"},
}
print(model.decide(state, questions))
{
"answers": {
"intent": {"type": "choice", "choice": "explain",
"probabilities": {"explain": 0.986, "solve": 0.011, "chitchat": 0.002}},
"unsafe": {"type": "noul", "noul": 0.004}
},
"scores": {
"intent": {"explain": 3.635, "solve": -4.482, "chitchat": -6.13},
"unsafe": {"false": 5.177, "true": -5.566}
}
}
decide_batch([...]) takes many requests at once.
HTTP server
CUDA_VISIBLE_DEVICES=0 python -m deepedu_decision.serve --model DeepEduVN/bungpt-open --port 9100
# multi-turn traffic: add --cache_mb 4096 to keep prefix states between requests
POST /v1/systemone takes the Jev Decision Index request {"state": ..., "questions": {...}} and returns its response
format (model, answers), so any Decision Index client works unchanged. Two fields are added:
scores: the raw score of every option,{question: {option: score}}. Higher is better. Use them for your own thresholds or to compare options across requests.usage:wall_ms,queue_ms,batch_rows,input_tokens,prefills,cache_hit_tokens.
GET /health returns the model name, limits and queue depth.
Errors: a request that does not fit (--max_length, --max_questions, body size) or uses an unsupported question type
gets HTTP 422 whose message contains maximum context length; a full queue gets 503 with Retry-After; a malformed
body gets 400.
Main flags: --token_budget (tokens per GPU forward, default 16384), --max_length, --max_questions (16),
--max_pending (64), --max_batch_rows, --batch_wait_ms, --attention {auto,flash,sdpa}, --no_fork,
--cache_mb.
Reading the output
- Probabilities. For a
choicequestion, option k gets ฯ(z_k) / ฮฃ_j ฯ(z_j), where z are the raw scores and ฯ is the logistic function. Fornoul,noulis the probability oftrue. - "None of these". Each option is scored on its own, so the raw scores can all be low when no option fits. Use
max(scores)(or ฯ of it) as a fit signal before trusting the chosen option. - Thresholds. For yes/no decisions where one error is worse than the other (for example a guardrail), choose the
cutoff on
noulfrom your own labelled data rather than using 0.5.
How it is fast
Attention is causal, so the shared part of a request is computed once: the state is read once for all questions, and
each question once for all of its options. Many questions about one long message cost little more than one. With
--cache_mb, later turns of a conversation reuse the earlier turns' states.
Median client latency on one RTX 3090, HTTP included (same architecture and server):
| Workload | Median / p95 (ms) |
|---|---|
| 1 question, 256 tokens | 129 / 132 |
| 1 question, 4,096 tokens | 649 / 698 |
| 16 questions, 1,024 tokens | 653 / 1,288 |
| 16 questions, 4,096 tokens | 1,091 / 1,805 |
| Multi-turn, turns 2โ6, with prefix cache | 173 / 502 |
Short single questions carry about 80 ms of serving overhead today (no CUDA graphs yet).
Results
DeepEdu benchmark (Vietnamese education product decisions; accuracy or macro-F1). All models are 4B and were run by us on the same records, each with its own published inference code.
| Model (all 4B) | Intent | Routing | Guardrail | Knowledge | Mean |
|---|---|---|---|---|---|
| Decider Flash v1.3 | 0.826 | 0.800 | 0.680 | 0.611 | 0.729 |
| StartLux-Decision v1.1 Instant | 0.824 | 0.808 | 0.703 | 0.568 | 0.726 |
| KnowLine 4B Gen5.5 | 0.807 | 0.808 | 0.691 | 0.585 | 0.723 |
| BunGPT-DeepEdu-Decision-4B-Open | 0.822 | 0.820 | 0.626 | 0.539 | 0.702 |
| Valen-Decision 1.5 4B | 0.798 | 0.758 | 0.437 | 0.591 | 0.646 |
It is best on routing and close to the best on intent, and behind on the guardrail and on Vietnamese knowledge.
Jev Decision Index 0.3 (full split): 60.67 in our own run of the public harness. For reference, the public board lists Valen-Decision 1.5 at 69.01, KnowLine Gen5.5 at 66.76 and Decider Flash v1.3 at 63.35; StartLux-Decision v1.1 reports 67.88 on its model card.
A stronger model on the same architecture, BunGPT-DeepEdu-Decision-4B-Pro, is described in our technical report; its weights are not released.
Limitations
- Guardrail. At the default cutoff it lets more sensitive messages through than the other models above (it rarely
blocks safe ones). Choose the
noulcutoff for safety questions from your own labelled data. - Knowledge. Weaker on Vietnamese legal and general-knowledge questions than the other models above; it does not know what it was not taught.
- General decisions. On the broad Decision Index it is behind the strongest 4B models; it is tuned for education product decisions.
- Scores a fixed set of options; it is not a chat model. Inputs up to the server's
--max_length(default 16,384 tokens).
Files
model-0000{1,2}-of-00002.safetensors,config.json, tokenizer files,chat_template.jinja: the backbone.score_head.safetensors,r2_config.json: the scoring head and its settings.source/: thedeepedu_decisioninference package (Python API and server), Apache-2.0.
License
Apache-2.0. Built on Qwen3.5-4B (Apache-2.0); see NOTICE.
- Downloads last month
- 9