BunGPT-DeepEdu-Decision-4B-Open

A decision model. It reads a state (a message, a conversation, a document) and one or more typed questions about it, and scores every option of every question in a single pass, without generating text. Use it for the small decisions inside a product: routing a turn, detecting intent, flagging unsafe messages, checking a claim.

  • Backbone: Qwen3.5-4B, causal attention, with a small scoring head that gives each option its own score.
  • Questions: choice (pick one of named options) and noul (yes/no).
  • Output: per question the chosen option and a probability for each option, plus the raw option scores.
  • Serving: a Python API and an HTTP server compatible with the Jev Decision Index POST /v1/systemone format.

Quickstart

huggingface-cli download DeepEduVN/bungpt-open --include "source/*" --local-dir bungpt-open
pip install ./bungpt-open/source
pip install --no-build-isolation -r bungpt-open/source/requirements-fast.txt   # optional: faster kernels

If the repository is private, set HF_TOKEN to a token with read access first. The weights are downloaded on the first from_pretrained call.

from deepedu_decision import DecisionModel

model = DecisionModel.from_pretrained("DeepEduVN/bungpt-open", device="cuda:0")

state = "Hแปc sinh: Em khรดng hiแปƒu vรฌ sao ฤ‘แบกo hร m cแปงa x^2 lร  2x, thแบงy giแบฃi thรญch lแบกi ฤ‘ฦฐแปฃc khรดng แบก?"
questions = {
    "intent": {
        "type": "choice",
        "instructions": "What does the student want?",
        "criteria": {
            "explain": "An explanation of a concept",
            "solve": "A worked solution to a specific exercise",
            "chitchat": "Small talk, nothing to answer",
        },
    },
    "unsafe": {"type": "noul", "instructions": "Is this message harmful or inappropriate for a school setting?"},
}
print(model.decide(state, questions))
{
  "answers": {
    "intent": {"type": "choice", "choice": "explain",
               "probabilities": {"explain": 0.986, "solve": 0.011, "chitchat": 0.002}},
    "unsafe": {"type": "noul", "noul": 0.004}
  },
  "scores": {
    "intent": {"explain": 3.635, "solve": -4.482, "chitchat": -6.13},
    "unsafe": {"false": 5.177, "true": -5.566}
  }
}

decide_batch([...]) takes many requests at once.

HTTP server

CUDA_VISIBLE_DEVICES=0 python -m deepedu_decision.serve --model DeepEduVN/bungpt-open --port 9100
# multi-turn traffic: add --cache_mb 4096 to keep prefix states between requests

POST /v1/systemone takes the Jev Decision Index request {"state": ..., "questions": {...}} and returns its response format (model, answers), so any Decision Index client works unchanged. Two fields are added:

  • scores: the raw score of every option, {question: {option: score}}. Higher is better. Use them for your own thresholds or to compare options across requests.
  • usage: wall_ms, queue_ms, batch_rows, input_tokens, prefills, cache_hit_tokens.

GET /health returns the model name, limits and queue depth.

Errors: a request that does not fit (--max_length, --max_questions, body size) or uses an unsupported question type gets HTTP 422 whose message contains maximum context length; a full queue gets 503 with Retry-After; a malformed body gets 400.

Main flags: --token_budget (tokens per GPU forward, default 16384), --max_length, --max_questions (16), --max_pending (64), --max_batch_rows, --batch_wait_ms, --attention {auto,flash,sdpa}, --no_fork, --cache_mb.

Reading the output

  • Probabilities. For a choice question, option k gets ฯƒ(z_k) / ฮฃ_j ฯƒ(z_j), where z are the raw scores and ฯƒ is the logistic function. For noul, noul is the probability of true.
  • "None of these". Each option is scored on its own, so the raw scores can all be low when no option fits. Use max(scores) (or ฯƒ of it) as a fit signal before trusting the chosen option.
  • Thresholds. For yes/no decisions where one error is worse than the other (for example a guardrail), choose the cutoff on noul from your own labelled data rather than using 0.5.

How it is fast

Attention is causal, so the shared part of a request is computed once: the state is read once for all questions, and each question once for all of its options. Many questions about one long message cost little more than one. With --cache_mb, later turns of a conversation reuse the earlier turns' states.

Median client latency on one RTX 3090, HTTP included (same architecture and server):

Workload Median / p95 (ms)
1 question, 256 tokens 129 / 132
1 question, 4,096 tokens 649 / 698
16 questions, 1,024 tokens 653 / 1,288
16 questions, 4,096 tokens 1,091 / 1,805
Multi-turn, turns 2โ€“6, with prefix cache 173 / 502

Short single questions carry about 80 ms of serving overhead today (no CUDA graphs yet).

Results

DeepEdu benchmark (Vietnamese education product decisions; accuracy or macro-F1). All models are 4B and were run by us on the same records, each with its own published inference code.

Model (all 4B) Intent Routing Guardrail Knowledge Mean
Decider Flash v1.3 0.826 0.800 0.680 0.611 0.729
StartLux-Decision v1.1 Instant 0.824 0.808 0.703 0.568 0.726
KnowLine 4B Gen5.5 0.807 0.808 0.691 0.585 0.723
BunGPT-DeepEdu-Decision-4B-Open 0.822 0.820 0.626 0.539 0.702
Valen-Decision 1.5 4B 0.798 0.758 0.437 0.591 0.646

It is best on routing and close to the best on intent, and behind on the guardrail and on Vietnamese knowledge.

Jev Decision Index 0.3 (full split): 60.67 in our own run of the public harness. For reference, the public board lists Valen-Decision 1.5 at 69.01, KnowLine Gen5.5 at 66.76 and Decider Flash v1.3 at 63.35; StartLux-Decision v1.1 reports 67.88 on its model card.

A stronger model on the same architecture, BunGPT-DeepEdu-Decision-4B-Pro, is described in our technical report; its weights are not released.

Limitations

  • Guardrail. At the default cutoff it lets more sensitive messages through than the other models above (it rarely blocks safe ones). Choose the noul cutoff for safety questions from your own labelled data.
  • Knowledge. Weaker on Vietnamese legal and general-knowledge questions than the other models above; it does not know what it was not taught.
  • General decisions. On the broad Decision Index it is behind the strongest 4B models; it is tuned for education product decisions.
  • Scores a fixed set of options; it is not a chat model. Inputs up to the server's --max_length (default 16,384 tokens).

Files

  • model-0000{1,2}-of-00002.safetensors, config.json, tokenizer files, chat_template.jinja: the backbone.
  • score_head.safetensors, r2_config.json: the scoring head and its settings.
  • source/: the deepedu_decision inference package (Python API and server), Apache-2.0.

License

Apache-2.0. Built on Qwen3.5-4B (Apache-2.0); see NOTICE.

Downloads last month
9
Safetensors
Model size
5B params
Tensor type
BF16
ยท
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for DeepEduVN/bungpt-open

Finetuned
Qwen/Qwen3.5-4B
Finetuned
(1,577)
this model

Space using DeepEduVN/bungpt-open 1