FogGen: edge-cloud router for multiple-choice questions

FogGen is Qwen3-0.6B fine-tuned to write a confidence value before its answer. If the confidence is at least a threshold tau, the edge answer is used. Otherwise the query goes to a cloud model (Qwen3-30B-A3B-Instruct-2507). This checkpoint is round 14 (R14) of a 14-round training chain over seven domains: finance, science, coding, law, math, Kazakh culture and medicine. In each round the current model answers each new training question 8 times, and the number of correct answers sets the confidence label it is trained on.

Data, per-round training files and per-question results: issai/foggen-data

Usage

from transformers import AutoTokenizer, AutoModelForCausalLM

model = AutoModelForCausalLM.from_pretrained("issai/foggen", torch_dtype="bfloat16", device_map="auto")
tokenizer = AutoTokenizer.from_pretrained("issai/foggen")

SYSTEM = """You are a self-aware multiple-choice assistant.

Rules:
- Do not output <think> tags.
- First, assess your confidence in solving this question.
- Then give your answer.
- Output format:
  Confidence: <0.0|0.25|0.5|0.75|1.0>
  Final answer: <OPTION_LETTER>"""

question = """A firm reports $400M in total liabilities and $600M in shareholders' equity.
What is the firm's debt-to-equity ratio?

A. 0.67
B. 1.00
C. 1.50
D. 2.00"""

messages = [{"role": "system", "content": SYSTEM}, {"role": "user", "content": question}]
inputs = tokenizer.apply_chat_template(messages, return_tensors="pt", add_generation_prompt=True,
                                       enable_thinking=False).to(model.device)
outputs = model.generate(inputs, max_new_tokens=64, do_sample=False)
print(tokenizer.decode(outputs[0][inputs.shape[-1]:], skip_special_tokens=True))

import re

def route_query(model_output, tau=0.5):
    """Returns (action, confidence, answer). An unparsed confidence is routed to the cloud."""
    conf = re.search(r"Confidence\s*:\s*([\d.]+)", model_output)
    ans = re.search(r"Final\s+answer\s*:\s*([A-D])", model_output)
    if not conf:
        return "route_to_cloud", None, None
    c = float(conf.group(1))
    return ("keep_local" if c >= tau else "route_to_cloud"), c, (ans.group(1) if ans else None)

Results

R14 at tau = 0.5 on the seven evaluation sets (16,217 questions). System accuracy counts the edge answer on kept queries and the cloud answer on routed queries. Random routing sends the same share of queries to the cloud at random. Scores use the corrected answer keys of issai/foggen-data, which leave out 1,582 items whose key could not be fixed.

Domain	Questions	Cloud only	Edge only	Random routing	FogGen	Routed to cloud
Finance	4,425	92.3%	62.3%	68.7%	73.9%	21.5%
Science	1,742	93.6%	61.4%	67.9%	72.2%	20.1%
Coding	1,762	93.5%	64.8%	70.1%	75.0%	18.6%
Law	1,846	87.7%	57.1%	63.2%	67.0%	19.8%
Math	1,703	95.5%	46.6%	69.5%	77.0%	46.8%
Kazakh culture	1,317	96.2%	90.9%	90.9%	91.5%	1.1%
Medical	1,840	88.9%	57.4%	63.8%	68.5%	20.2%
Mean		92.5%	62.9%	70.6%	75.0%	21.2%

FogGen is 4.4 points above random routing at R14 and 5.6 points on average over rounds 1 to 14. The mean expected calibration error is 8.3 at R14. Against the stored keys of the first release, R14 reaches 65.0% against 63.0% for random routing. evaluate.py in issai/foggen-data reproduces every number in this table.

Earlier versions of this card reported 67.8% system accuracy and a lift of 4.6 points. Those numbers credited routed queries with the cloud's average accuracy over the whole set instead of its answers to the routed queries, and they used the stored keys.

Training

Base model	Qwen/Qwen3-0.6B
Cloud model and data generator	Qwen3-30B-A3B-Instruct-2507
Rounds	R0 (1,800-row calibration buffer) and R1 to R14, one domain per round
Rows per round	1,800: up to 300 new-domain rows, 60 replay rows per earlier domain, R0 buffer rows
Confidence label	correct answers in 8 samples at temperature 0.7, mapped to 0, 0.25, 0.5, 0.75 or 1
Fine-tuning	LoRA rank 16, alpha 32, all linear layers, learning rate 5e-5, 2 epochs per round, bf16

Related checkpoints

issai/foggen-qwen3-1.7b, issai/foggen-gemma3-1b and issai/foggen-gemma3-270m run the same chain with other edge models. issai/foggen-r15-oe adds one round of open-ended tasks on top of this checkpoint.