qwen3.5-4b-base-finance-head-200k

Paper: S²-MAS. The fixed-budget adaptation results and matched downstream methods define the comparison reported here.

Project background

S²-MAS is a society-scale multi-agent simulation framework for studying how household decisions affect consumption, cash reserves and payments. The paper connects a constructed resource of 500 million household-persona records, learned decision policies, reusable financial scenarios and a state-transition engine. Its purchase-funding application evaluates 12 million distinct synthetic household agents under paired conditions. Full-data Fin-Laya supplies the policy for that application; this repository supplies a fixed-budget adaptation checkpoint for the common decision-model comparison. Decision learning uses real-survey data. The GENESE/MAS engine implementation remains private.

The benchmark groups twelve systems into six decision-model routes: encoder option scoring; language models with learned decision heads; next-token decision SFT; dynamic-schema classification; multimodal encoder decisions; and provider decision interfaces. All receive text inputs in the common benchmark. The 17 model-repository entries comprise 10 fixed-budget trained variants—five native Laya and five Qwen variants—plus six checkpoint/runtime entries and one compatibility alias.

Test-set QA examples

These four QA excerpts come from the existing held-out consumer financial-choice test set. They retain the original English questions, candidate menus and recorded reference answers. Full household states and original identifiers are omitted. The answers belong to the households in the original records; provide a household state when running inference.

Download the examples as JSON.

1. Active credit-card use

Does your household use credit cards? Exclude cards that have not been activated.

1 — Yes
2 — No

Recorded reference answer: 2 — No

2. Demand-deposit account

Does your household currently have a renminbi demand-deposit account?

1 — Yes
2 — No

Recorded reference answer: 1 — Yes

3. Stock trading account

Does your household currently have a stock trading account?

1 — Yes
2 — No

Recorded reference answer: 2 — No

4. Unmatured time deposits

Does your household currently have any renminbi time deposits that have not yet matured?

1 — Yes
2 — No

Recorded reference answer: 2 — No

This model's role

This checkpoint follows the language-model plus learned decision-head route. The official Qwen3.5-4B-Base backbone stays frozen while a shared scalar head learns to score independently encoded candidates. It provides the 200k frozen-backbone reference for the matched LoRA comparison.

Training design and checkpoint

Method: head_only.
Distinct downstream training rows: 200,000 from the fixed decision-QA training partition.
Initialization: Qwen/Qwen3.5-4B-Base.
This stage terminal row exposures: 800,000; selected checkpoint stage exposures: 200,000.
Selected checkpoint cumulative row exposures: 200,000; selected cumulative distinct rows: 200,000.
A fixed 20,000-example development set selects this checkpoint. The complete 412,873-example development partition compares the two 200k Qwen4B adaptation methods.

Performance and matched comparisons

Language: English. Inference precision: BF16 Qwen backbone with FP32 candidate scoring.

The common held-out test contains 240,458 QA records, 119 question versions and 8,275 respondent/household groups. The checkpoint is selected using development data before this test.

Checkpoint / reference	Accuracy (%)	Question-macro F1 (%)	Invalid decisions
This checkpoint	67.2841	30.8995	0
Qwen4B LoRA + head, same 200k unique QA	74.1435	36.7343	0

Accuracy weights every test record equally; invalid decisions count as incorrect. Question-macro F1 averages over every defined candidate class within each question version and then weights the 119 versions equally.

The matched LoRA checkpoint updates the backbone and shared scalar head using the same unique examples. This frozen-backbone checkpoint isolates learning in the decision head.

Package and inference

A shared scalar decision head on a frozen official backbone; obtain the official backbone separately.

The 4B initialization is the official pre-trained-only Base release.

Model files: head.pt.
Reported common-test compute: BF16 Qwen backbone with FP32 candidate scoring.
Obtain the official Qwen backbone and tokenizer named in the configuration separately.
Inference scores the full state, question and complete candidate menu.

The supplied qwen_decision.py exposes the candidate-scoring inference interface. Supply the local package and the official backbone and tokenizer named in decision_config.json.

License and access

The project model contribution is offered under the S²-MAS Controlled Model Research License v1.2: noncommercial scientific use, no redistribution, and no unauthorized reverse engineering, extraction or distillation. Training and evaluation records are not distributed. Source-data access requires the provider's authorization. Permission requests: yli9919@hku.hk.

Project repository: s2-mas.