SkillScreen-2B-Package-SBR

2B decision model for SKILL.md and related file contents, trained on the recovered SRC010 packages in the original MaliciousSkillBench SBR training split, with a 32K input budget and seed 42.

Formerly SkillScreen-2B-Package-Experimental. The repository name changed; weights and existing v1 / v1.1 tags are unchanged. For the latest all-data deployment release, use SkillScreen-2B-Package.

Based on OpenJev 2B. Full weights are at the root. The original LoRA adapter and classification head are in adapter/.

Source-checkpoint evaluation

Reference results for source LoRA checkpoint package-sbr-s42-r001, seed 42, update 180. These use its original held-out predictions. Validation checks for the merged full-weight export are in merge_verification.json.

• SRC010 recovered-package test: 811 skills, 510 malicious and 301 benign, from the original MaliciousSkillBench SBR test partition. Input is SKILL.md plus eligible companion files, with a 32,768-token budget.
• SkillsBench-1650 external: 1,650 skills, 150 malicious and 1,500 benign, from zenith6888/SkillsBench-1650 at revision 68c0d4bf857f48e5fa4f3baa4613f0c0980a7fb2. Input is the original Markdown plus the provided script_content as a companion file, with the same 32K budget.

Test set	Threshold rule	Native score threshold	Recall (%)	FPR (%)
SRC010 recovered-package test	Default	0.5	99.80	0.66
SRC010 recovered-package test	Target 99% validation recall	0.99748853624125544	99.02	0.33
SkillsBench-1650 external	Default	0.5	66.67	0.40
SkillsBench-1650 external	Target 99% validation recall	0.99748853624125544	32.67	0.13

The native score is the entailment softmax; flag scores greater than or equal to the threshold. Both cutoffs were fixed before test and external scoring. The 99% validation target describes threshold selection; the table reports achieved recall on each test set.

Use

Requires CUDA with BF16 support.

pip install huggingface_hub
hf download SkillScreen/SkillScreen-2B-Package-SBR --revision v1.1 --local-dir skillscreen
pip install -r skillscreen/requirements.txt
python skillscreen/score.py path/to/package.json

Native maliciousness score ≥ 0.5 is flagged as suspicious.

package.json contains Markdown and companion file text.

{"skill_text": "Contents of SKILL.md", "files": [{"path": "scripts/helper.py", "text": "File contents"}]}

Checkpoint details · Export verification · Project page · Code

Jerrold Huang, Junwei Quan, Saiyue Lyu, Zihan Guo, Yingqiang Ge.

Citation

If you use SkillScreen's code, models, or predictions, or build models based on SkillScreen in your research, please cite the SkillScreen paper for the underlying method. Package-SBR is a separate extension; its package-context results are not reported in that paper. If you release a derived model, please identify SkillScreen as its source and link to the original checkpoint.

@article{huang2026skillscreen,
  title   = {SkillScreen: High-Recall Screening of Malicious Agent Skills with Decision Models},
  author  = {Huang, Jerrold and Quan, Junwei and Lyu, Saiyue and Guo, Zihan and Ge, Yingqiang},
  journal = {arXiv preprint arXiv:2610.17727},
  year    = {2026},
  doi     = {10.48550/arXiv.2610.17727},
  url     = {https://arxiv.org/abs/2610.17727}
}

Machine-readable citation metadata is provided in CITATION.cff. These citation requests do not add conditions to the licenses below.

License

Model weights (including the LoRA adapter and classification head), project-authored code, configuration, and code examples are licensed under Apache-2.0, for rights held by the SkillScreen contributors. The model card, documentation, citation metadata, and released provenance/evaluation reports are licensed under CC BY 4.0.

Research and commercial use, fine-tuning, and redistribution are permitted under these licenses without individual permission requests. Preserve the applicable attribution and license notices, and identify modifications as required by each license.

Upstream OpenJev and Qwen rights and notices remain applicable. See THIRD_PARTY_NOTICES.md, LICENSE-OPENJEV, and LICENSE-QWEN.

The v1.1 revision updates licensing and citation information. Its model weights, adapters, tokenizer, scoring code, and configurations are unchanged from v1.