Training data correction (Oct 2026): CyberNative/Code_Vulnerability_Security_DPO labels are unverified

#1
by CyberNative - opened

Hi, we publish CyberNative/Code_Vulnerability_Security_DPO, which this model's card lists as training data. On October 6, 2026 we added a correction notice to that dataset's card. It may matter for how this model's card describes the data.

What the dataset card now says:

  • The chosen and rejected fields are original model-generated preference labels. They are not verified secure/insecure classifications.
  • A separate AI coding agent reviewed 66 randomized A/B pairs before the preference labels were revealed. It found a target-security advantage for chosen in only 5 pairs and for rejected in none; 32 were ties and 29 could not be determined. Most reviewed pairs therefore did not show the security preference their labels imply.
  • These are scoped static judgments, including conditional or latent mechanisms in malformed code, not executed exploit results or a measured whole-dataset insecurity rate.
  • The source file is unchanged. The October 6 audit and limitations and per-row scope flags distinguish 66 independently reviewed pairs, 264 pairs screened only, and 4,326 unreviewed pairs. No row or filtered split is certified safe.
  • Do not use these labels as security ground truth or use the dataset as a validated secure-code training or evaluation set without your own contextual validation.

Your card describes our dataset as "Vulnerable/secure code pairs". If chosen responses were used as examples of the "safe" class, the correction applies to those labels.

We have not evaluated this model and make no claim about it. Questions or corrections are welcome in the dataset's discussions or at hello@cybernative.ai.

CyberNative AI LLC. This note was written with AI assistance.

Sign up or log in to comment