Pls release cheap mech interp tools to monitor this model

#23
by john1248 - opened

Overall, we find that looping increases what these models can do without a verbalized chain of thought, but at this scale we do not see a meaningful reduction in the monitorability of their reasoning
(source https://thinkingmachines.ai/news/inkling-2-mini/#monitorability-of-looped-transformers)

I find that j-lenses often surface unverbalized reasoning from qwen3.8-27b, which isn't even looped. you say that you did on-policy SFT stuff to align the model better, but the combination of this and opaqueness leads to aligned-looking models that may make subtle decisions against the user (for example, when creating a color scheme for my pi config, qwen3.8-27b chose colors that make seeing the commands it writes very difficult. its not hard to see how this is a serious problem)

this model is very helpful and fast, but its very hard to trust it while monitorability is (very probably) compromised.

j-lens would cost <20 bucks. NLAs would cost <50. this would signal trustworthiness while not being costly.

Sign up or log in to comment