The lever is the harness. The same Qwen scores 1.90 with RoboProbe, 7.75 with RPent, 21.01 with GAP (Hybrid) and 31.39 with uni-robot-agent. Under that harness GPT-6 Astra reaches 51.90, so a gap to closed models remains.
š The numbers
⢠Qwen costs $1.09 per episode; Astra costs $21.29
⢠Best VLA/WAM baseline, Xiaomi-Robotics-1: 20.07 Score
⢠Real robot, 19 tabletop tasks: Qwen 49.50, fine-tuned pi0.5 47.75, zero-shot Dream-1 35.50
⢠No depth keeps 87.5% of Qwen's score; RGB only without camera parameters drops it to 9.13
š ļø Try it
⢠pip install uni-robot-agent, then uni-robot-agent setup --integration robodojo (MIT, Python 3.11, Linux GPU host)
⢠Reusable ideas: immutable observation snapshots, duplicate-action rejection, pipelined execution (84.5% to 93.8% of motion overlaps model calls)
ā ļø Caveats
⢠Scores are low in absolute terms; Qwen needs 95.5 min to reach a score of 25, Astra 42.6
⢠The package has no physical-robot support yet; the real-world benchmark is in-house on one dual-arm platform
Democratizing Robot-Use Agents: A Unified Harness with Open Models (2611.01560)
https://github.com/Open-RUA/uni-robot-agent
https://arxiv.org/abs/2611.01560
Before buying depth sensors or a frontier API, how much of your robot agent's gap is really the harness?
š Read it elsewhere
⢠The deep dive on Medium: https://medium.com/p/51efcc243344
⢠The long read on Substack: https://sajjadpakdamansavoji.substack.com/p/open-qwen38-flash-next-beats-gpt
⢠The thread on X: https://x.com/saavoji/status/2117670449111007255
⢠Every version on sajjadpsavoji.github.io: https://sajjadpsavoji.github.io/blog#qwen3-8-flash