maximilianofir/newton-newreward-ram-fixed50-e8m8

SO-101 vial placement — new reward, eight-environment PPO

GR00T N1.7 policy trained for 50 RLinf PPO iterations from the SFT parent. Training run b10c4877 finished on October 16, 2026. The instruction is “Pick up the vial and place it in the rack”; the executed action horizon is 16 controls.

The Newton/Isaac Lab task uses three vials and standard workshop domain randomization: vial and rack poses, mat rotation, robot base tilt, external camera tilt/zoom, light exposure, and HDRI lighting. Only vials starting on the mat are eligible. Success requires upright, seated, released placement in a rack slot for ten consecutive controls; returning home is not required. The deadline is 640 controls (about 21.3 seconds).

Reward adds ordered, one-time reach/lift/hover/insert bonuses of 0.02/0.08/0.05/0.15, each confirmed for three controls, to terminal rewards of +1 for success and −1 for unsuccessful termination. These shaping bonuses are history-dependent and do not imply policy-invariance guarantees.

Training

Setting	Value
Collection	8 environments × 4 rounds × 640 controls; 16 controls/chunk
PPO batches	1,280 fresh chunks/iteration; global minibatch 640; microbatch 8; two PPO passes
Budget	50 iterations; 64,000 fresh chunks; 1,024,000 control slots; 200 configured optimizer updates
Actor / critic learning rate	1e-6 / 1e-4
PPO / GAE	Clip 0.2; chunk discount 0.99; lambda 0.95; normalized advantages
Real-data regularization	Flow-matching loss weight 0.2; batch 8; episodes 0–9 of sreetz-nv/so101_real_manual
Exploration	Flow-SDE noise 0.25; four denoising steps
Critic warmup	None; fresh critic and optimizer
Monitoring / checkpoints	Every five iterations; 20 evaluation episodes
Seeds	Actor/rollout 152; training environment 42; evaluation scenes 18317–18336
Hardware / logged runtime	One RTX PRO 6000 Blackwell Server Edition; approximately 7 h 51 min

Control-slot totals include held terminal suffixes. Full action-head/projector tuning is configured, with LoRA disabled and visual/language backbone tuning disabled; exact trainability and precision settings are in training provenance. Any inherited optimizer fields in the deployment config.json describe the SFT parent, not this RL run.

Simulation results

Evaluation	Success	Timeouts	Safety failures
SFT parent baseline	15/20 (75%)	5	0
Final native training evaluation, iteration 50	15/20 (75%)	5	0
Fresh native reload of iteration 50	17/20 (85%)	2	1

The baseline uses the same task, scene seeds, action horizon, deadline, and stochastic evaluation settings. The final native score matches the baseline; the reload has two additional successes. Separate stochastic evaluations of twenty episodes do not establish a reliable improvement and are not pooled. The same twenty scene seeds were reused for training monitoring; this is not an independent held-out test.

These are native RLinf simulation results. Exported LeRobot policy-server task performance, the standard 100-episode Newton benchmark, and real-robot success have not been measured for this checkpoint. Full monitoring history and protocol are in evaluation.json.

Raw plotted metrics · Evaluation protocol and results · Training provenance · Curves PDF

Deployment

The model ID is maximilianofir/newton-newreward-ram-fixed50-e8m8. Use the workshop's GR00T N1.7 policy server with action horizon 16 and the instruction above. Preserve the parent processors, normalization, camera identities, and measured SO-101 joint/gripper calibration. Native RLinf exploration and the value head are training components; the deployment actor uses the LeRobot inference path. No real-robot performance claim is made.

The export preserves all 1,030 serialized actor tensors exactly, omits the RL critic, and deduplicates identical tied embedding storage. Parent processors and normalization assets match byte for byte. See the tensor and processor verification and frozen source hashes. Ordinary LeRobot strict loading and four-step inference passed on one recorded real observation, producing a finite 1 × 16 × 6 decoded action chunk; see the load and inference report. Stored weights are FP32 and inference uses BF16. This check did not connect to a robot or measure task success.

License

The bundled NVIDIA License is copied unchanged from the model lineage at GR00T N1.7 revision 2fc962b. Its use terms apply to the model weights and are distinct from the training-code license.