pepijn223/rebot-flux3-subtask-augmented-3500-20260928

ReBot FLUX3 — subtask/goal conditioning, 3,500 updates (EMA)

Final EMA LoRA adapter plus fully trained ReBot heads from the September 2026 ReBot FLUX3 experiment. This is not full-parameter fine-tuning. Its complete required base is public at pepijn223/rebot-flux3-rebot-base-20260928, pinned by adapter_config.json to 15b2bf6d982eb119c4b87a6c2d1c5b69e08a1832. Use the base and this adapter together; the original BFL trunk alone lacks the correct ReBot head topology.

Training

• Original FLUX3 action-pretrained weights: black-forest-labs/flux-3-action-base at 62878e2925e59b7a89ec14463ce89932624c490d.
• Rank/alpha 32 LoRA and fresh fully trained 14-channel action/conditioning heads.
• 136,912,896 trainable parameters of 7,084,045,568 registered policy parameters; frozen encoders are additional.
• 3,500 optimizer updates / 7,000 microsteps; effective batch 128 = 16 per GPU × 4 GPUs × accumulation 2.
• All 100 episodes / 131,716 frames of pepijn223/rebot_diverse_picking_100_annotated, source revision 93c97807c46535745d0587d4296416bf2d4aa80d. About 3.40 nominal frame epochs.
• One caption per frame: 20% augmented episode-goal variants, 80% latest active semantic subtask; selected before tokenization. No subtask-text generation, caption dropout 0.
• BF16, adapter LR 1e-4, head LR 5e-4, gradient clipping 1, EMA decay 0.999, no image augmentation.
• Adapter freeze for 200 optimizer updates, adapter warmup for 600, head warmup for 200.
• Training run.

Observation and action contract

Three views (observation.images.base, observation.images.left_wrist, observation.images.right_wrist) form a 384×512 grid: base top-left, left wrist top-right, right wrist bottom-left, unused cell black. Each view is 192×256 after resizing. One observed frame, no past actions, 32 future action steps at 30 Hz.

Native action order is left then right, each with shoulder_pan, shoulder_lift, elbow_flex, wrist_flex, wrist_yaw, wrist_roll, gripper. Joint outputs represent consecutive command deltas; grippers at indices 6 and 13 remain absolute. Observed joint states are absolute. Use the supplied preprocessing/postprocessing and q01/q99 normalization over all 100 episodes; do not pad/reinterpret as SO-101 actions.

Video and action guidance are both 1 because caption dropout was zero. The default sampler uses 4 inference steps.

Loading

Requires a LeRobot version with FLUX3 support and the flux3 and peft extras. Training used LeRobot main commit e595b7902714ba51f91e47523f66f89c5181b649 plus experiment scripts (local commit 760c84ebc). Match its APIs; older LeRobot releases may not support this architecture.

from peft import PeftConfig, PeftModel
from lerobot.policies.flux3 import Flux3Config, Flux3Policy
from lerobot.policies.factory import make_pre_post_processors

repo = "pepijn223/rebot-flux3-subtask-augmented-3500-20260928"
cfg = Flux3Config.from_pretrained(repo)
cfg.device = "cuda"
adapter_cfg = PeftConfig.from_pretrained(repo)
base = Flux3Policy.from_pretrained(
    adapter_cfg.base_model_name_or_path,
    revision=adapter_cfg.revision,
    config=cfg,
)
policy = PeftModel.from_pretrained(base, repo, is_trainable=False).eval()
preprocessor, postprocessor = make_pre_post_processors(cfg, pretrained_path=repo)

This loads model and processors only; it does not connect to robot hardware. The frozen video VAE and Qwen text encoder resolve from upstream revision 62878e2925e59b7a89ec14463ce89932624c490d. No private cluster paths are required. The EMA tensor file is unchanged from the verified training checkpoint; SHA256 f6db78aaa568cdbbcc19b6cb8e5e0dc1d2c09c082cdc990ac1b6f0a9e98076ff.

Validation and limitations

Saved training state, finite raw/EMA parameters, hashes, adapter/base tensor compatibility, and saved processor loading were checked. This publication is the EMA variant; raw adapters and optimizer state remain archived with the experiment.

All 100 episodes were used for training, including episodes held out in earlier experiments. There is no held-out evaluation for this run and no measured robot task success rate. Training loss is not evidence of picking success. Fresh physical trials are needed; none were run as part of publication.

License and attribution

Weights are distributed under the FLUX Kommunity License v1.0, with the required NOTICE. Rights under that license are granted directly by Black Forest Labs Inc. This is an independent modified model and is not endorsed, approved, or validated by Black Forest Labs. The upstream license file was retrieved at 7f49687d2935d24aebe179c2d03bf7a356e00c0e; the weights remain pinned to 62878e2925e59b7a89ec14463ce89932624c490d.