savoji/qwen3.5-0.8b-tablevqa-lora
Updated ā¢ 33
Sajjad Pakdamansavoji
savoji
AI & ML interests
VLM, VLA, LLM, Robotics, Computer Vision
Recent Activity
posted an update about 4 hours ago
SLA reaches 72.3% on RoboTwin 2.0 Clean2Random against 46.0% for the pi0.5 it starts from. It does this by aligning VLA visual features to a frozen 3D model in both continual pre-training and post-training.
2D VLAs inherit vision from image-text pretraining. In the authors' depth probe, unaligned VLA embeddings show no meaningful spatial structure. SLA adds a cosine alignment loss to OmniVGGT features: L = L_action + alpha * L_align. The teacher, projector and gates are dropped after training, so inference needs no depth or calibration.
š The recipe
ā¢ Gates mix visual embeddings from all VLA layers; state dropout stops the policy shortcutting on robot state
ā¢ Data: 6 public datasets, about 1.7M episodes, 14,330 h; delta end-effector actions as rotation vectors (Euler subtraction gives outliers near +/-2pi); episode downsampling; no dexterous hands or base motion
ā¢ 36k steps, batch 4,608, about 110 h on 48 H100s
š The numbers
ā¢ Clean2Random: SLA 72.3, Spatial Forcing (post-training only) 62.6, pi0.5 46.0
ā¢ LIBERO-Plus 87.4 vs 84.4; RoboDojo-Real 33.9 vs 12.8
ā¢ RoboCasa-Panda 68.4 with 50 demos per task
ā ļø Caveats
ā¢ Per-proportion ablation numbers are only in Fig 4; the text says both stages win at every tested proportion
ā¢ ME-Dex-1.0 beats it on Clean2Clean (89.6 vs 86.0); the LIBERO-Plus lead over Qwen-RobotManip is 0.3
ā¢ The eight real-world conditions are private
https://huggingface.co/papers/2611.08373
https://arxiv.org/abs/2611.08373
Would you add a training-only 3D alignment loss to your VLA, and in which stage?
š Read it elsewhere
ā¢ The deep dive on Medium: https://medium.com/p/32363a577404
ā¢ The long read on Substack: https://sajjadpakdamansavoji.substack.com/p/sla-aligns-a-vla-to-a-frozen-3d-model
ā¢ The thread on X: https://x.com/saavoji/status/2119503826579542064
ā¢ Every version on sajjadpsavoji.github.io: https://sajjadpsavoji.github.io/blog#sla-teaches-pi0 posted an update 1 day ago
Idol, a new Apache-2.0 image-to-3D model from @speridlabs, makes a 3D Gaussian splat from one image in 7.2 GB of peak VRAM. It is the #1 trending model on the Hub with 1,057 likes.
It writes a standard 3DGS .ply with up to 1,048,576 Gaussians, including sides the picture never showed, and does background removal and framing itself. The catch: the only quality evidence is the authors' own benchmark, on a repo created 2026-10-28.
š The numbers
ā¢ Authors' benchmark, 1 x H100: best in 6 of 8 columns vs TripoSplat, TRELLIS and LGM (SSIM 0.726, LPIPS 0.213, DINOv2 0.781, CLIP 0.885)
ā¢ LGM wins PSNR (12.46 vs 12.25) and speed (1.1 s). Idol takes 6.8 s vs 7.1 s for TRELLIS and 9.9 s for TripoSplat
ā¢ 262,144 Gaussians, 50 steps: 3.4 s on B300, 6.7 s on H100, 19.1 s on RTX 4090, 46.2 s on a 10 GB RTX 3080
š ļø Try it
ā¢ Install idol-sdk, accept the terms of the gated DINOv3 and FLUX.2-dev models, run hf auth login
ā¢ python -m idol.sample --image photo.jpg --mode real
ā¢ --num_gaussians runs from 32,768 (0.4 MB) to 1,048,576 (12.2 MB)
ā ļø Caveats
ā¢ Test set and size are not named, and no independent evaluation exists yet
ā¢ The card says "under 2 seconds", but the timing table shows 3.4 s at best
ā¢ Single objects only, NVIDIA CUDA required, no failure cases shown
https://huggingface.co/speridlabs/idol
https://speridlabs.com/research/idol
If you run it on your own messy photos, how do the unseen sides hold up?
š Read it elsewhere
ā¢ The deep dive on Medium: https://medium.com/p/5f660ae6ae56
ā¢ The long read on Substack: https://sajjadpakdamansavoji.substack.com/p/speridlabs-idol-builds-3d-splats
ā¢ The thread on X: https://x.com/saavoji/status/2119137739580858545
ā¢ Every version on sajjadpsavoji.github.io: https://sajjadpsavoji.github.io/blog#speridlabs-idol posted an update 2 days ago
VFlash cuts per-step latency 5.1 to 5.2x on Jetson Thor and DGX Spark, and its one-pass drafter beats the autoregressive 4B model it drafts for on 3D robot trajectories.
A planner does not have to write points, boxes and trajectories one coordinate token at a time. A primitive's type, dimensionality and length fix its layout before decoding, so a lightweight drafter fills the whole primitive in one forward pass. The decoder accepts it if it is structurally valid, with no token agreement with the target. Text keeps verified speculative decoding.
š Why token matching fails
ā¢ DFlash commits 5.5 tokens per iteration on subtask text but only 2.2 inside trajectory blocks, so it only about halves decoding time
ā¢ A coordinate one bin off fails the check even when the geometric error is small
š The numbers
ā¢ AgiBot waypoints within 5 cm: 50% to 73% (3D only) and 40% to 74% (2D+3D); ADE falls 18 to 61% across InternA1, AgiBot and MolmoBot
ā¢ Under AR, decoding is 75 to 87% of a step on Thor; VFlash cuts it 12 to 42x. H100 gets 3.6x
ā¢ MolmoSpaces success: 70.2% with VFlash plans, 62.6% baseline, 61.0% with AR plans
ā ļø Caveats
ā¢ Latency is a pipeline measurement at batch size one, with a 4B target and 36 to 84 token primitives
ā¢ Boxes need an AR fallback: without it, mean F1 drops about 20 points on Dense200
ā¢ The 25x dense point motion figure is preliminary, and accuracy there is mixed
https://huggingface.co/papers/2611.08449
https://arxiv.org/abs/2611.08449
If your planner's cost is mostly coordinate decoding, would you swap token-level verification for a structure validator?
š Read it elsewhere
ā¢ The deep dive on Medium: https://medium.com/p/26ed0ca0bb84
ā¢ The long read on Substack: https://sajjadpakdamansavoji.substack.com/p/vflash-drafts-whole-robot-plans-in
ā¢ The thread on X: https://x.com/saavoji/status/2118773042503623062
ā¢ Every version on sajjadpsavoji.github.io: https://sajjadpsavoji.github.io/blog#vflash-draftsOrganizations
None yet