logits/sft_robodojo_eefabs_sana_pixel_320x512_vanilla34k_aligned_f33fps8

sft_robodojo_vanilla34k_eefabs_f33fps8_sana_pixel_aligned

Model weights only of the RoboDojo ARX-X5 EEF-only policy SFT on the sana_pixel 2x2 pixel canvas at 320x512 with strided video (video_fps 8), absolute EEF targets and the aligned RoPE, warm-started from the vanilla robot pretrain logits/sana_rwm_pretrained_vanilla_e6s34500 (wandb lzknus/sana-rwm/sft_robodojo_eefabs_sana_pixel_320x512_vanilla34k_aligned_f33fps8), at checkpoint epoch_10_step_36180 (epoch 10, optimizer step 36,180 = the full 36,180-step budget, 10 epochs). Final checkpoint of the finished run (Slurm array 19285791, training reached step 36,180 on 2026-09-25 PDT; uploaded 2026-09-26). Its canvas twin on the same donor and recipe is logits/sft_robodojo_vanilla34k_eefabs_f33fps8_openwam_aligned.

Files

file	bytes	sha256
model/pytorch_model_fsdp.bin	17874895202	36a083ebb16b58aae5d6f0c4d5db4d3468c8a0391be1e2ae92bf0bc06cf84112
metadata.pth	37269	cb70c60e927f6f0e350f56423cd4755b6c78f1c59676f39ab4cff1f93b1e7f7c
config.yaml	11814	d2b0d62dcdb066fc13d1bf4d74a2a30031a1318bea42e1f4825cf58e69f6cb56
normalization/robodojo_arx_x5_model_fps_25_f33_normalization.json	45608	1fe3b7e72e8fa93eca5efb8a8deeace1daf4434b4b97e9f4445670ac758287ed

• model/pytorch_model_fsdp.bin: accelerate FSDP consolidated state dict, 805 tensors, 4,468,977,840 parameters (804 float32, 1 bfloat16). The eight robot-module tensors (state_embed.proj.*, action_embed.proj.*, action_head.*, plucker_embed.weight) came with the robot-pretrained donor and were fine-tuned here (the donor loaded strictly, missing only pos_embed); pos_embed present.
• metadata.pth: epoch / step / scheduler / RNG bookkeeping read next to the weights.
• config.yaml: the trainer's frozen, fully resolved training config, unchanged (paths are cluster-local). It already uses the current data.extra.robot_sft block and declares model.extra.rope: aligned; it parses with rwm/zekai-merge b86f8a2f9.
• normalization/...json: the robot80 normalization artifact the run trained with (sha256 1fe3b7e72e8fa93eca5efb8a8deeace1daf4434b4b97e9f4445670ac758287ed; f33, absolute targets, frame-aligned corpus). Grippers are q01/q99-normalized too (closedness [0, 1] -> [-1, 1]).
• Not included: model/optimizer.bin, model/scheduler.bin, random_states_*.pkl, the training log (weights only).

Recipe (from config.yaml)

• Model SanaRWMVideoQwenNextSubAttnResV2SelfFlowWorldModelCameraConditionMultiViewPolicy_5B_P1_D36 (32 blocks, softmax attention every 4th, GatedDeltaNet elsewhere), bf16, fp32 attention; data.extra.multiview: sana_pixel.
• Donor model.load_from: sana_rwm_pretrained_vanilla_e6s34500 (Hub logits/sana_rwm_pretrained_vanilla_e6s34500: the 256px vanilla absolute EEF/joint unified robot pretrain at 7.5 fps, epoch 6 step 34500).
• Visual stream: ONE sana_pixel canvas per frame. The three cameras are composited in pixel space into one 320x512 RGB image of 160x256 tiles (head top-left, left wrist bottom-left, right wrist bottom-right, the unused top-right quadrant black at -1.0, aspect_ratio_type ASPECT_RATIO_SANA_PIXEL_2X2_320_512) BEFORE the LTX-2.3 VAE; 160x256 tiles are whole 32x32 VAE cells, so the policy sees one view (V = 1, plain mRoPE) on a 10x16 latent grid, the black quadrant a 5x8 block that stays in the video loss. No camera conditioning. Latents were read from the verified stores sana-pixel-rgb-v1-f33-vs4-bf1a2eeb5f2dafc4 + tail sana-pixel-rgb-v1-f33-vs4-tail-a814d255156e25cb.
• Windows: 33 source rows at video_fps 8 (frame stride 4): the observation frame plus 8 sampled frames = 9 canvas frames = 2 latent frames, while the actions stay dense: 32 action rows at 25 fps.
• RoPE: aligned (model.extra.rope). Video and actions share one physical clock in base-fps (16) latent-frame units: video latent j at 16 * j * 4 / 25 (0 and 2.56), action row k at 16 * k / (8 * 25) = 0.08 k (0.08 .. 2.56), the state at 0.
• Targets: EEF-only (action_mode_sample_ratio [0.0, 1.0, 0.0], robot_base_eef: both arms' EEF position + Rot6D in the robot base frame plus the grippers), eef_target_mode absolute. Normalization pin 1fe3b7e7....
• Data contract: frame-aligned. Video frame r is paired with source row r-1 (the official RoboDojo one-frame shift), state rows are the measured state of the current frame. All 3,500 episodes (holdout_episodes_per_task_split 0), tail windows (min_rows 2, padding freeze): 1,744,102 full + 108,500 tail = 1,852,602 windows, 3,618 steps per epoch at 512 windows per step (8 nodes x 8 GPUs x bs 8).
• Text contract: G = 1, ONE shared prompt (the composite view's: embodiment, action mode, the canvas layout, instruction); instruction dropout 0.1 per scene (the unconditional row drops the Instruction line).
• Noise schedule (the 2026-09-24 SFT default, rwm/yuyang's): flow shift 5.0 standard for the video, a separate action flow shift 1.0 on the same raw timestep draw, inference 5.0 / 1.0, OpenWAM timestep loss weighting with min_weight 0.1, min_train_timestep 1.
• Optimizer: AdamW peak lr 0.0001 after 2,000 warmup steps, cosine to 1e-06 over 36,180 steps, weight decay 0.0001 on weight matrices only, grad clip 1.0, action_loss_weight 1.0.

Validation (seen-episode monitor: 35 tasks x 1 episode the model trained on; normalized masked action MSE)

Milestones validated by the run's watcher from the training code (rwm/zekai-merge 5295c208d, sampler without the old gripper clamp, CFG off, 50 steps):

ckpt step	n	mean	median	max	tasks > 0.2
20,000	35	0.0033	0.0004	0.0821	0
25,000	35	0.0027	0.0003	0.0649	0
30,000	35	0.0022	0.0002	0.0600	0
35,000	35	0.0004	0.0001	0.0044	0
36,180	35	0.0004	0.0001	0.0051	0

Steps 5,000 / 10,000 / 15,000 have no row: the milestone watcher waited in the queue until about step 19,000 and those rolling checkpoints had already been rotated out. The final checkpoint was validated again from rwm/zekai-merge b86f8a2f9 with this config.yaml: 35/35 samples bitwise equal to the watcher's (predictions, targets, masks and MSE).

Loading

--model.load_from=<local dir holding model/ and metadata.pth> for the Sana-RWM trainers and validator on rwm/zekai-merge 875b3f659 or later (the config's data.extra.robot_sft block); the bidirectional deploy takes the same directory with config.yaml and normalization/...f33_normalization.json. The model output is the normalized action; saturate the gripper closedness to [0, 1] after denormalizing, not before.