sft_robodojo_vanilla48k_eefabs_f33fps8_openwam_aligned_videoaug

Model weights only of the RoboDojo ARX-X5 EEF-only policy SFT on the OpenWAM canvas (H 384 x W 320) with strided video (video_fps 8), absolute EEF targets, the aligned RoPE and video augmentation on half of the samples, warm-started from the vanilla robot pretrain logits/sana_rwm_pretrained_vanilla_e9s48000 (wandb lzknus/sana-rwm/sft_robodojo_eefabs_openwam_384x320_vanilla48k_aligned_f33fps8_videoaug), at checkpoint epoch_10_step_36180 (epoch 10, optimizer step 36,180 = the full 36,180-step budget, 10 epochs). Final checkpoint of the finished run (Slurm array 19351080, training reached step 36,180 on 2026-09-28; uploaded 2026-09-28). Its canvas twin on the same donor and recipe is logits/sft_robodojo_vanilla48k_eefabs_f33fps8_sana_pixel_aligned_videoaug.

Files

file	bytes	sha256
model/pytorch_model_fsdp.bin	17874895202	7c519e4aa6bb807f8736493f98f8c736c06f4c8ef2d042f53762c4c7f25b39c2
metadata.pth	37141	7dfb717b5842e37d9a28187652754a5977d31a772f9c04f638eba1b72f5141b4
config.yaml	12241	d5caf7c5983b124bcbbfced669c4784961f7151c3a10ffe6c61efd1b3b4735a6
normalization/robodojo_arx_x5_model_fps_25_f33_normalization.json	45608	1fe3b7e72e8fa93eca5efb8a8deeace1daf4434b4b97e9f4445670ac758287ed

• model/pytorch_model_fsdp.bin: accelerate FSDP consolidated state dict, 805 tensors, 4,468,977,840 parameters (804 float32, 1 bfloat16). The eight robot-module tensors (state_embed.proj.*, action_embed.proj.*, action_head.*, plucker_embed.weight) came with the robot-pretrained donor and were fine-tuned here (the donor loaded strictly, missing only pos_embed); pos_embed present.
• metadata.pth: epoch / step / scheduler / RNG bookkeeping read next to the weights.
• config.yaml: the trainer's frozen, fully resolved training config, unchanged (paths are cluster-local); it declares model.extra.rope: aligned and the train.extra.video_augmentation block.
• normalization/...json: the robot80 normalization artifact the run trained with (sha256 1fe3b7e72e8fa93eca5efb8a8deeace1daf4434b4b97e9f4445670ac758287ed; f33, absolute targets, frame-aligned corpus). Grippers are q01/q99-normalized too (closedness [0, 1] -> [-1, 1]).
• Not included: model/optimizer.bin, model/scheduler.bin, random_states_*.pkl, the training log (weights only).

Recipe (from config.yaml)

• Model SanaRWMVideoQwenNextSubAttnResV2SelfFlowWorldModelCameraConditionMultiViewPolicy_5B_P1_D36 (32 blocks, softmax attention every 4th, GatedDeltaNet elsewhere), bf16, fp32 attention; data.extra.multiview: openwam. Training code: rwm/zekai-merge 7070353ec.
• Donor model.load_from: sana_rwm_pretrained_vanilla_e9s48000 (Hub logits/sana_rwm_pretrained_vanilla_e9s48000: the 256px vanilla absolute EEF/joint unified robot pretrain at 7.5 fps, epoch 9 step 48000).
• Visual stream: ONE OpenWAM canvas per frame. The three cameras are composited in pixel space into the OpenWAM L-shape RGB canvas of H 384 x W 320 (openwam_canvas.layout openwam_lshape_rgb_v1, encode_mode joint_rgb_canvas, aspect_ratio_type ASPECT_RATIO_OPENWAM_LSHAPE_384_320; every raw frame stretched whole into its slot) BEFORE the LTX-2.3 VAE, so the policy sees one view (V = 1, plain mRoPE) on a 12x10 latent grid. No camera conditioning.
• Video augmentation (train.extra.video_augmentation): crop_scale 0.95, brightness 0.3, contrast 0.4, saturation 0.5, hue 0.08, apply_prob 0.5: on the GPU before the VAE, one draw per sample shared by all its frames, applied to the composited canvas as one image; about half of every batch is left untouched. The VAE encodes online (no latent store). Validation is clean.
• Windows: 33 source rows at video_fps 8 (frame stride 4): the observation frame plus 8 sampled frames = 9 canvas frames = 2 latent frames, while the actions stay dense: 32 action rows at 25 fps.
• RoPE: aligned (model.extra.rope). Video and actions share one physical clock in base-fps (16) latent-frame units: video latent j at 16 * j * 4 / 25 (0 and 2.56), action row k at 16 * k / (8 * 25) = 0.08 k (0.08 .. 2.56), the state at 0.
• Targets: EEF-only (action_mode_sample_ratio [0.0, 1.0, 0.0], robot_base_eef: both arms' EEF position + Rot6D in the robot base frame plus the grippers), eef_target_mode absolute. Normalization pin 1fe3b7e7....
• Data contract: frame-aligned. All 3,500 episodes (holdout_episodes_per_task_split 0), tail windows (min_rows 2, padding freeze): 1,744,102 full + 108,500 tail = 1,852,602 windows, 3,618 steps per epoch at 512 windows per step (8 nodes x 8 GPUs x bs 8).
• Text contract: G = 1, ONE shared prompt (the composite view's: embodiment, action mode, the canvas layout, instruction); instruction dropout 0.1 per scene.
• Noise schedule (the 2026-09-24 SFT default): flow shift 5.0 standard for the video, a separate action flow shift 1.0 on the same raw timestep draw, inference 5.0 / 1.0, OpenWAM timestep loss weighting with min_weight 0.1, min_train_timestep 1.
• Optimizer: AdamW peak lr 0.0001 after 2,000 warmup steps, cosine to 1e-06 over 36,180 steps, weight decay 0.0001 on weight matrices only, grad clip 1.0, action_loss_weight 1.0.

Validation (seen-episode monitor: 35 tasks x 1 episode the model trained on; normalized masked action MSE)

The run had no milestone watcher; the final checkpoint was validated from the training code (rwm/zekai-merge 7070353ec, CFG off, 50 steps):

ckpt step	n	mean	median	max	tasks > 0.2
36,180	35	0.0036	0.0003	0.0617	0

Validated again from rwm/zekai-merge 77cf81fbf with this config.yaml: 35/35 samples bitwise equal (predictions, targets, masks and MSE); the OpenWAM pixel path did not change after the training code.

Loading

--model.load_from=<local dir holding model/ and metadata.pth> for the Sana-RWM trainers and validator on rwm/zekai-merge 7070353ec or later (verified bitwise at 77cf81fbf); the bidirectional deploy takes the same directory with config.yaml and normalization/...f33_normalization.json. The model output is the normalized action; saturate the gripper closedness to [0, 1] after denormalizing, not before.