Track4D-2026

Public model adapters and exact evaluation assets for Track4D, a conditional-video approach to dense 3D tracking. The repository contains separate RTV, NCMV, and Mixture-of-Transformers (MoT) model lines. Their checkpoints are not interchangeable.

Model adapters

File	Model and intended use
mot/track4d_mot_wan22_5b_recovery1000.safetensors	MoT recovery1000 adapter for Wan2.2-TI2V-5B; dense RGB + geometry tracking. Contains 1,475 trained tensors (588 Wan, 822 MoT, 62 DPT, 3 XYZ encoder). It is not a standalone base model and contains no optimizer state.
ncmv/wan21_t2v_14b_512x512x49_lora_r64_step4000.safetensors	NCMV-only Wan2.1-T2V-14B adapter, native 512 × 512 × 49.
rtv/joint_rtv_ncmv_wan21_i2v_14b_lora_r64_step5000.safetensors	Joint RTV + NCMV Wan2.1-I2V-14B adapter, trained with 49-frame diverse-resolution data.
rtv/rtv_only_noqi_wan21_i2v_14b_lora_r64_step9000.safetensors	RTV-only/noQI Wan2.1-I2V-14B adapter, trained with 49-frame diverse-resolution data.
experimental/da3_vdm_vitdpt_step016000/	Retained Wan LoRA, ViT/DPT probe, and DA3 LoRA for the live-DA3 3D pipeline.
experimental/dense2d_step013500/	Retained Wan LoRA and ViT/DPT probe for dense UV tracking plus visibility.

The MoT recovery1000 adapter SHA256 is adb6584dee2d640c299dc8b9fc84ae1377ad46ba6a977960c55536dd7ca5bc26 and its size is 7,129,090,808 bytes. The exact byte sizes and hashes for all released adapters are in checkpoint_manifest.json.

MoT inference

The public GitHub repository includes the strict model loader, inference protocol, and versioned config. With the separately obtained Wan2.2-TI2V-5B base model, T5 encoder/tokenizer, VAE, and prepared geometry cache installed, the config resolves the MoT adapter from this repository automatically:

python scripts/train/launch_v2v.py \
  configs/experiments/infer_mot_recovery1000_worldtrack49.yaml

Set MOT_CHECKPOINT=/path/to/local.safetensors to override the hosted adapter with a local file. The strict loader verifies the trained topology. Model design, evaluation protocol, and limitations are documented in docs/MOT.md.

Wan base models, geometry models, and datasets are obtained separately. The MoT Omega-DPT component retains its upstream FAIR Noncommercial Research License; see third_party/ in the code repository. This Hub adapter does not grant rights to redistribute or use third-party base models beyond their own terms.

RTV/NCMV evaluation assets

The code repository downloads each group selectively with scripts/download_eval_assets.py:

• evaluation/worldtrack_release/: 208 benchmark input/GT NPZ files.
• evaluation/stream3r_short256_49f/: 200 aligned RGB/geometry NPZ files for the stable RTV replay.
• evaluation/da3base_step012500_predictions/: exact historical 3D predictions plus evaluator JSON/TXT.
• evaluation/dense2d_step011500/: five-source metric JSONs and qualitative NPZ/MP4 examples.

For the released WorldTrack RTV scorecard, short256 is both the aligned membership and the short-side-256 Stream3R scaffold contract. ADT uses 256 × 256; PO, PStudio, and DS/DR preserve landscape aspect and use the 448 × 256 model bucket. Joint RTV+NCMV uses one denoising step; RTV-only uses 50. Exact replay uses base seed 42 plus the global video index over the fixed ADT, PO, PStudio, DS/DR order.

The joint RTV+NCMV model is a continuation from RTV-only. Its historical score improvement describes the complete two-stage recipe and is not an isolated causal estimate of the NCMV loss. Old experimental 3D step-12.5K and dense-2D step-11.5K model weights were rotated during continued training; saved outputs reproduce their published numbers, while the retained 16K/13.5K weights are not misidentified as those historical metric checkpoints.

Dataset-specific redistribution terms remain separate from the Track4D code license. Historical NPZ files can contain provenance-only source path strings; evaluation uses only their embedded arrays.