🛰️ Model Card: downstream-satvision-toa-3dclouds

⚙️ Model Overview

• Model Name: downstream-satvision-toa-3dclouds
• Base Model: SatVision-TOA (Giant, 3B parameters)
• Architecture: SwinV2 Transformer (ViT backbone)
• Pretraining Objective: Masked Image Modeling (MIM)
• Pretraining Dataset: 100M globally-distributed MODIS TOA image chips across 14 bands
• Resolution: 128×128 px at ~1 km
• Pretraining Conditions: All-sky (cloud, aerosol, ocean, land)

🗂️ Intended Use

• Task: 3D cloud vertical reconstruction from satellite TOA imagery
• Downstream Data: GOES-ABI chips paired with CloudSat/CALIPSO cloud curtain observations
• Output: Per-pixel cloud vertical class (e.g., cloud top/base detection, multilayer structure)

🧠 Strengths

• Learns spatial-spectral relationships across diverse global conditions
• Generalizes well across sensors (MODIS → GOES-ABI)
• Outperforms baseline on thin, multilayer, and obscured clouds
• Pretraining improves sample efficiency for fine-tuning

⚠️ Limitations

• Temporal bias: Terra-MODIS sampling (~9 AM local) may limit temporal generalization
• Resolution: Only supports ~1 km scale chips; sub-km cloud structures not resolved
• Sensor adaptation: While GOES-ABI is supported, optimal results may require minor domain tuning

🛠️ Fine-Tuning & Usage

• Decoder: Lightweight FCN head on frozen SatVision-TOA encoder
• Training Data: ~7,000 labeled GOES-ABI chips aligned with CloudSat/CALIPSO
• Validation Set: 1,300 chips
• Typical Inference Output: 2D maps of vertical cloud structure per chip

🔁 Adaptation Ideas

• Extend to aerosol, water vapor, or ice phase classification
• Fine-tune on nighttime or different orbital sensors (e.g., VIIRS, Himawari)
• Use as encoder backbone for multitask satellite cloud analysis

📝 Citation

If you use this model, please cite:

@article{satvision2024, title={SatVision-TOA: A Geospatial Foundation Model for Coarse-Resolution All-Sky Remote Sensing Imagery}, author={Zhu, Le and Caraballo-Vega, Jordan and Gentine, Pierre and Tao, Wenzhong and et al.}, journal={arXiv preprint arXiv:2406.06561}, year={2024} }

🔗 Resources

• 🗃️ Hugging Face Model Repo
• 📄 SatVision-TOA Preprint
• 🌐 Earthdata Overview

---

📌 Summary:
This model leverages a powerful foundation transformer trained on MODIS TOA data to deliver high-fidelity 3D cloud reconstructions from GOES-ABI imagery. It serves as a critical step toward operational cloud analysis from geostationary satellites using foundation model paradigms.