File size: 2,599 Bytes
c84f1dd ce21de1 c84f1dd 907198a 440c377 907198a 440c377 907198a c84f1dd ce21de1 427a166 ce21de1 f77deb0 427a166 ce21de1 f77deb0 ce21de1 f77deb0 ce21de1 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 |
---
license: apache-2.0
library_name: terratorch
datasets:
- ibm-esa-geospatial/TerraMesh
tags:
- Earth Observation
- TerraMind
- IBM
- ESA
---
# TerraMind 1.0 S-2 L2A Tokenizer
TerraMind is the first multimodal any-to-any generative foundation model for Earth Observation jointly developed by IBM, ESA, and Forschungszentrum Jülich.
The model is pre-trained using FSQ-VAE tokens as targets. This tokenizer encodes and decodes Sentinel-2 L2A satellite images for the TerraMind model.
The tokenizer uses FSQ with five dimensions and a codebook size of 15'360 tokens.
The decoding process uses diffusion steps for the reconstruction.
The model was pre-trained for 100 epochs on nine million S-2 L2A images from the TerraMesh dataset.
## Usage
The tokenizer is fully integrated into the fine-tuning toolkit [TerraTorch](https://ibm.github.io/terratorch/).
You can initialize the pre-trained tokenizer with:
```python
from terratorch.registry import FULL_MODEL_REGISTRY
model = FULL_MODEL_REGISTRY.build('terramind_v1_tokenizer_s2l2a', pretrained=True)
```
Once the model is build, it can be used to encode image and decode tokens.
The number of diffusion steps is defined with `timesteps`.
Increasing the diffusion steps adds more details to the reconstruction which can also lead to hallucinations.
```python
# Encode image
_, _, tokens = model.encode(s2l2a_tensor)
# Decode tokens
reconstruction = model.decode_tokens(tokens, verbose=True, timesteps=10)
# Encode & decode
reconstruction = model(s2l2a_tensor)
```
This tokenizer is automatically loaded with TerraMind generation models like `terramind_v1_base_generate`, see [here](https://huggingface.co/ibm-esa-geospatial/TerraMind-1.0-base#generations) for details.
We provide example code for the tokenizer at https://github.com/IBM/terramind.
## Feedback
If you have feedback or any questions, please start a discussion in this HF repository or submitting an issue to [TerraMind](https://github.com/IBM/terramind) on GitHub.
## Citation
If you use TerraMind in your research, please cite our [TerraMind](https://arxiv.org/abs/2504.11171) pre-print.
```text
@article{jakubik2025terramind,
title={TerraMind: Large-Scale Generative Multimodality for Earth Observation},
author={Jakubik, Johannes and Yang, Felix and Blumenstiel, Benedikt and Scheurer, Erik and Sedona, Rocco and Maurogiovanni, Stefano and Bosmans, Jente and Dionelis, Nikolaos and Marsocci, Valerio and Kopp, Niklas and others},
journal={arXiv preprint arXiv:2504.11171},
year={2025}
}
``` |