TerraTorch
Earth Observation
TerraMind
IBM
ESA
blumenstiel commited on
Commit
ce21de1
·
1 Parent(s): c84f1dd

Update ReadMe

Browse files
Files changed (2) hide show
  1. .gitattributes +1 -0
  2. README.md +55 -1
.gitattributes CHANGED
@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
 
 
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ *.png filter=lfs diff=lfs merge=lfs -text
README.md CHANGED
@@ -1,5 +1,59 @@
1
  ---
 
2
  library_name: terratorch
3
  ---
4
 
5
- Model weights for the TerraMind 1.0 Tokenizer for S2L2A.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
+ license: apache-2.0
3
  library_name: terratorch
4
  ---
5
 
6
+ # TerraMind 1.0 S-2 L2A Tokenizer
7
+
8
+ TerraMind is the first multimodal any-to-any generative foundation model for Earth Observation jointly developed by IBM, ESA, and Forschungszentrum Jülich.
9
+
10
+ The model is pre-trained using FSQ-VAE tokens as targets. This tokenizer encodes and decodes Sentinel-2 L2A satellite images.
11
+
12
+ ![s2l2a_tokenizer.png](assets%2Fs2l2a_tokenizer.png)
13
+
14
+ The tokenizer uses FSQ with five dimensions and a codebook size of 15'360 tokens.
15
+ It was pre-trained for 100 epochs on the S-2 L2A images from the TerraMesh dataset.
16
+
17
+ ## Usage
18
+
19
+ The tokenizer is fully integrated into the fine-tuning toolkit [TerraTorch](https://ibm.github.io/terratorch/).
20
+
21
+ You can initialize the pre-trained tokenizer with:
22
+ ```python
23
+ from terratorch.registry import FULL_MODEL_REGISTRY
24
+ model = FULL_MODEL_REGISTRY.build('terramind_v1_tokenizer_s2l2a', pretrained=True)
25
+ ```
26
+
27
+ Once the model is build, you can run encode and decode tokens, or test the reconstruction.
28
+ Define the number of diffusion steps with `timesteps`.
29
+
30
+ ```python
31
+ # Encode image
32
+ _, _, tokens = model.encode(s2l2a_tensor)
33
+ # Decode tokens
34
+ reconstruction = model.decode_tokens(tokens, verbose=True, timesteps=10)
35
+ # Encode & decode
36
+ reconstruction = model(s2l2a_tensor)
37
+ ```
38
+
39
+ This tokenizer is automatically loaded with TerraMind generation models like `terramind_v1_base_generate`, see [here](https://huggingface.co/ibm-esa-geospatial/TerraMind-1.0-base#generations) for details.
40
+
41
+ We provide example code for the tokenizer at https://github.com/IBM/terramind.
42
+
43
+
44
+ ## Feedback
45
+
46
+ If you have feedback or any questions, please start a discussion in this HF repository or submitting an issue to [TerraMind](https://github.com/IBM/terramind) on GitHub.
47
+
48
+ ## Citation
49
+
50
+ If you use TerraMind in your research, please cite our [TerraMind](https://arxiv.org/abs/2504.11171) pre-print.
51
+
52
+ ```text
53
+ @article{jakubik2025terramind,
54
+ title={TerraMind: Large-Scale Generative Multimodality for Earth Observation},
55
+ author={Jakubik, Johannes and Yang, Felix and Blumenstiel, Benedikt and Scheurer, Erik and Sedona, Rocco and Maurogiovanni, Stefano and Bosmans, Jente and Dionelis, Nikolaos and Marsocci, Valerio and Kopp, Niklas and others},
56
+ journal={arXiv preprint arXiv:2504.11171},
57
+ year={2025}
58
+ }
59
+ ```