Commit
·
ce21de1
1
Parent(s):
c84f1dd
Update ReadMe
Browse files- .gitattributes +1 -0
- README.md +55 -1
.gitattributes
CHANGED
@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
|
33 |
*.zip filter=lfs diff=lfs merge=lfs -text
|
34 |
*.zst filter=lfs diff=lfs merge=lfs -text
|
35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
|
|
|
33 |
*.zip filter=lfs diff=lfs merge=lfs -text
|
34 |
*.zst filter=lfs diff=lfs merge=lfs -text
|
35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
36 |
+
*.png filter=lfs diff=lfs merge=lfs -text
|
README.md
CHANGED
@@ -1,5 +1,59 @@
|
|
1 |
---
|
|
|
2 |
library_name: terratorch
|
3 |
---
|
4 |
|
5 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
---
|
2 |
+
license: apache-2.0
|
3 |
library_name: terratorch
|
4 |
---
|
5 |
|
6 |
+
# TerraMind 1.0 S-2 L2A Tokenizer
|
7 |
+
|
8 |
+
TerraMind is the first multimodal any-to-any generative foundation model for Earth Observation jointly developed by IBM, ESA, and Forschungszentrum Jülich.
|
9 |
+
|
10 |
+
The model is pre-trained using FSQ-VAE tokens as targets. This tokenizer encodes and decodes Sentinel-2 L2A satellite images.
|
11 |
+
|
12 |
+

|
13 |
+
|
14 |
+
The tokenizer uses FSQ with five dimensions and a codebook size of 15'360 tokens.
|
15 |
+
It was pre-trained for 100 epochs on the S-2 L2A images from the TerraMesh dataset.
|
16 |
+
|
17 |
+
## Usage
|
18 |
+
|
19 |
+
The tokenizer is fully integrated into the fine-tuning toolkit [TerraTorch](https://ibm.github.io/terratorch/).
|
20 |
+
|
21 |
+
You can initialize the pre-trained tokenizer with:
|
22 |
+
```python
|
23 |
+
from terratorch.registry import FULL_MODEL_REGISTRY
|
24 |
+
model = FULL_MODEL_REGISTRY.build('terramind_v1_tokenizer_s2l2a', pretrained=True)
|
25 |
+
```
|
26 |
+
|
27 |
+
Once the model is build, you can run encode and decode tokens, or test the reconstruction.
|
28 |
+
Define the number of diffusion steps with `timesteps`.
|
29 |
+
|
30 |
+
```python
|
31 |
+
# Encode image
|
32 |
+
_, _, tokens = model.encode(s2l2a_tensor)
|
33 |
+
# Decode tokens
|
34 |
+
reconstruction = model.decode_tokens(tokens, verbose=True, timesteps=10)
|
35 |
+
# Encode & decode
|
36 |
+
reconstruction = model(s2l2a_tensor)
|
37 |
+
```
|
38 |
+
|
39 |
+
This tokenizer is automatically loaded with TerraMind generation models like `terramind_v1_base_generate`, see [here](https://huggingface.co/ibm-esa-geospatial/TerraMind-1.0-base#generations) for details.
|
40 |
+
|
41 |
+
We provide example code for the tokenizer at https://github.com/IBM/terramind.
|
42 |
+
|
43 |
+
|
44 |
+
## Feedback
|
45 |
+
|
46 |
+
If you have feedback or any questions, please start a discussion in this HF repository or submitting an issue to [TerraMind](https://github.com/IBM/terramind) on GitHub.
|
47 |
+
|
48 |
+
## Citation
|
49 |
+
|
50 |
+
If you use TerraMind in your research, please cite our [TerraMind](https://arxiv.org/abs/2504.11171) pre-print.
|
51 |
+
|
52 |
+
```text
|
53 |
+
@article{jakubik2025terramind,
|
54 |
+
title={TerraMind: Large-Scale Generative Multimodality for Earth Observation},
|
55 |
+
author={Jakubik, Johannes and Yang, Felix and Blumenstiel, Benedikt and Scheurer, Erik and Sedona, Rocco and Maurogiovanni, Stefano and Bosmans, Jente and Dionelis, Nikolaos and Marsocci, Valerio and Kopp, Niklas and others},
|
56 |
+
journal={arXiv preprint arXiv:2504.11171},
|
57 |
+
year={2025}
|
58 |
+
}
|
59 |
+
```
|