Image-to-Image
Diffusers

Improve model card

#1
by nielsr HF Staff - opened
Files changed (1) hide show
  1. README.md +31 -3
README.md CHANGED
@@ -1,3 +1,31 @@
1
- ---
2
- license: mit
3
- ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: mit
3
+ library_name: diffusers
4
+ pipeline_tag: image-to-image
5
+ ---
6
+
7
+ ---
8
+ license: mit
9
+ library_name: diffusers
10
+ pipeline_tag: image-to-image
11
+ ---
12
+
13
+ # REPA-E: Unlocking VAE for End-to-End Tuning of Latent Diffusion Transformers
14
+
15
+ ## About
16
+ This model addresses the question of whether latent diffusion models and their VAE tokenizer can be trained end-to-end. Using a representation-alignment (REPA) loss, REPA-E enables stable and effective joint training of both components, leading to significant training acceleration and improved VAE performance. The resulting E2E-VAE serves as a drop-in replacement for existing VAEs, improving convergence and generation quality across diverse LDM architectures.
17
+
18
+ This model is based on the paper [REPA-E: Unlocking VAE for End-to-End Tuning of Latent Diffusion Transformers](https://huggingface.co/papers/2504.10483) and its official implementation is available on [Github](https://github.com/REPA-E/REPA-E). The project page can be found at [https://end2end-diffusion.github.io](https://end2end-diffusion.github.io).
19
+
20
+ ## Usage
21
+
22
+ To use the REPA-E model, you can load it via the Hugging Face `DiffusionPipeline`. Below is a simplified example of how to use a pretrained REPA-E model for inference. For training examples and further details, please refer to the [Github repository](https://github.com/REPA-E/REPA-E).
23
+
24
+ ```python
25
+ from diffusers import DiffusionPipeline
26
+
27
+ pipeline = DiffusionPipeline.from_pretrained("REPA-E/sit-repae-sdvae", trust_remote_code=True)
28
+ image = pipeline().images[0]
29
+
30
+ image.save("generated_image.png")
31
+ ```