Lawrence-cj commited on
Commit
b0dbf5a
·
verified ·
1 Parent(s): 268b7e3

Create README.md

Browse files
Files changed (1) hide show
  1. README.md +119 -0
README.md ADDED
@@ -0,0 +1,119 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ library_name: sana, sana-sprint
3
+ tags:
4
+ - text-to-image
5
+ - SANA-Sprint
6
+ - 1024px_based_image_size
7
+ - BF16
8
+ - One-step diffusion
9
+ language:
10
+ - en
11
+ - zh
12
+ base_model:
13
+ - Efficient-Large-Model/Sana_Sprint_0.6B_1024px
14
+ pipeline_tag: text-to-image
15
+ ---
16
+ <p align="center" style="border-radius: 10px">
17
+ <img src="https://nvlabs.github.io/Sana/Sprint/asset/SANA-Sprint.png" width="50%" alt="logo"/>
18
+ </p>
19
+
20
+ <div style="display:flex;justify-content: center">
21
+ <a href="https://huggingface.co/collections/Efficient-Large-Model/sana-sprint-67d6810d65235085b3b17c76"><img src="https://img.shields.io/static/v1?label=Weights&message=Huggingface&color=yellow"></a> &ensp;
22
+ <a href="https://github.com/NVlabs/Sana"><img src="https://img.shields.io/static/v1?label=Code&message=Github&color=blue&logo=github"></a> &ensp;
23
+ <a href="https://nvlabs.github.io/Sana/Sprint/"><img src="https://img.shields.io/static/v1?label=Project&message=Github&color=blue&logo=github-pages"></a> &ensp;
24
+ <!-- <a href="https://hanlab.mit.edu/projects/sana/"><img src="https://img.shields.io/static/v1?label=Page&message=MIT&color=darkred&logo=github-pages"></a> &ensp; -->
25
+ <a href="https://arxiv.org/pdf/2503.09641"><img src="https://img.shields.io/static/v1?label=Arxiv&message=SANA-Sprint&color=red&logo=arxiv"></a> &ensp;
26
+ <a href="https://nv-sana.mit.edu/sprint"><img src="https://img.shields.io/static/v1?label=Demo&message=MIT&color=yellow"></a> &ensp;
27
+ <a href="https://discord.gg/rde6eaE5Ta"><img src="https://img.shields.io/static/v1?label=Discuss&message=Discord&color=purple&logo=discord"></a> &ensp;
28
+ </div>
29
+
30
+ # 🐱 Sana Model Card
31
+
32
+ ## Demos
33
+
34
+ <div align="center">
35
+ <a href="https://www.youtube.com/watch?v=nI_Ohgf8eOU" target="_blank">
36
+ <img src="https://img.youtube.com/vi/nI_Ohgf8eOU/0.jpg" alt="Demo Video of SANA-Sprint" style="width: 48%; display: block; margin: 0 auto; display: inline-block;">
37
+ </a>
38
+ <a href="https://www.youtube.com/watch?v=OOZzkirgsAc" target="_blank">
39
+ <img src="https://img.youtube.com/vi/OOZzkirgsAc/0.jpg" alt="Demo Video of SANA-Sprint" style="width: 48%; display: block; margin: 0 auto; display: inline-block;">
40
+ </a>
41
+ </div>
42
+
43
+
44
+ ## Training Pipeline
45
+
46
+ <p align="center" border-raduis="10px">
47
+ <img src="https://nvlabs.github.io/Sana/Sprint/asset/content/paradigm.png" width="90%" alt="teaser_page1"/>
48
+ </p>
49
+
50
+ ## Model Efficiency
51
+
52
+ <p align="center" border-raduis="10px">
53
+ <img src="https://nvlabs.github.io/Sana/Sprint/asset/content/teaser.png" width="95%" alt="teaser_page1"/>
54
+ </p>
55
+
56
+ SANA-Sprint is an ultra-efficient diffusion model for text-to-image (T2I) generation, reducing inference steps from 20 to 1-4 while achieving state-of-the-art performance.
57
+ Key innovations include:
58
+ (1) A training-free approach for continuous-time consistency distillation (sCM), eliminating costly retraining;
59
+ (2) A unified step-adaptive model for high-quality generation in 1-4 steps; and
60
+ (3) ControlNet integration for real-time interactive image generation.
61
+ SANA-Sprint achieves **7.59 FID and 0.74 GenEval in just 1 step** — outperforming FLUX-schnell (7.94 FID / 0.71 GenEval) while being 10× faster (0.1s vs 1.1s on H100).
62
+ With latencies of **0.1s (T2I) and 0.25s (ControlNet)** for 1024×1024 images on H100, and 0.31s (T2I) on an RTX 4090, SANA-Sprint is ideal for AI-powered consumer applications (AIPC).
63
+
64
+
65
+ Source code is available at https://github.com/NVlabs/Sana.
66
+
67
+ ### Model Description
68
+
69
+ - **Developed by:** NVIDIA, Sana
70
+ - **Model type:** One-Step Diffusion with Continuous-Time Consistency Distillation
71
+ - **Model size:** 0.6B parameters
72
+ - **Model precision:** torch.bfloat16 (BF16)
73
+ - **Model resolution:** This model is developed to generate 1024px based images with multi-scale heigh and width.
74
+ - **License:** [NSCL v2-custom](./LICENSE.txt). Governing Terms: NVIDIA License. Additional Information: [Gemma Terms of Use | Google AI for Developers](https://ai.google.dev/gemma/terms) for Gemma-2-2B-IT, [Gemma Prohibited Use Policy | Google AI for Developers](https://ai.google.dev/gemma/prohibited_use_policy).
75
+ - **Model Description:** This is a model that can be used to generate and modify images based on text prompts.
76
+ It is a Linear Diffusion Transformer that uses one fixed, pretrained text encoders ([Gemma2-2B-IT](https://huggingface.co/google/gemma-2-2b-it))
77
+ and one 32x spatial-compressed latent feature encoder ([DC-AE](https://hanlab.mit.edu/projects/dc-ae)).
78
+ - **Resources for more information:** Check out our [GitHub Repository](https://github.com/NVlabs/Sana) and the [SANA-Sprint report on arXiv](https://arxiv.org/pdf/2503.09641).
79
+
80
+ ### Model Sources
81
+
82
+ For research purposes, we recommend our `generative-models` Github repository (https://github.com/NVlabs/Sana), which is more suitable for both training and inference
83
+ [MIT Han-Lab](https://nv-sana.mit.edu/sprint) provides free SANA-Sprint inference.
84
+ - **Repository:** https://github.com/NVlabs/Sana
85
+ - **Demo:** https://nv-sana.mit.edu/sprint
86
+ - **Guidance:** https://github.com/NVlabs/Sana/asset/docs/sana_sprint.md
87
+
88
+
89
+ ## Uses
90
+
91
+ ### Direct Use
92
+
93
+ The model is intended for research purposes only. Possible research areas and tasks include
94
+
95
+ - Generation of artworks and use in design and other artistic processes.
96
+ - Applications in educational or creative tools.
97
+ - Research on generative models.
98
+ - Safe deployment of models which have the potential to generate harmful content.
99
+
100
+ - Probing and understanding the limitations and biases of generative models.
101
+
102
+ Excluded uses are described below.
103
+
104
+ ### Out-of-Scope Use
105
+
106
+ The model was not trained to be factual or true representations of people or events, and therefore using the model to generate such content is out-of-scope for the abilities of this model.
107
+
108
+ ## Limitations and Bias
109
+
110
+ ### Limitations
111
+
112
+
113
+ - The model does not achieve perfect photorealism
114
+ - The model cannot render complex legible text
115
+ - fingers, .etc in general may not be generated properly.
116
+ - The autoencoding part of the model is lossy.
117
+
118
+ ### Bias
119
+ While the capabilities of image generation models are impressive, they can also reinforce or exacerbate social biases.