Qwen Image Edit Accelerated Inference
Collection
Creative applications and accelerated demos with QwenImageEdit
•
10 items
•
Updated
•
4
ZeroGPU AoTI
community
AI & ML interests
AoT compilation, ZeroGPU inference optimization
Recent Activity
View all activity
sayakpaul
updated
a
collection
4 days ago
sayakpaul
updated
a
Space
4 days ago
linoyts
updated
a
Space
5 days ago
linoyts
updated
a
collection
6 days ago
cbensimon
published
a
Space
7 days ago
Update README.md
1
#5 opened 18 days ago
by
mojito2
multimodalart
updated
a
Space
7 days ago
Update README.md
#6 opened 17 days ago
by
severgo00
Support aspect-ratio–preserving output (avoid forced 832×480)
🔥
👍
2
2
#9 opened 9 days ago
by
jiuface
Dynamic Axis
2
#10 opened 7 days ago
by
multimodalart
Dynamic Axis
2
#10 opened 7 days ago
by
multimodalart
ZeroGPU Spaces go brrr with ahead-of-time compilation
2
#1 opened 8 days ago
by
linoyts
Trouble duplicating space
3
#7 opened 15 days ago
by
SETRusty
cbensimon
updated
a
Space
11 days ago
cbensimon
updated
a
collection
11 days ago
cbensimon
updated
2
Spaces
11 days ago
Post
1202
Fast LoRA inference for Flux with Diffusers and PEFT 🚨
There are great materials that demonstrate how to optimize inference for popular image generation models, such as Flux. However, very few cover how to serve LoRAs fast, despite LoRAs being an inseparable part of their adoption.
In our latest post, @BenjaminB and I show different techniques to optimize LoRA inference for the Flux family of models for image generation. Our recipe includes the use of:
1.
2. Flash Attention 3 (when compatible)
3. Dynamic FP8 weight quantization (when compatible)
4. Hotswapping for avoiding recompilation during swapping new LoRAs 🤯
We have tested our recipe with Flux.1-Dev on both H100 and RTX 4090. We achieve at least a *2x speedup* in either of the GPUs. We believe our recipe is grounded in the reality of how LoRA-based use cases are generally served. So, we hope this will be beneficial to the community 🤗
Even though our recipe was tested primarily with NVIDIA GPUs, it should also work with AMD GPUs.
Learn the details and the full code here:
https://huggingface.co/blog/lora-fast
There are great materials that demonstrate how to optimize inference for popular image generation models, such as Flux. However, very few cover how to serve LoRAs fast, despite LoRAs being an inseparable part of their adoption.
In our latest post, @BenjaminB and I show different techniques to optimize LoRA inference for the Flux family of models for image generation. Our recipe includes the use of:
1.
torch.compile2. Flash Attention 3 (when compatible)
3. Dynamic FP8 weight quantization (when compatible)
4. Hotswapping for avoiding recompilation during swapping new LoRAs 🤯
We have tested our recipe with Flux.1-Dev on both H100 and RTX 4090. We achieve at least a *2x speedup* in either of the GPUs. We believe our recipe is grounded in the reality of how LoRA-based use cases are generally served. So, we hope this will be beneficial to the community 🤗
Even though our recipe was tested primarily with NVIDIA GPUs, it should also work with AMD GPUs.
Learn the details and the full code here:
https://huggingface.co/blog/lora-fast