Hugging Face's logo
Nishith Jain's picture
254 190 717

Nishith Jain

KingNish
Marcozart's profile picture QQS0498's profile picture tomatobobot's profile picture
·

AI & ML interests

AI is fun actually.

Recent Activity

updated a Space 29 minutes ago
KingNish/Realtime-FLUX
reacted to a-r-r-o-w's post with 🧠 33 minutes ago
Caching is an essential technique used in diffusion inference serving for speeding up image/video generations. Diffusers just added support for another caching method: First Block Cache - a technique developed by @chengzeyi building upon the ideas of TeaCache. The idea in short is: if the model predictions do not vary much over successive inference steps, we can skip certain steps where the prediction difference is small. To figure out whether an inference step will make a significant improvement to the overall velocity/noise prediction, we calculate the relative difference of the output of the first transformer block at timestep $t$ with $t-1$, and compare it against a selected threshold. If the difference is lower than the threshold, we skip the step. A higher threshold will lead to more steps being skipped. However, skipping many steps is bad because it can throw off the model predictions, and so we need to test and select the threshold based on level of quality-speed tradeoff for every model we use it with. Diffusers usage with CogView4: ```python import torch from diffusers import CogView4Pipeline from diffusers.hooks import apply_first_block_cache, FirstBlockCacheConfig pipe = CogView4Pipeline.from_pretrained("THUDM/CogView4-6B", torch_dtype=torch.bfloat16) pipe.to("cuda") apply_first_block_cache(pipe.transformer, FirstBlockCacheConfig(threshold=0.2)) prompt = "A photo of an astronaut riding a horse on mars" image = pipe(prompt, generator=torch.Generator().manual_seed(42)).images[0] image.save("output.png") ``` Below, you'll find the benchmarks and visualizations of the predicted output at different blocks of the Flux DiT. Docs: https://huggingface.co/docs/diffusers/main/en/optimization/cache PR: https://github.com/huggingface/diffusers/pull/11180 References: - First Block Cache: https://github.com/chengzeyi/ParaAttention - TeaCache: https://github.com/ali-vilab/TeaCache
reacted to a-r-r-o-w's post with 🔥 34 minutes ago
Caching is an essential technique used in diffusion inference serving for speeding up image/video generations. Diffusers just added support for another caching method: First Block Cache - a technique developed by @chengzeyi building upon the ideas of TeaCache. The idea in short is: if the model predictions do not vary much over successive inference steps, we can skip certain steps where the prediction difference is small. To figure out whether an inference step will make a significant improvement to the overall velocity/noise prediction, we calculate the relative difference of the output of the first transformer block at timestep $t$ with $t-1$, and compare it against a selected threshold. If the difference is lower than the threshold, we skip the step. A higher threshold will lead to more steps being skipped. However, skipping many steps is bad because it can throw off the model predictions, and so we need to test and select the threshold based on level of quality-speed tradeoff for every model we use it with. Diffusers usage with CogView4: ```python import torch from diffusers import CogView4Pipeline from diffusers.hooks import apply_first_block_cache, FirstBlockCacheConfig pipe = CogView4Pipeline.from_pretrained("THUDM/CogView4-6B", torch_dtype=torch.bfloat16) pipe.to("cuda") apply_first_block_cache(pipe.transformer, FirstBlockCacheConfig(threshold=0.2)) prompt = "A photo of an astronaut riding a horse on mars" image = pipe(prompt, generator=torch.Generator().manual_seed(42)).images[0] image.save("output.png") ``` Below, you'll find the benchmarks and visualizations of the predicted output at different blocks of the Flux DiT. Docs: https://huggingface.co/docs/diffusers/main/en/optimization/cache PR: https://github.com/huggingface/diffusers/pull/11180 References: - First Block Cache: https://github.com/chengzeyi/ParaAttention - TeaCache: https://github.com/ali-vilab/TeaCache
View all activity

Organizations

Stanford AI's profile picture Wikimedia's profile picture OpenGVLab's profile picture Blog-explorers's profile picture Multi🤖Transformers's profile picture The Collectionists's profile picture HelpingAI's profile picture ZeroGPU Explorers's profile picture Project Fluently's profile picture Poscye's profile picture INNOVA AI's profile picture Narra's profile picture Social Post Explorers's profile picture Dev Mode Explorers's profile picture Stable Diffusion Community (Unofficial, Non-profit)'s profile picture ONNX Community's profile picture Hugging Face Discord Community's profile picture Nerdy Face's profile picture HelpingAI's profile picture None yet's profile picture Project R's profile picture Doge Face's profile picture Hugging Face MCP Course's profile picture Agents-MCP-Hackathon's profile picture Testing-Lab's profile picture