Nishith Jain's picture

Nishith Jain

KingNish

·

AI & ML interests

AI is fun actually.

Recent Activity

updated a Space 5 minutes ago

KingNish/Realtime-FLUX

reacted to a-r-r-o-w's post with 🧠 9 minutes ago

Caching is an essential technique used in diffusion inference serving for speeding up image/video generations. Diffusers just added support for another caching method: First Block Cache - a technique developed by @chengzeyi building upon the ideas of TeaCache. The idea in short is: if the model predictions do not vary much over successive inference steps, we can skip certain steps where the prediction difference is small. To figure out whether an inference step will make a significant improvement to the overall velocity/noise prediction, we calculate the relative difference of the output of the first transformer block at timestep $t$ with $t-1$, and compare it against a selected threshold. If the difference is lower than the threshold, we skip the step. A higher threshold will lead to more steps being skipped. However, skipping many steps is bad because it can throw off the model predictions, and so we need to test and select the threshold based on level of quality-speed tradeoff for every model we use it with. Diffusers usage with CogView4: ```python import torch from diffusers import CogView4Pipeline from diffusers.hooks import apply_first_block_cache, FirstBlockCacheConfig pipe = CogView4Pipeline.from_pretrained("THUDM/CogView4-6B", torch_dtype=torch.bfloat16) pipe.to("cuda") apply_first_block_cache(pipe.transformer, FirstBlockCacheConfig(threshold=0.2)) prompt = "A photo of an astronaut riding a horse on mars" image = pipe(prompt, generator=torch.Generator().manual_seed(42)).images[0] image.save("output.png") ``` Below, you'll find the benchmarks and visualizations of the predicted output at different blocks of the Flux DiT. Docs: https://huggingface.co/docs/diffusers/main/en/optimization/cache PR: https://github.com/huggingface/diffusers/pull/11180 References: - First Block Cache: https://github.com/chengzeyi/ParaAttention - TeaCache: https://github.com/ali-vilab/TeaCache

reacted to a-r-r-o-w's post with 🔥 9 minutes ago

Caching is an essential technique used in diffusion inference serving for speeding up image/video generations. Diffusers just added support for another caching method: First Block Cache - a technique developed by @chengzeyi building upon the ideas of TeaCache. The idea in short is: if the model predictions do not vary much over successive inference steps, we can skip certain steps where the prediction difference is small. To figure out whether an inference step will make a significant improvement to the overall velocity/noise prediction, we calculate the relative difference of the output of the first transformer block at timestep $t$ with $t-1$, and compare it against a selected threshold. If the difference is lower than the threshold, we skip the step. A higher threshold will lead to more steps being skipped. However, skipping many steps is bad because it can throw off the model predictions, and so we need to test and select the threshold based on level of quality-speed tradeoff for every model we use it with. Diffusers usage with CogView4: ```python import torch from diffusers import CogView4Pipeline from diffusers.hooks import apply_first_block_cache, FirstBlockCacheConfig pipe = CogView4Pipeline.from_pretrained("THUDM/CogView4-6B", torch_dtype=torch.bfloat16) pipe.to("cuda") apply_first_block_cache(pipe.transformer, FirstBlockCacheConfig(threshold=0.2)) prompt = "A photo of an astronaut riding a horse on mars" image = pipe(prompt, generator=torch.Generator().manual_seed(42)).images[0] image.save("output.png") ``` Below, you'll find the benchmarks and visualizations of the predicted output at different blocks of the Flux DiT. Docs: https://huggingface.co/docs/diffusers/main/en/optimization/cache PR: https://github.com/huggingface/diffusers/pull/11180 References: - First Block Cache: https://github.com/chengzeyi/ParaAttention - TeaCache: https://github.com/ali-vilab/TeaCache

View all activity

Organizations

KingNish 's Spaces 32

Starting on Zero

FLUX Realtime

High quality Images in Realtime

Personalized Video Gen

Generate a greeting for a name

Running on Zero

Qwen2.5-VL

Describe and analyze images and videos

Bagel 7B Demo

Generate images and understand/edit them using text prompts

OpenGPT 4o

GPT 4o like bot.

Voicee

Super fastest Voice Assistant

Running on Zero

Realtime Whisper Turbo

Realtime implementation of Whisper large turbo

VIDEO CHAT

World's fastest Video Chat Assistant

OpenCHAT Mini

Chatbot with Unlimited Vision,Image generation and WebSearch

Image Gen Pro

Advanced Image Generator

OpenCHAT Mini

Chatbot with Unlimited Vision,Image generation and WebSearch

My Whisper Train Test

Running on Zero

LTX Video Fast

ultra-fast video model, LTX 0.9.7 13B distilled

Api Mcp

Don't use this

Gradio Client Test

Generate a video from a prompt and image

Running on Zero

Sarvam M Demo

Generate conversational responses based on chat history

Patram 7b Demo

Demo of Patram 7b Model

Test Space

Browse and search products like Amazon

Running on Zero

SDXL Flash

High quality image generation in 3 second

Dora The Reader

Fetch, sort, and summarize academic papers

Transformers.js Playground

Run AI Models in Browser locally.

First Agent Template

Find the current time in any timezone

Doc Reader And Chat

Extract and clean text from various documents

Fetch Content