22 25 24

Naga Sai Abhinay Devarinti

NagaSaiAbhinay

AI & ML interests

Diffusion Models, NLP, LLM's

Recent Activity

new activity 2 days ago

MizzenAI/HPSv3:Fixes the dataset link in the badge

liked a model 14 days ago

ByteDance/Hyper-SD

liked a model 16 days ago

StyleXX/OmniStyle

View all activity

Organizations

New activity in MizzenAI/HPSv3 2 days ago

Fixes the dataset link in the badge

#2 opened 2 days ago by

NagaSaiAbhinay

liked a model 14 days ago

ByteDance/Hyper-SD

Text-to-Image • Updated Dec 5, 2024 • 129k • • 1.28k

liked a model 16 days ago

StyleXX/OmniStyle

Updated Jul 11 • 2

upvoted a paper 21 days ago

EarthCrafter: Scalable 3D Earth Generation via Dual-Sparse Latent Diffusion

Paper • 2507.16535 • Published 23 days ago • 20

liked a model about 1 month ago

RaphaelLiu/PusaV1

Image-to-Video • Updated 19 days ago • 188

upvoted an article about 1 month ago

Article

cocogold: training Marigold for text-grounded segmentation

•

Jul 8

• 30

reacted to a-r-r-o-w's post with 🔥 about 1 month ago

Post

3283

Caching is an essential technique used in diffusion inference serving for speeding up image/video generations. Diffusers just added support for another caching method: First Block Cache - a technique developed by @chengzeyi building upon the ideas of TeaCache.

The idea in short is: if the model predictions do not vary much over successive inference steps, we can skip certain steps where the prediction difference is small. To figure out whether an inference step will make a significant improvement to the overall velocity/noise prediction, we calculate the relative difference of the output of the first transformer block at timestep $t$ with $t-1$, and compare it against a selected threshold. If the difference is lower than the threshold, we skip the step. A higher threshold will lead to more steps being skipped. However, skipping many steps is bad because it can throw off the model predictions, and so we need to test and select the threshold based on level of quality-speed tradeoff for every model we use it with.

Diffusers usage with CogView4:

import torch
from diffusers import CogView4Pipeline
from diffusers.hooks import apply_first_block_cache, FirstBlockCacheConfig

pipe = CogView4Pipeline.from_pretrained("THUDM/CogView4-6B", torch_dtype=torch.bfloat16)
pipe.to("cuda")

apply_first_block_cache(pipe.transformer, FirstBlockCacheConfig(threshold=0.2))

prompt = "A photo of an astronaut riding a horse on mars"
image = pipe(prompt, generator=torch.Generator().manual_seed(42)).images[0]
image.save("output.png")