Add readme

README.md

@@ -0,0 +1,64 @@
+---
+license: creativeml-openrail-m
+tags:
+- text-to-video
+- stable-diffusion
+- animatediff
+library_name: diffusers
+inference: false
+---
+# AnimateDiff-Lightning
+
+<video src='https://huggingface.co/ByteDance/AnimateDiff-Lightning/resolve/main/animatediff_lightning_samples.mp4' width="100%" autoplay muted loop></video>
+
+AnimateDiff-Lightning is a lightning-fast text-to-video generation model. It can generate 16-frame 512px videos in a few steps. For more information, please refer to our research paper: [AnimateDiff-Lightning: Cross-Model Diffusion Distillation](https://huggingface.co/ByteDance/AnimateDiff-Lightning/resolve/main/animatediff_lightning_report.pdf). We release the model as part of the research.
+
+Our models are distilled from [AnimateDiff SD1.5 v2](https://huggingface.co/guoyww/animatediff). This repository contains checkpoints for 1-step, 2-step, 4-step, and 8-step distilled models. The generation quality of our 2-step, 4-step, and 8-step model is great. Our 1-step model is only provided for research purposes.
+
+
+## Recommendation
+
+AnimateDiff-Lightning produces the best results when used with stylized base models. We recommend using the following base models:
+
+Realistic
+- [epiCRealism](https://civitai.com/models/25694)
+- [Realistic Vision](https://civitai.com/models/4201)
+- [DreamShaper](https://civitai.com/models/4384)
+- [AbsoluteReality](https://civitai.com/models/81458)
+- [MajicMix Realistic](https://civitai.com/models/43331)
+
+Anime & Cartoon
+- [ToonYou](https://civitai.com/models/30240)
+- [IMP](https://civitai.com/models/56680)
+- [Mistoon Anime](https://civitai.com/models/24149)
+- [DynaVision](https://civitai.com/models/75549)
+- [RCNZ Cartoon 3d](https://civitai.com/models/66347)
+- [MajicMix Reverie](https://civitai.com/models/65055)
+
+Additionally, feel free to explore different settings. We find using 3 inference steps on the 2-step model produces great results. We find certain base models produces better results with CFG.
+
+## Diffusers Usage
+
+```python
+import torch
+from diffusers import AnimateDiffPipeline, MotionAdapter, EulerDiscreteScheduler
+from diffusers.utils import export_to_gif
+from huggingface_hub import hf_hub_download
+from safetensors.torch import load_file
+
+device = "cuda"
+dtype = torch.float16
+
+step = 4  # Options: [1,2,4,8]
+repo = "AnimateDiff-Lightning"
+ckpt = f"animatediff_lightning_{step}step_diffusers.safetensors"
+base = "SG161222/Realistic_Vision_V5.1_noVAE"  # Choose to your favorite base model.
+
+adapter = MotionAdapter().to(device, dtype)
+adapter.load_state_dict(load_file(hf_hub_download(repo ,ckpt), device=device))
+pipe = AnimateDiffPipeline.from_pretrained(base, motion_adapter=adapter, torch_dtype=dtype).to(device)
+pipe.scheduler = EulerDiscreteScheduler.from_config(pipe.scheduler.config, timestep_spacing="trailing", beta_schedule="linear")
+
+output = pipe(prompt="A girl smiling", guidance_scale=1.0, num_inference_steps=step)
+export_to_gif(output.frames[0], "animation.gif")
+```