vfontech commited on
Commit
8fd2d97
·
verified ·
1 Parent(s): 0d42fed

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +72 -6
README.md CHANGED
@@ -1,11 +1,77 @@
1
  ---
2
- language: en
 
3
  tags:
4
- - model_hub_mixin
5
  - pytorch_model_hub_mixin
 
 
 
 
 
6
  ---
7
 
8
- This model has been pushed to the Hub using the [PytorchModelHubMixin](https://huggingface.co/docs/huggingface_hub/package_reference/mixins#huggingface_hub.PyTorchModelHubMixin) integration:
9
- - Code: https://github.com/VicFonch/Multi-Input-Resshift-Diffusion-VFI
10
- - Paper: https://arxiv.org/pdf/2504.05402
11
- - Docs: [More Information Needed]
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
+ language:
3
+ - en
4
  tags:
 
5
  - pytorch_model_hub_mixin
6
+ - animation
7
+ - video-frame-interpolation
8
+ - uncertainty-estimation
9
+ license: mit
10
+ pipeline_tag: image-to-image
11
  ---
12
 
13
+ # 🤖 Multi‑Input ResShift Diffusion VFI
14
+
15
+ <div align="left" style="display: flex; flex-direction: row; gap: 15px">
16
+ <a href='https://arxiv.org/pdf/2504.05402'><img src='https://img.shields.io/badge/arXiv-2405.17933-b31b1b.svg'></a>
17
+ <a href='https://github.com/VicFonch/Multi-Input-Resshift-Diffusion-VFI'><img src='https://img.shields.io/badge/Repo-Code-blue'></a>
18
+ <a href='https://colab.research.google.com/drive/1MGYycbNMW6Mxu5MUqw_RW_xxiVeHK5Aa#scrollTo=EKaYCioiP3tQ'><img src='https://img.shields.io/badge/Colab-Demo-Green'></a>
19
+ </div>
20
+
21
+ ## ⚙️ Setup
22
+
23
+ Start by downloading the source code directly from GitHub.
24
+
25
+ ```bash
26
+ git clone https://github.com/VicFonch/Multi-Input-Resshift-Diffusion-VFI.git
27
+ ```
28
+
29
+ Create a conda environment and install all the requirements
30
+
31
+ ```bash
32
+ conda create -n multi-input-resshift python=3.10
33
+ conda activate multi-input-resshift
34
+ pip install -r requirements.txt
35
+ ```
36
+
37
+ **Note**: Make sure your system is compatible with **CUDA 12.4**. If not, install [CuPy](https://docs.cupy.dev/en/stable/install.html) according to your current CUDA version.
38
+
39
+ ## 🚀 Inference Example
40
+
41
+ ```python
42
+ import os
43
+ from PIL import Image
44
+ import numpy as np
45
+ import matplotlib.pyplot as plt
46
+
47
+ from torchvision.transforms import Compose, ToTensor, Resize, Normalize
48
+ from utils.utils import denorm
49
+ from model.hub import MultiInputResShiftHub
50
+
51
+ model = MultiInputResShiftHub.from_pretrained("vfontech/Multiple-Input-Resshift-VFI")
52
+ model.requires_grad_(False).cuda().eval()
53
+
54
+ img0_path = r"_data\example_images\frame1.png"
55
+ img2_path = r"_data\example_images\frame3.png"
56
+
57
+ transforms = Compose([
58
+ Resize((256, 448)),
59
+ ToTensor(),
60
+ Normalize(mean=[0.5, 0.5, 0.5], std=[0.5, 0.5, 0.5]),
61
+ ])
62
+
63
+ img0 = transforms(Image.open(img0_path).convert("RGB")).unsqueeze(0).cuda()
64
+ img2 = transforms(Image.open(img2_path).convert("RGB")).unsqueeze(0).cuda()
65
+ tau = 0.5
66
+
67
+ img1 = model.reverse_process([img0, img2], tau)
68
+
69
+ plt.figure(figsize=(10, 5))
70
+ plt.subplot(1, 3, 1)
71
+ plt.imshow(denorm(img0, mean=[0.5, 0.5, 0.5], std=[0.5, 0.5, 0.5]).squeeze().permute(1, 2, 0).cpu().numpy())
72
+ plt.subplot(1, 3, 2)
73
+ plt.imshow(denorm(img1, mean=[0.5, 0.5, 0.5], std=[0.5, 0.5, 0.5]).squeeze().permute(1, 2, 0).cpu().numpy())
74
+ plt.subplot(1, 3, 3)
75
+ plt.imshow(denorm(img2, mean=[0.5, 0.5, 0.5], std=[0.5, 0.5, 0.5]).squeeze().permute(1, 2, 0).cpu().numpy())
76
+ plt.show()
77
+ ```