doc: update README
Browse files
README.md
CHANGED
|
@@ -24,43 +24,48 @@ You can create your own pictures with [the demo](https://huggingface.co/spaces/f
|
|
| 24 |
|
| 25 |
Refer to [our report](https://wandb.ai/dalle-mini/dalle-mini/reports/DALL-E-mini--Vmlldzo4NjIxODA).
|
| 26 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 27 |
## Development
|
| 28 |
|
| 29 |
### Dependencies Installation
|
| 30 |
|
| 31 |
For inference only, use `pip install git+https://github.com/borisdayma/dalle-mini.git`.
|
| 32 |
|
| 33 |
-
For development, clone the repo and use `pip install -e ".[dev]"`.
|
|
|
|
| 34 |
|
| 35 |
-
###
|
| 36 |
|
| 37 |
-
|
| 38 |
|
| 39 |
-
|
| 40 |
|
| 41 |
-
|
| 42 |
|
| 43 |
-
|
| 44 |
-
|
| 45 |
-
### Training of Seq2Seq
|
| 46 |
|
| 47 |
Use [`tools/train/train.py`](tools/train/train.py).
|
| 48 |
|
| 49 |
You can also adjust the [sweep configuration file](https://docs.wandb.ai/guides/sweeps) if you need to perform a hyperparameter search.
|
| 50 |
|
| 51 |
-
### Inference Pipeline
|
| 52 |
-
|
| 53 |
-
To generate sample predictions and understand the inference pipeline step by step, refer to [`tools/inference/inference_pipeline.ipynb`](tools/inference/inference_pipeline.ipynb).
|
| 54 |
-
|
| 55 |
-
[](https://colab.research.google.com/github/borisdayma/dalle-mini/blob/main/tools/inference/inference_pipeline.ipynb)
|
| 56 |
-
|
| 57 |
## FAQ
|
| 58 |
|
| 59 |
### Where to find the latest models?
|
| 60 |
|
| 61 |
Trained models are on 馃 Model Hub:
|
| 62 |
|
| 63 |
-
- [VQGAN-f16-16384](https://huggingface.co/
|
| 64 |
- [DALL路E mini](https://huggingface.co/flax-community/dalle-mini) for generating images from a text prompt
|
| 65 |
|
| 66 |
### Where does the logo come from?
|
|
@@ -91,11 +96,6 @@ Many thanks to the people who helped make it better:
|
|
| 91 |
- the [DALLE-Pytorch](https://discord.gg/xBPBXfcFHd) and [EleutherAI](https://www.eleuther.ai/) communities for testing and exchanging cool ideas
|
| 92 |
- [Rohan Anil](https://github.com/rohan-anil) for adding Distributed Shampoo optimizer
|
| 93 |
|
| 94 |
-
### Contributing
|
| 95 |
-
|
| 96 |
-
Join the community on the [DALLE-Pytorch Discord](https://discord.gg/xBPBXfcFHd).
|
| 97 |
-
Any contribution is welcome, from reporting issues to proposing fixes/improvements or testing the model with cool prompts!
|
| 98 |
-
|
| 99 |
## Citing DALL路E mini
|
| 100 |
|
| 101 |
If you find DALL路E mini useful in your research or wish to refer, please use the following BibTeX entry.
|
|
|
|
| 24 |
|
| 25 |
Refer to [our report](https://wandb.ai/dalle-mini/dalle-mini/reports/DALL-E-mini--Vmlldzo4NjIxODA).
|
| 26 |
|
| 27 |
+
## Inference Pipeline
|
| 28 |
+
|
| 29 |
+
To generate sample predictions and understand the inference pipeline step by step, refer to [`tools/inference/inference_pipeline.ipynb`](tools/inference/inference_pipeline.ipynb).
|
| 30 |
+
|
| 31 |
+
[](https://colab.research.google.com/github/borisdayma/dalle-mini/blob/main/tools/inference/inference_pipeline.ipynb)
|
| 32 |
+
|
| 33 |
+
## Contributing
|
| 34 |
+
|
| 35 |
+
Join the community on the [DALLE-Pytorch Discord](https://discord.gg/xBPBXfcFHd).
|
| 36 |
+
Any contribution is welcome, from reporting issues to proposing fixes/improvements or testing the model with cool prompts!
|
| 37 |
+
|
| 38 |
+
|
| 39 |
## Development
|
| 40 |
|
| 41 |
### Dependencies Installation
|
| 42 |
|
| 43 |
For inference only, use `pip install git+https://github.com/borisdayma/dalle-mini.git`.
|
| 44 |
|
| 45 |
+
For development, clone the repo and use `pip install -e ".[dev]"`.
|
| 46 |
+
Before making a PR, check style with `make style`.
|
| 47 |
|
| 48 |
+
### Image Encoder
|
| 49 |
|
| 50 |
+
We use a VQGAN from [taming-transformers](https://github.com/CompVis/taming-transformers), which can also be fine-tuned.
|
| 51 |
|
| 52 |
+
Use [patil-suraj/vqgan-jax](https://github.com/patil-suraj/vqgan-jax) if you want to convert a checkpoint to JAX (does not support Gumbel).
|
| 53 |
|
| 54 |
+
Any image encoder that turns an image into a fixed sequence of tokens can be used.
|
| 55 |
|
| 56 |
+
### Training of DALL路E mini
|
|
|
|
|
|
|
| 57 |
|
| 58 |
Use [`tools/train/train.py`](tools/train/train.py).
|
| 59 |
|
| 60 |
You can also adjust the [sweep configuration file](https://docs.wandb.ai/guides/sweeps) if you need to perform a hyperparameter search.
|
| 61 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 62 |
## FAQ
|
| 63 |
|
| 64 |
### Where to find the latest models?
|
| 65 |
|
| 66 |
Trained models are on 馃 Model Hub:
|
| 67 |
|
| 68 |
+
- [VQGAN-f16-16384](https://huggingface.co/dalle-mini/vqgan_imagenet_f16_16384) for encoding/decoding images
|
| 69 |
- [DALL路E mini](https://huggingface.co/flax-community/dalle-mini) for generating images from a text prompt
|
| 70 |
|
| 71 |
### Where does the logo come from?
|
|
|
|
| 96 |
- the [DALLE-Pytorch](https://discord.gg/xBPBXfcFHd) and [EleutherAI](https://www.eleuther.ai/) communities for testing and exchanging cool ideas
|
| 97 |
- [Rohan Anil](https://github.com/rohan-anil) for adding Distributed Shampoo optimizer
|
| 98 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 99 |
## Citing DALL路E mini
|
| 100 |
|
| 101 |
If you find DALL路E mini useful in your research or wish to refer, please use the following BibTeX entry.
|