(*If you experience issues downloading the PDF with Chrome try restarting/updating or use a different browser)
The Nanotron team focus on sharing open knowledge and developping open-source libraries for efficient distributed training of large-scale AI models. Some of its contributions are: - the [Nanotron library](https://github.com/huggingface/nanotron) - the [Picotron library](https://github.com/huggingface/picotron) - the [Ultrascale-Playbook](https://huggingface.co/spaces/nanotron/ultrascale-playbook), a comprehensive book covering all distributed/parallelisation and low-level techniques that can be used to efficiently train models at the largest scales.