Spaces:
Running
Running
| title: README | |
| emoji: 💻 | |
| colorFrom: purple | |
| colorTo: blue | |
| sdk: static | |
| pinned: false | |
| # Software-Delivered AI Inference | |
| Neural Magic helps developers in accelerating deep learning performance using automated model sparsification technologies and a CPU inference engine. | |
| Download our sparsity-aware inference engine and open source tools for GPU-class performance on CPUs. | |
| * [DeepSparse](https://github.com/neuralmagic/deepsparse): Inference runtime offering GPU-class performance on CPUs and APIs to integrate ML into your application | |
| * [SparseML](https://github.com/neuralmagic/sparseml): Libraries for applying sparsification recipes to neural networks with a few lines of code, enabling faster and smaller models | |
| * [SparseZoo](https://sparsezoo.neuralmagic.com/): Neural network model repository for highly sparse and sparse-quantized models with matching sparsification recipes | |
| #### ✨NEW✨ DeepSparse LLMs | |
| We are excited to announce our paper on Sparse Fine-tuning of LLMs, starting with MosaicML's MPT-7b. Check out the [paper](https://arxiv.org/abs/2310.06927), [models](https://sparsezoo.neuralmagic.com/?datasets=gsm8k&ungrouped=true), and [usage](https://research.neuralmagic.com/mpt-sparse-finetuning). | |