parinitarahi commited on
Commit
8456b32
·
verified ·
1 Parent(s): 3559324

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +12 -1
README.md CHANGED
@@ -12,7 +12,18 @@ license: mit
12
  This repository hosts the optimized versions of [DeepSeek-R1-Distill-Qwen-1.5B](https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B/) and [DeepSeek-R1-Distill-Qwen-7B](https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-7B/) to accelerate inference with ONNX Runtime.
13
  Optimized models are published here in [ONNX](https://onnx.ai) format to run with [ONNX Runtime](https://onnxruntime.ai/) on CPU and GPU across devices, including server platforms, Windows, Linux and Mac desktops, and mobile CPUs, with the precision best suited to each of these targets.
14
 
15
- To easily get started with the model, you can use our newly introduced ONNX Runtime Generate() API. See [here](https://aka.ms/generate-tutorial) for instructions on how to run it.
 
 
 
 
 
 
 
 
 
 
 
16
 
17
  ## ONNX Models
18
  Here are some of the optimized configurations we have added:
 
12
  This repository hosts the optimized versions of [DeepSeek-R1-Distill-Qwen-1.5B](https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B/) and [DeepSeek-R1-Distill-Qwen-7B](https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-7B/) to accelerate inference with ONNX Runtime.
13
  Optimized models are published here in [ONNX](https://onnx.ai) format to run with [ONNX Runtime](https://onnxruntime.ai/) on CPU and GPU across devices, including server platforms, Windows, Linux and Mac desktops, and mobile CPUs, with the precision best suited to each of these targets.
14
 
15
+ To easily get started with the model, you can use our newly introduced ONNX Runtime Generate() API.
16
+
17
+ ```bash
18
+ # Download the model directly using the huggingface cli
19
+ huggingface-cli download onnxruntime/DeepSeek-R1-Distill-ONNX --include 'deepseek-r1-distill-qwen-1.5B/*' --local-dir .
20
+ ```
21
+
22
+ ```bash
23
+ # CPU Chat inference. If you pulled the model from huggingface, adjust the model directory (-m) accordingly
24
+ curl -o https://raw.githubusercontent.com/microsoft/onnxruntime-genai/refs/heads/main/examples/python/model-chat.py
25
+ python model-chat.py -m deepseek-r1-distill-qwen-1.5B/model -e cpu --chat_template "<|begin▁of▁sentence|><|User|>{input}<|Assistant|>"
26
+ ```
27
 
28
  ## ONNX Models
29
  Here are some of the optimized configurations we have added: