Update README.md
Browse files
README.md
CHANGED
|
@@ -28,31 +28,4 @@ This is a code LM finetuned(or so-called continue pretrianed) from the 500B Tiny
|
|
| 28 |
|
| 29 |
The HumanEval accuracy is **14**.
|
| 30 |
|
| 31 |
-
|
| 32 |
-
You will need the transformers>=4.31
|
| 33 |
-
Do check the [TinyLlama](https://github.com/jzhang38/TinyLlama) github page for more information.
|
| 34 |
-
```
|
| 35 |
-
from transformers import AutoTokenizer
|
| 36 |
-
import transformers
|
| 37 |
-
import torch
|
| 38 |
-
model = "PY007/TinyLlama-1.1B-intermediate-step-240k-503b"
|
| 39 |
-
tokenizer = AutoTokenizer.from_pretrained(model)
|
| 40 |
-
pipeline = transformers.pipeline(
|
| 41 |
-
"text-generation",
|
| 42 |
-
model=model,
|
| 43 |
-
torch_dtype=torch.float16,
|
| 44 |
-
device_map="auto",
|
| 45 |
-
)
|
| 46 |
-
|
| 47 |
-
sequences = pipeline(
|
| 48 |
-
'The TinyLlama project aims to pretrain a 1.1B Llama model on 3 trillion tokens. With some proper optimization, we can achieve this within a span of "just" 90 days using 16 A100-40G GPUs 🚀🚀. The training has started on 2023-09-01.',
|
| 49 |
-
do_sample=True,
|
| 50 |
-
top_k=10,
|
| 51 |
-
num_return_sequences=1,
|
| 52 |
-
repetition_penalty=1.5,
|
| 53 |
-
eos_token_id=tokenizer.eos_token_id,
|
| 54 |
-
max_length=500,
|
| 55 |
-
)
|
| 56 |
-
for seq in sequences:
|
| 57 |
-
print(f"Result: {seq['generated_text']}")
|
| 58 |
-
```
|
|
|
|
| 28 |
|
| 29 |
The HumanEval accuracy is **14**.
|
| 30 |
|
| 31 |
+
**It can be used as the draft model to speculative-decode larger models such as models in the CodeLlama family**.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|