Update README.md
Browse files
README.md
CHANGED
|
@@ -3,9 +3,12 @@ datasets:
|
|
| 3 |
- cerebras/SlimPajama-627B
|
| 4 |
language:
|
| 5 |
- en
|
|
|
|
|
|
|
| 6 |
---
|
| 7 |
|
| 8 |
-
|
|
|
|
| 9 |
|
| 10 |
| Tasks |Version|Filter|n-shot| Metric |Value | |Stderr|
|
| 11 |
|-------------|-------|------|-----:|--------|-----:|---|-----:|
|
|
|
|
| 3 |
- cerebras/SlimPajama-627B
|
| 4 |
language:
|
| 5 |
- en
|
| 6 |
+
tags:
|
| 7 |
+
- llama
|
| 8 |
---
|
| 9 |
|
| 10 |
+
200m-ish parameter model (I think the param count in the graphic here is wrong, but the bench values are correct) with the token embedding and language modelling head of Llama2-70b attached, with linear transformations from Llama2-70b's 8192d space down to this model's 1024d space.
|
| 11 |
+
|
| 12 |
|
| 13 |
| Tasks |Version|Filter|n-shot| Metric |Value | |Stderr|
|
| 14 |
|-------------|-------|------|-----:|--------|-----:|---|-----:|
|