Upload README.md with huggingface_hub
Browse files
README.md
CHANGED
|
@@ -6,6 +6,8 @@ tags:
|
|
| 6 |
- draft-model
|
| 7 |
- vllm
|
| 8 |
base_model: thinkingmachines/Inkling-Small-NVFP4
|
|
|
|
|
|
|
| 9 |
---
|
| 10 |
|
| 11 |
# DSpark drafter for Inkling-Small-NVFP4
|
|
@@ -89,6 +91,12 @@ epochs (`checkpoint_best` = best validation epoch). Validation at the selected
|
|
| 89 |
checkpoint: accept_len 3.67, accept_rate 0.40, pos-0 acc 0.776. Loss
|
| 90 |
`{"ce": 0.1, "tv": 0.9}`, lr 1e-4, seq len 8192, block size 16.
|
| 91 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 92 |
Training curves (run `dspark_inkling_small_v2`, logged with trackio):
|
| 93 |
|
| 94 |

|
|
|
|
| 6 |
- draft-model
|
| 7 |
- vllm
|
| 8 |
base_model: thinkingmachines/Inkling-Small-NVFP4
|
| 9 |
+
datasets:
|
| 10 |
+
- orestis-z/Inkling-Small-NVFP4-Regenerated-Collection
|
| 11 |
---
|
| 12 |
|
| 13 |
# DSpark drafter for Inkling-Small-NVFP4
|
|
|
|
| 91 |
checkpoint: accept_len 3.67, accept_rate 0.40, pos-0 acc 0.776. Loss
|
| 92 |
`{"ce": 0.1, "tv": 0.9}`, lr 1e-4, seq len 8192, block size 16.
|
| 93 |
|
| 94 |
+
**Training data:**
|
| 95 |
+
[`orestis-z/Inkling-Small-NVFP4-Regenerated-Collection`](https://huggingface.co/datasets/orestis-z/Inkling-Small-NVFP4-Regenerated-Collection)
|
| 96 |
+
— on-policy data where Inkling-Small-NVFP4 regenerates the assistant responses
|
| 97 |
+
(turn-by-turn, thinking effort 0.9) over Magpie + UltraChat prompts (~500k
|
| 98 |
+
conversations). General chat/instruct mix.
|
| 99 |
+
|
| 100 |
Training curves (run `dspark_inkling_small_v2`, logged with trackio):
|
| 101 |
|
| 102 |

|