orestis-z commited on
Commit
6a33c4d
·
verified ·
1 Parent(s): 018a98b

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +8 -0
README.md CHANGED
@@ -6,6 +6,8 @@ tags:
6
  - draft-model
7
  - vllm
8
  base_model: thinkingmachines/Inkling-Small-NVFP4
 
 
9
  ---
10
 
11
  # DSpark drafter for Inkling-Small-NVFP4
@@ -89,6 +91,12 @@ epochs (`checkpoint_best` = best validation epoch). Validation at the selected
89
  checkpoint: accept_len 3.67, accept_rate 0.40, pos-0 acc 0.776. Loss
90
  `{"ce": 0.1, "tv": 0.9}`, lr 1e-4, seq len 8192, block size 16.
91
 
 
 
 
 
 
 
92
  Training curves (run `dspark_inkling_small_v2`, logged with trackio):
93
 
94
  ![Training curves](./training_curves.png)
 
6
  - draft-model
7
  - vllm
8
  base_model: thinkingmachines/Inkling-Small-NVFP4
9
+ datasets:
10
+ - orestis-z/Inkling-Small-NVFP4-Regenerated-Collection
11
  ---
12
 
13
  # DSpark drafter for Inkling-Small-NVFP4
 
91
  checkpoint: accept_len 3.67, accept_rate 0.40, pos-0 acc 0.776. Loss
92
  `{"ce": 0.1, "tv": 0.9}`, lr 1e-4, seq len 8192, block size 16.
93
 
94
+ **Training data:**
95
+ [`orestis-z/Inkling-Small-NVFP4-Regenerated-Collection`](https://huggingface.co/datasets/orestis-z/Inkling-Small-NVFP4-Regenerated-Collection)
96
+ — on-policy data where Inkling-Small-NVFP4 regenerates the assistant responses
97
+ (turn-by-turn, thinking effort 0.9) over Magpie + UltraChat prompts (~500k
98
+ conversations). General chat/instruct mix.
99
+
100
  Training curves (run `dspark_inkling_small_v2`, logged with trackio):
101
 
102
  ![Training curves](./training_curves.png)