Update README.md
Browse files
README.md
CHANGED
|
@@ -3,4 +3,8 @@
|
|
| 3 |
|
| 4 |
# StableLManticore 7B
|
| 5 |
|
|
|
|
|
|
|
|
|
|
|
|
|
| 6 |
|
|
|
|
| 3 |
|
| 4 |
# StableLManticore 7B
|
| 5 |
|
| 6 |
+
Yeah, don't use this. It was mostly an experiment if it's even plausible. Unfortunately StableLM has poor support for SFT with the huggingface trainer, so no things like flash attention, etc. Ed result is this is nearly impossible to train efficiently.
|
| 7 |
+
Yes, it's plausible to try to train this with LoRA, but it's not very usable at all.
|
| 8 |
+
|
| 9 |
+
WandB: https://wandb.ai/wing-lian/stable-manticore-7b/runs/b1qqzf2s
|
| 10 |
|