File size: 441 Bytes
283f87f 50fc442 45b69cb 283f87f |
1 2 3 4 5 6 7 8 9 10 11 |
---
---
# StableLManticore 7B
Yeah, don't use this. It was mostly an experiment if it's even plausible. Unfortunately StableLM has poor support for SFT with the huggingface trainer, so no things like flash attention, etc. Ed result is this is nearly impossible to train efficiently.
Yes, it's plausible to try to train this with LoRA, but it's not very usable at all.
WandB: https://wandb.ai/wing-lian/stable-manticore-7b/runs/b1qqzf2s
|