Update README.md
Browse files
README.md
CHANGED
|
@@ -15,12 +15,14 @@ pipeline_tag: text-generation
|
|
| 15 |
|
| 16 |
## OpenHermes x Notus x Neural
|
| 17 |
|
| 18 |
-
This is an RL fine tuned OpenHermes using the Intel/orca_dpo_pairs and argilla/ultrafeedback-binarized-preferences preference datasets for reinforcement learning using Direct Preference Optimization (DPO)
|
| 19 |
|
| 20 |
DPOpenHermes is trained using qLoRA. The adapter is also provided in this model repo.
|
| 21 |
|
| 22 |
# Training Details
|
| 23 |
|
|
|
|
|
|
|
| 24 |
DPOpenHermes was trained on a single H100 80GB hosted on RunPod for ~10h for 0.6 epochs of the dataset.
|
| 25 |
|
| 26 |
https://wandb.ai/oaaic/openhermes-dpo/reports/DPOpenHermes--Vmlldzo2MTQ3NDg2
|
|
|
|
| 15 |
|
| 16 |
## OpenHermes x Notus x Neural
|
| 17 |
|
| 18 |
+
This is an RL fine tuned [OpenHermes-2.5-Mistral-7B](https://huggingface.co/teknium/OpenHermes-2.5-Mistral-7B) using the [Intel/orca_dpo_pairs](https://huggingface.co/datasets/Intel/orca_dpo_pairs) and [argilla/ultrafeedback-binarized-preferences](https://huggingface.co/datasets/argilla/ultrafeedback-binarized-preferences) preference datasets for reinforcement learning using Direct Preference Optimization (DPO)
|
| 19 |
|
| 20 |
DPOpenHermes is trained using qLoRA. The adapter is also provided in this model repo.
|
| 21 |
|
| 22 |
# Training Details
|
| 23 |
|
| 24 |
+
[<img src="https://raw.githubusercontent.com/OpenAccess-AI-Collective/axolotl/main/image/axolotl-badge-web.png" alt="Built with Axolotl" width="200" height="32"/>](https://github.com/OpenAccess-AI-Collective/axolotl)
|
| 25 |
+
|
| 26 |
DPOpenHermes was trained on a single H100 80GB hosted on RunPod for ~10h for 0.6 epochs of the dataset.
|
| 27 |
|
| 28 |
https://wandb.ai/oaaic/openhermes-dpo/reports/DPOpenHermes--Vmlldzo2MTQ3NDg2
|