Update README.md

README.md

@@ -1,6 +1,6 @@
 ---
 library_name: transformers
-license: apache-2.0
+license: llama3.1
 base_model: Heralax/test-model-4-pretrain
 tags:
 - axolotl
@@ -11,12 +11,17 @@ datasets:
 - pretraining_subset_2170418.jsonl
 - factual_sft_completion/combined_all_0.jsonl
 - factual_sft_completion/combined_all_1.jsonl
-- generic_sft_completion/Augmentoolkit-Augmentoolkit-LMsys-800k-Thoughts_534422.jsonl
-- generic_sft_completion/Augmentoolkit-Augmentoolkit-Generic-Grabbag-Thoughts_1068845.jsonl
-- generic_sft_completion/Augmentoolkit-Augmentoolkit-Capybara-2point5mil-Thoughts_534422.jsonl
+- >-
+  generic_sft_completion/Augmentoolkit-Augmentoolkit-LMsys-800k-Thoughts_534422.jsonl
+- >-
+  generic_sft_completion/Augmentoolkit-Augmentoolkit-Generic-Grabbag-Thoughts_1068845.jsonl
+- >-
+  generic_sft_completion/Augmentoolkit-Augmentoolkit-Capybara-2point5mil-Thoughts_534422.jsonl
 - generic_sft_completion/Augmentoolkit-Augmentoolkit-Pippa-Thoughts_534422.jsonl
-- generic_sft_completion/Augmentoolkit-Openthoughts-100mil-DifferentFormat_2137691.jsonl
-- generic_sft_completion/Augmentoolkit-Augmentoolkit-Bluemoon-1mil-thoughts_534422.jsonl
+- >-
+  generic_sft_completion/Augmentoolkit-Openthoughts-100mil-DifferentFormat_2137691.jsonl
+- >-
+  generic_sft_completion/Augmentoolkit-Augmentoolkit-Bluemoon-1mil-thoughts_534422.jsonl
 model-index:
 - name: test-model-4-sft
   results: []
@@ -24,11 +29,8 @@ model-index:
 
 <!-- This model card has been generated automatically according to the information the Trainer had access to. You
 should probably proofread and complete it, then remove this comment. -->
-
-[<img src="https://raw.githubusercontent.com/axolotl-ai-cloud/axolotl/main/image/axolotl-badge-web.png" alt="Built with Axolotl" width="200" height="32"/>](https://github.com/axolotl-ai-cloud/axolotl)
-<details><summary>See axolotl config</summary>
-
-axolotl version: `0.10.0.dev0`
+<details>
+  
 ```yaml
 base_model: Heralax/test-model-4-pretrain
 tokenizer_type: AutoTokenizer
@@ -110,60 +112,38 @@ wandb_run_id: ''
 wandb_log_model: ''
 hub_model_id: Heralax/test-model-4-sft
 hub_strategy: all_checkpoints
-
 ```
-
 </details><br>
 
-# test-model-4-sft
+# llama-Augmentoolkit-Quickstart-Factual-Demo-Example
 
-This model is a fine-tuned version of [Heralax/test-model-4-pretrain](https://huggingface.co/Heralax/test-model-4-pretrain) on the axolotl_rag_conversations_facts.jsonl, the axolotl_correction_conversations_facts.json, the pretraining_subset_2170418.jsonl, the factual_sft_completion/combined_all_0.jsonl, the factual_sft_completion/combined_all_1.jsonl, the generic_sft_completion/Augmentoolkit-Augmentoolkit-LMsys-800k-Thoughts_534422.jsonl, the generic_sft_completion/Augmentoolkit-Augmentoolkit-Generic-Grabbag-Thoughts_1068845.jsonl, the generic_sft_completion/Augmentoolkit-Augmentoolkit-Capybara-2point5mil-Thoughts_534422.jsonl, the generic_sft_completion/Augmentoolkit-Augmentoolkit-Pippa-Thoughts_534422.jsonl, the generic_sft_completion/Augmentoolkit-Openthoughts-100mil-DifferentFormat_2137691.jsonl and the generic_sft_completion/Augmentoolkit-Augmentoolkit-Bluemoon-1mil-thoughts_534422.jsonl datasets.
-It achieves the following results on the evaluation set:
+This model is achieves the following results on the evaluation set:
 - Loss: 0.6876
 
-## Model description
-
-More information needed
-
-## Intended uses & limitations
+(See? Number go down. Augmentoolkit works.)
 
-More information needed
+This is a demo model produced by running through the quickstart of [Augmentoolkit's](https://github.com/e-p-armstrong/augmentoolkit) Factual Finetuning pipeline. The model was taught about some of the US Army Field Manuals.
 
-## Training and evaluation data
-
-More information needed
-
-## Training procedure
-
-### Training hyperparameters
+The following manuals were trained on:
+```
+ARN14613_FM 1-05 FINAL WEB.pdf.txt		ARN19639_FM 3-14 FINAL WEB.pdf.txt		ARN31505-FM_3-96-000-WEB-1.pdf.txt		ARN34470-FM_6-99-000-WEB-1.pdf.txt		ARN35577-FM_3-55-000-WEB-0.pdf.txt
+ARN15310-FM_3-13.4-000-WEB-2.pdf.txt		ARN21797_FM_3-04_FINAL_WEB_wfix.pdf.txt		ARN33094-FM_3-57-000-WEB-1.pdf.txt		ARN34770-FM_3-94-000-WEB-1.pdf.txt		ARN35791-FM_4-02-001-WEB-3.pdf.txt
+ARN17082-FM_3-11-000-WEB-1.pdf.txt		ARN30964-FM_7-22-001-WEB-4.pdf.txt		ARN33127-FM_3-12-000-WEB-1.pdf.txt		ARN34864-FM_3-61-000-WEB-1.pdf.txt		ARN35838-FM_3-01.44-000-WEB-1.pdf.txt
+ARN19185_FM 6-02_FINAL_WEB.pdf.txt		ARN31339-FM_3-01-000-WEB-1.pdf.txt		ARN33331-FM_1-0-000-WEB-1.pdf.txt		ARN35076-FM_7-0-000-WEB-1.pdf.txt		ARN36290-FM_3-0-000-WEB-2.pdf.txt
+ARN19354_FM 6-27 _C1_FINAL_WEB_v2.pdf.txt	ARN31353-FM_3-34-000-WEB-1.pdf.txt		ARN34192-FM_3-81-000-WEB-1.pdf.txt		ARN35404-FM_6-0-000-WEB-1.pdf.txt		ARN36735-FM_6-22-000-WEB-1.pdf.txt
+```
 
-The following hyperparameters were used during training:
-- learning_rate: 2e-05
-- train_batch_size: 2
-- eval_batch_size: 4
-- seed: 1337
-- gradient_accumulation_steps: 75
-- total_train_batch_size: 150
-- optimizer: Use OptimizerNames.PAGED_ADAMW_8BIT with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
-- lr_scheduler_type: constant
-- lr_scheduler_warmup_steps: 8
-- training_steps: 85
+The `prompt.txt`, `template.txt`, RAG dataset, and GGUF file are all inside this folder so that people can run this model themselves using Augmentoolkit's chat interface. Just download the things not in the checkpoint-xx/ folders (not the model.safetensors files), put them all in a folder, and configure the basic-server or rag-server config to point at the prompt, template, etc., (see the documentation pages for those utility pipelines) and bang, Augmentoolkit will run these models with the correct prompt template and configuration.
 
-### Training results
+Stop sequence == "\*\*Finished.\*\*"
 
-| Training Loss | Epoch  | Step | Validation Loss |
-|:-------------:|:------:|:----:|:---------------:|
-| 1.5402        | 0.0564 | 1    | 1.2586          |
-| 0.5945        | 0.9594 | 17   | 0.5595          |
-| 0.443         | 1.9029 | 34   | 0.5419          |
-| 0.3117        | 2.8465 | 51   | 0.5845          |
-| 0.1713        | 3.7901 | 68   | 0.6350          |
-| 0.1231        | 4.7336 | 85   | 0.6876          |
+Why did I do it like that? Because the more SFT text resembles the pretraining text, the more that knowledge and capabilities from the pretraining will carry over to the SFT. Convention and chatml be damned, I like better performance.
 
+Related Links:
+- [Augmentoolkit](https://github.com/e-p-armstrong/augmentoolkit)
+- [gRPo model (thoughts)](https://huggingface.co/Heralax/llama-gRPo-thoughtprocess)
+- [gRPo model (no thoughts)](https://huggingface.co/Heralax/llama-gRPo-emotions-nothoughts)
 
-### Framework versions
+Q: Why the Llama license?
 
-- Transformers 4.52.3
-- Pytorch 2.6.0+cu124
-- Datasets 3.6.0
-- Tokenizers 0.21.1
+A: The quickstart uses Llama 3 to generate the data for the sake of speed and hardware compatibility. Therefore, the Llama license applies to this demo model.