kevin009 commited on
Commit
79a49ee
·
verified ·
1 Parent(s): 53ad49a

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +26 -14
README.md CHANGED
@@ -1,22 +1,34 @@
1
  ---
2
- base_model: llama_merged_16bit
3
- tags:
4
- - text-generation-inference
5
- - transformers
6
- - unsloth
7
- - llama
8
- - trl
9
  license: apache-2.0
10
  language:
11
  - en
 
 
 
 
 
 
 
 
12
  ---
13
 
14
- # Uploaded model
15
-
16
- - **Developed by:** kevin009
17
- - **License:** apache-2.0
18
- - **Finetuned from model :** llama_merged_16bit
19
 
20
- This llama model was trained 2x faster with [Unsloth](https://github.com/unslothai/unsloth) and Huggingface's TRL library.
 
21
 
22
- [<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>](https://github.com/unslothai/unsloth)
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
 
 
 
 
 
 
 
2
  license: apache-2.0
3
  language:
4
  - en
5
+ base_model:
6
+ - meta-llama/Llama-3.1-8B-instruct
7
+ pipeline_tag: text-generation
8
+ tags:
9
+ - lora
10
+ - adapter
11
+ - Math
12
+ - CoT
13
  ---
14
 
15
+ ## Model Details
16
+ - Base Model: meta-llama/Llama-3.1-8B-instruct
17
+ - DPO 1300st 1 epoch
18
+ - KTO 800st 2 epoch
 
19
 
20
+ ## Datasets:
21
+ - revision and rewriting txt
22
 
23
+ ### Source Adapters
24
+ All source adapters share the following configuration:
25
+ - Rank (r): 32
26
+ - Alpha: 32
27
+ - Target Modules:
28
+ - q_proj (Query projection)
29
+ - k_proj (Key projection)
30
+ - v_proj (Value projection)
31
+ - o_proj (Output projection)
32
+ - up_proj (Upsampling projection)
33
+ - down_proj (Downsampling projection)
34
+ - gate_proj (Gate projection)