update new info

README.md

@@ -1,14 +1,12 @@
 ---
 license: apache-2.0
-datasets:
-- togethercomputer/RedPajama-Data-1T
 language:
 - en
-metrics:
-- accuracy
+tags:
+- hawk
 pipeline_tag: text-generation
 ---
-# Model Card for Model ID
+# Hawk-demo
 
 <!-- Provide a quick summary of what the model is/does. -->
 
@@ -22,20 +20,18 @@ This modelcard aims to be a base template for new models. It has been generated
 
 
 
-- **Developed by:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
+- **Developed by:** Rexopia
+- **Reach me:** [email protected]
+- **Language(s) (NLP):** English
+- **License:** Apache license 2.0
+- **Pretrained model [optional]:** True
 
-### Model Sources [optional]
+### Model Sources
 
 <!-- Provide the basic links for the model. -->
 
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
+- **Github Repository:** Coming soon
+- **Demo version:** True
 
 ## Uses
 
@@ -75,7 +71,13 @@ Users (both direct and downstream) should be made aware of the risks, biases and
 
 Use the code below to get started with the model.
 
-[More Information Needed]
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer
+
+tokenizer = AutoTokenizer.from_pretrained("Rexopia/HawkLM-demo", trust_remote_code=True)
+
+model = AutoModelForCausalLM.from_pretrained("Rexopia/HawkLM-demo", device_map="auto", trust_remote_code=True)
+```
 
 ## Training Details
 
@@ -83,7 +85,7 @@ Use the code below to get started with the model.
 
 <!-- This should link to a Data Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
 
-[More Information Needed]
+We sampled from Redpajama 1T datasets without any Arxiv and GitHub tags.
 
 ### Training Procedure 
 

config.json

@@ -12,7 +12,7 @@
   "embd_pdrop": 0.0,
   "eos_token_id": 65535,
   "initializer_range": 0.02,
-  "layer_norm_epsilon": 1e-05,
+  "layer_norm_epsilon": 1e-06,
   "model_type": "hawk",
   "n_embd": 1024,
   "n_head": 16,

tokenizer_config.json

@@ -1,6 +1,12 @@
 {
   "add_bos_token": false,
   "add_prefix_space": false,
+  "auto_map": {
+    "AutoTokenizer": [
+      "tokenization_hawk.HawkTokenizer",
+      null
+    ]
+  },
   "bos_token": {
     "__type": "AddedToken",
     "content": "<s>",
@@ -36,11 +42,5 @@
     "normalized": true,
     "rstrip": false,
     "single_word": false
-  },
-  "auto_map": {
-    "AutoTokenizer": [
-      "tokenization_hawk.HawkTokenizer",
-      null
-    ]
   }
 }