Xenova HF Staff commited on
Commit
e223072
·
verified ·
1 Parent(s): 5d040fc

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +25 -1
README.md CHANGED
@@ -6,7 +6,31 @@ tags: []
6
  # Model Card for Model ID
7
 
8
  <!-- Provide a quick summary of what the model is/does. -->
9
-
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
10
 
11
 
12
  ## Model Details
 
6
  # Model Card for Model ID
7
 
8
  <!-- Provide a quick summary of what the model is/does. -->
9
+ ## Code to generate
10
+ ```python
11
+ import torch
12
+ from transformers import OlmoConfig, OlmoForCausalLM, AutoTokenizer
13
+
14
+
15
+ model_id = 'allenai/OLMo-1B-0724-hf'
16
+ config = OlmoConfig.from_pretrained(
17
+ model_id,
18
+ hidden_size=32,
19
+ intermediate_size=64,
20
+ num_attention_heads=4,
21
+ num_hidden_layers=2,
22
+ num_key_value_heads=4,
23
+ )
24
+
25
+ # Create model and randomize all weights
26
+ model = OlmoForCausalLM(config)
27
+
28
+ torch.manual_seed(0) # Set for reproducibility
29
+ for name, param in model.named_parameters():
30
+ param.data = torch.randn_like(param)
31
+
32
+ tokenizer = AutoTokenizer.from_pretrained(model_id)
33
+ ```
34
 
35
 
36
  ## Model Details