ahmadisakina commited on
Commit
fbe2800
·
verified ·
1 Parent(s): 471fdbf

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +25 -21
README.md CHANGED
@@ -1,22 +1,26 @@
1
- # Decoder Language Model
2
- Ein kleiner autoregressiver Decoder-only Transformer, trainiert auf Tiny Shakespeare.
3
-
4
- ## Architektur
5
- - d_model=128, num_layers=2, nhead=4
6
- - ~500k Parameter
7
-
8
- ## Metriken
9
- - Loss (Train): 0.6342
10
- - Perplexity (Train): 1.8854
11
-
12
- ## Laden
13
- ```python
14
- from transformers import GPT2Tokenizer
15
- import torch
16
- from model import DecoderLanguageModel
17
-
18
- tokenizer = GPT2Tokenizer.from_pretrained("ahmadisakina/decoder-language-model")
19
- model = DecoderLanguageModel(vocab_size=tokenizer.vocab_size, d_model=128, nhead=4, num_layers=2)
20
- model.load_state_dict(torch.load("pytorch_model.bin"))
21
- model.eval()
 
 
 
 
22
  ```
 
1
+ ---
2
+ language:
3
+ - en
4
+ ---
5
+ # Decoder Language Model
6
+ Ein kleiner autoregressiver Decoder-only Transformer, trainiert auf Tiny Shakespeare.
7
+
8
+ ## Architektur
9
+ - d_model=128, num_layers=2, nhead=4
10
+ - ~500k Parameter
11
+
12
+ ## Metriken
13
+ - Loss (Train): 0.6342
14
+ - Perplexity (Train): 1.8854
15
+
16
+ ## Laden
17
+ ```python
18
+ from transformers import GPT2Tokenizer
19
+ import torch
20
+ from model import DecoderLanguageModel
21
+
22
+ tokenizer = GPT2Tokenizer.from_pretrained("ahmadisakina/decoder-language-model")
23
+ model = DecoderLanguageModel(vocab_size=tokenizer.vocab_size, d_model=128, nhead=4, num_layers=2)
24
+ model.load_state_dict(torch.load("pytorch_model.bin"))
25
+ model.eval()
26
  ```