hf-internal-testing
/

tiny-random-gptj

Feature Extraction

Model card Files Files and versions

joaogante HF Staff commited on Jul 12, 2022

Commit

a96e5bc

·

1 Parent(s): 8ae7319

Update config.json

Files changed (1) hide show

config.json +1 -1

config.json CHANGED Viewed

@@ -20,7 +20,7 @@
   "n_positions": 512,
   "pad_token_id": 98,
   "resid_pdrop": 0.0,
-  "rotary_dim": 64,
   "scale_attn_weights": true,
   "transformers_version": "4.11.0.dev0",
   "type_vocab_size": 16,

   "n_positions": 512,
   "pad_token_id": 98,
   "resid_pdrop": 0.0,
+  "rotary_dim": 4,
   "scale_attn_weights": true,
   "transformers_version": "4.11.0.dev0",
   "type_vocab_size": 16,