syntez
/

parva-tokenizer

Model card Files Files and versions

jstephencorey commited on Apr 2, 2023

Commit

d3b3ad5

·

1 Parent(s): 53bbe1a

Update Readme

Files changed (1) hide show

README.md +5 -0

README.md CHANGED Viewed

@@ -1,3 +1,8 @@
 ---
 license: apache-2.0
 ---

 ---
 license: apache-2.0
+datasets:
+- EleutherAI/the_pile
+language:
+- en
 ---
+This is a tokenizer for the Parva models, based off of the GPT-Neox tokenizers