deepugaur commited on
Commit
6049c7f
·
verified ·
1 Parent(s): fdb1a12

Create tokenizer.json

Browse files
Files changed (1) hide show
  1. tokenizer.json +11 -0
tokenizer.json ADDED
@@ -0,0 +1,11 @@
 
 
 
 
 
 
 
 
 
 
 
 
1
+
2
+
3
+
4
+ # Save the tokenizer after training:
5
+ from tensorflow.keras.preprocessing.text import tokenizer_from_json
6
+ import json
7
+
8
+ with open("tokenizer.json", "w") as f:
9
+ json.dump(tokenizer.to_json(), f)
10
+
11
+ # Upload this file (tokenizer.json) to Hugging Face Spaces.