AkashDataScience commited on
Commit
9aa2b0c
·
1 Parent(s): f5c5cfa

Updated readme

Browse files
Files changed (1) hide show
  1. README.md +10 -1
README.md CHANGED
@@ -10,4 +10,13 @@ pinned: false
10
  license: mit
11
  ---
12
 
13
- Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
 
 
 
 
 
 
 
 
 
 
10
  license: mit
11
  ---
12
 
13
+ # App to visualize results of BPE tokenizer trained on wikipedia and opus dataset.
14
+
15
+ ## Features
16
+ - Input/output text: Write some word(s) and see tokenized output
17
+ - Tokenizer: Select tokenizer
18
+
19
+ ## Usage
20
+ - Enter sentence in Hindi, English or both languages
21
+ - Select tokenizer model
22
+ - Hit submit and see result