Spaces:
Running
Running
gpantaz
commited on
Commit
·
7b4fa23
1
Parent(s):
b4dc5cb
Add application file
Browse files
README.md
CHANGED
|
@@ -1 +1,12 @@
|
|
| 1 |
-
# tokenization_playground
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# tokenization_playground
|
| 2 |
+
---
|
| 3 |
+
title: Tokenization Playground
|
| 4 |
+
emoji: 📝
|
| 5 |
+
colorFrom: indigo
|
| 6 |
+
colorTo: purple
|
| 7 |
+
sdk: static
|
| 8 |
+
pinned: false
|
| 9 |
+
short_description: Compare different tokenizers
|
| 10 |
+
---
|
| 11 |
+
|
| 12 |
+
Link to source code: https://github.com/athnlp/tokenization_playground
|