Commit
·
7accec1
1
Parent(s):
ed79b5b
Update README.md
Browse files
README.md
CHANGED
@@ -61,6 +61,13 @@ If you use this ASR pipeline for research, please cite:
|
|
61 |
}
|
62 |
```
|
63 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
64 |
## Evaluation
|
65 |
|
66 |
To evalue this pipeline yourself and/or on your own data, see the `HF Eval Script.ipynb` Jupyter Notebook
|
|
|
61 |
}
|
62 |
```
|
63 |
|
64 |
+
## TEVR Tokenizer Creation / Testing
|
65 |
+
|
66 |
+
See https://huggingface.co/fxtentacle/tevr-token-entropy-predictor-de for:
|
67 |
+
- our trained ByT5 model used to calculate the entropies in the paper
|
68 |
+
- a Jupyter Notebook to generate a TEVR Tokenizer from a text corpus
|
69 |
+
- a Jupyter Notebook to generate the illustration image in the paper
|
70 |
+
|
71 |
## Evaluation
|
72 |
|
73 |
To evalue this pipeline yourself and/or on your own data, see the `HF Eval Script.ipynb` Jupyter Notebook
|