Update README.md
Browse files
README.md
CHANGED
|
@@ -1,3 +1,13 @@
|
|
| 1 |
---
|
|
|
|
|
|
|
| 2 |
license: mit
|
|
|
|
|
|
|
| 3 |
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
---
|
| 2 |
+
language:
|
| 3 |
+
- en
|
| 4 |
license: mit
|
| 5 |
+
tags:
|
| 6 |
+
- english
|
| 7 |
---
|
| 8 |
+
This is a version of the [google/mt5-base](https://huggingface.co/google/mt5-base) model only for English with some embeddings left.
|
| 9 |
+
|
| 10 |
+
|
| 11 |
+
* Using `sentencepiece` vocabulary was shrinking from 250K to 20K (top 20K English tokens) the number of model parameters reduced to 244M parameters, and model size reduced from 2.2GB to 0.873GB - 39% of the original one.
|
| 12 |
+
|
| 13 |
+
Approach was taken from article: [How to adapt a multilingual T5 model for a single language](https://cointegrated.medium.com/how-to-adapt-a-multilingual-t5-model-for-a-single-language-b9f94f3d9c90).
|