binding-case-lstm-3
This model is a fine-tuned version of on the None dataset.
It achieves the following results on the evaluation set:
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 5e-05
- train_batch_size: 32
- eval_batch_size: 32
- seed: 3
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- training_steps: 3052726
Training results
| Training Loss |
Epoch |
Step |
Validation Loss |
| 4.793 |
0.03 |
76320 |
4.7610 |
| 4.5078 |
1.03 |
152640 |
4.4815 |
| 4.3662 |
0.03 |
228960 |
4.3454 |
| 4.2772 |
1.03 |
305280 |
4.2625 |
| 4.2165 |
0.03 |
381600 |
4.2062 |
| 4.1704 |
0.03 |
457920 |
4.1645 |
| 4.1336 |
1.03 |
534240 |
4.1334 |
| 4.0998 |
0.03 |
610560 |
4.1087 |
| 4.0754 |
0.03 |
686880 |
4.0897 |
| 4.0505 |
1.03 |
763200 |
4.0733 |
| 4.0282 |
0.03 |
839520 |
4.0602 |
| 4.0115 |
1.03 |
915840 |
4.0487 |
| 3.9943 |
0.03 |
992160 |
4.0396 |
| 3.9766 |
1.03 |
1068480 |
4.0313 |
| 3.9675 |
0.03 |
1144800 |
4.0246 |
| 3.9447 |
1.03 |
1221120 |
4.0192 |
| 3.9354 |
0.03 |
1297440 |
4.0142 |
| 3.9278 |
1.03 |
1373760 |
4.0092 |
| 3.918 |
0.03 |
1450080 |
4.0055 |
| 3.9146 |
1.03 |
1526400 |
4.0013 |
| 3.9107 |
0.03 |
1602720 |
3.9987 |
| 3.9089 |
1.03 |
1679040 |
3.9956 |
| 3.9035 |
0.03 |
1755360 |
3.9926 |
| 3.898 |
1.03 |
1831680 |
3.9903 |
| 3.8927 |
0.03 |
1908000 |
3.9885 |
| 3.8853 |
1.03 |
1984320 |
3.9868 |
| 3.8795 |
0.03 |
2060640 |
3.9850 |
| 3.876 |
0.03 |
2136960 |
3.9838 |
| 3.871 |
1.03 |
2213280 |
3.9824 |
| 3.8615 |
0.03 |
2289600 |
3.9814 |
| 3.8613 |
1.03 |
2365920 |
3.9803 |
| 3.8485 |
0.03 |
2442240 |
3.9792 |
| 3.8443 |
1.03 |
2518560 |
3.9786 |
| 3.8438 |
0.03 |
2594880 |
3.9778 |
| 3.8407 |
0.03 |
2671200 |
3.9770 |
| 3.842 |
1.03 |
2747520 |
3.9764 |
| 3.8433 |
0.03 |
2823840 |
3.9758 |
| 3.8447 |
0.03 |
2900160 |
3.9755 |
| 3.8456 |
0.03 |
2976480 |
3.9751 |
| 3.8445 |
0.02 |
3052726 |
3.9748 |
Framework versions
- Transformers 4.33.3
- Pytorch 2.0.1
- Datasets 2.12.0
- Tokenizers 0.13.3
