google
/

fnet-base

Transformers

PyTorch

Rust

English

fnet

pretraining

Model card Files Files and versions Community

patrickvonplaten commited on Sep 20, 2021

Commit

332c5af

1 Parent(s): a0b71b6

Update README.md

Browse files

Files changed (1) hide show

README.md +2 -2

README.md CHANGED Viewed

@@ -157,10 +157,10 @@ According to [the official paper](https://arxiv.org/abs/2105.03824) (*cf.* with
 The following table contains test results on the HuggingFace model in comparison with [bert-base-cased](https://hf.co/models/bert-base-cased). The training was done on a single 16GB NVIDIA Tesla V100 GPU. For MRPC/WNLI, the models were trained for 5 epochs, while for other tasks, the models were trained for 3 epochs. Please refer to the checkpoints linked with the scores. The sequence length used for 512 with batch size 16 and learning rate 2e-5.
-| Task  | Metric                 | Result                                                                          |                                                                           | Training time |          |
 | ----- | ---------------------- | ------------------------------------------------------------------------------- | ------------------------------------------------------------------------- | ------------- | -------- |
 |       |                        | Bert (PyTorch) - Reproduced                                                                            | FNet (PyTorch) - Reproduced | FNet (Flax) - Official                                                              | Bert          | FNet     |
-| MNLI  | Accuracy               | [84.10](https://huggingface.co/gchhablani/bert-base-cased-finetuned-mnli)       | [76.75](https://huggingface.co/gchhablani/fnet-base-finetuned-mnli)       |  | 09:52:33      | 06:40:55 |
 | QQP   | mean(Accuracy,F1)            | [89.26](https://huggingface.co/gchhablani/bert-base-cased-finetuned-qqp)  | [86.5](https://huggingface.co/gchhablani/fnet-base-finetuned-qqp)  |  | 09:25:01      | 06:21:16 |
 | QNLI  | Accuracy               | [90.99](https://huggingface.co/gchhablani/bert-base-cased-finetuned-qnli)       | [84.39](https://huggingface.co/gchhablani/fnet-base-finetuned-qnli)       |   |02:40:22      | 01:48:22 |
 | SST-2 | Accuracy               | [92.32](https://huggingface.co/gchhablani/bert-base-cased-finetuned-sst2)       | [89.45](https://huggingface.co/gchhablani/fnet-base-finetuned-sst2)       |   | 01:42:17      | 01:09:27 |

 The following table contains test results on the HuggingFace model in comparison with [bert-base-cased](https://hf.co/models/bert-base-cased). The training was done on a single 16GB NVIDIA Tesla V100 GPU. For MRPC/WNLI, the models were trained for 5 epochs, while for other tasks, the models were trained for 3 epochs. Please refer to the checkpoints linked with the scores. The sequence length used for 512 with batch size 16 and learning rate 2e-5.
+| Task  | Metric                 | Result                                                        |                  |                                                                           | Training time |          |
 | ----- | ---------------------- | ------------------------------------------------------------------------------- | ------------------------------------------------------------------------- | ------------- | -------- |
 |       |                        | Bert (PyTorch) - Reproduced                                                                            | FNet (PyTorch) - Reproduced | FNet (Flax) - Official                                                              | Bert          | FNet     |
+| MNLI  | Accuracy               | [84.10](https://huggingface.co/gchhablani/bert-base-cased-finetuned-mnli)       | [76.75](https://huggingface.co/gchhablani/fnet-base-finetuned-mnli)       | 72/73 (Match/Mismatch) | 09:52:33      | 06:40:55 |
 | QQP   | mean(Accuracy,F1)            | [89.26](https://huggingface.co/gchhablani/bert-base-cased-finetuned-qqp)  | [86.5](https://huggingface.co/gchhablani/fnet-base-finetuned-qqp)  |  | 09:25:01      | 06:21:16 |
 | QNLI  | Accuracy               | [90.99](https://huggingface.co/gchhablani/bert-base-cased-finetuned-qnli)       | [84.39](https://huggingface.co/gchhablani/fnet-base-finetuned-qnli)       |   |02:40:22      | 01:48:22 |
 | SST-2 | Accuracy               | [92.32](https://huggingface.co/gchhablani/bert-base-cased-finetuned-sst2)       | [89.45](https://huggingface.co/gchhablani/fnet-base-finetuned-sst2)       |   | 01:42:17      | 01:09:27 |