Spaces:
Runtime error
Runtime error
| # Language Modeling with Gated Convolutional Networks (Dauphin et al., 2017) | |
| ## Example usage | |
| First download and preprocess the data following the main [language modeling README](README.md). | |
| Then to train a convolutional LM using the `fconv_lm_dauphin_wikitext103` | |
| architecture: | |
| ```bash | |
| fairseq-train --task language_modeling \ | |
| data-bin/wikitext-103 \ | |
| --save-dir checkpoints/fconv_wikitext-103 \ | |
| --arch fconv_lm_dauphin_wikitext103 \ | |
| --adaptive-softmax-cutoff 10000,20000,200000 \ | |
| --dropout 0.2 \ | |
| --criterion adaptive_loss \ | |
| --optimizer nag --clip-norm 0.1 --weight-decay 5e-06 \ | |
| --lr 1.0 --lr-scheduler reduce_lr_on_plateau --lr-shrink 0.5 \ | |
| --max-tokens 1024 --tokens-per-sample 1024 \ | |
| --ddp-backend legacy_ddp \ | |
| --max-epoch 35 | |
| ``` | |
| And evaluate with: | |
| ```bash | |
| fairseq-eval-lm data-bin/wikitext-103 --path checkpoints/fconv_wiki103/checkpoint_best.pt | |
| ``` | |
| ## Citation | |
| ```bibtex | |
| @inproceedings{dauphin2017language, | |
| title={Language Modeling with Gated Convolutional Networks}, | |
| author={Dauphin, Yann N and Fan, Angela and Auli, Michael and Grangier, David}, | |
| booktitle={Proceedings of the 34th International Conference on Machine Learning-Volume 70}, | |
| pages={933--941}, | |
| year={2017}, | |
| organization={JMLR} | |
| } | |
| ``` | |