Getting NaN values when trying to finetune
#2
by
imerad7
- opened
Hello,
I'm trying to finetune this pretrained model on a classification task by adding a classification head. However, the training quickly starts giving NaN values after a few iterations. Do you know how this could be fixed ?
Thanks
I found the issue. The classification head parameters were not properly initialized and had absurd values from the start which threw the training off. Manually initializing them solved it.