Fixed seed 42？
#2
by
LogINNN
- opened
Hello,
I noticed that the seed is fixed to 42 when generating the mask for each CSI sample. As a result, the masked positions remain the same across all samples. I'm wondering if this might be detrimental to training — or perhaps even a bug?
My understanding is that the masked positions are typically expected to be randomized.
Based on my tests, setting "seed=None" does not seem to significantly affect the training results.
I have another question I’d like to discuss with you. During the pretraining process, I noticed that the training_loss recorded in the CSV file is consistently higher than the validation_loss. Could this be due to the use of dropout?