2025-06-07 20:23:13,293 - INFO - Converting C:\Users\kunya\PycharmProjects\DataVolt\Tokenization\scientific_corpus_325M.jsonl to Arrow format at scientific_corpus_325M.arrow ...
2025-06-07 20:23:36,951 - ERROR - An error occurred while generating the dataset: An error occurred while generating the dataset
2025-06-07 20:23:36,951 - ERROR - Process failed: An error occurred while generating the dataset
2025-06-07 20:23:36,952 - INFO - Cleaned up local files.