tiny-audio-moe-shared

This model is a fine-tuned version of on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.2364

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0003
• train_batch_size: 6
• eval_batch_size: 6
• seed: 42
• gradient_accumulation_steps: 3
• total_train_batch_size: 18
• optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.95) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_steps: 1000
• num_epochs: 1

Training results

Training Loss	Epoch	Step	Validation Loss
0.6015	0.0168	1000	0.5517
0.4664	0.0335	2000	0.3880
0.5131	0.0503	3000	0.3627
0.3958	0.0670	4000	0.3375
0.4086	0.0838	5000	0.3291
0.3871	0.1005	6000	0.3227
0.4305	0.1173	7000	0.3183
0.4414	0.1341	8000	0.3132
0.3828	0.1508	9000	0.3097
0.3562	0.1676	10000	0.2963
0.4145	0.1843	11000	0.3021
0.3633	0.2011	12000	0.2975
0.3743	0.2179	13000	0.2941
0.3715	0.2346	14000	0.2932
0.3788	0.2514	15000	0.2806
0.3532	0.2681	16000	0.2816
0.348	0.2849	17000	0.2816
0.3472	0.3016	18000	0.2765
0.3541	0.3184	19000	0.2714
0.3225	0.3352	20000	0.2744
0.3369	0.3519	21000	0.2738
0.3409	0.3687	22000	0.2723
0.3787	0.3854	23000	0.2684
0.323	0.4022	24000	0.2645
0.3286	0.4189	25000	0.2655
0.379	0.4357	26000	0.2681
0.3253	0.4525	27000	0.2634
0.2972	0.4692	28000	0.2625
0.3249	0.4860	29000	0.2578
0.3264	0.5027	30000	0.2611
0.3591	0.5195	31000	0.2580
0.3503	0.5363	32000	0.2531
0.3184	0.5530	33000	0.2542
0.2927	0.5698	34000	0.2474
0.328	0.5865	35000	0.2503
0.3113	0.6033	36000	0.2509
0.3223	0.6200	37000	0.2476
0.2963	0.6368	38000	0.2457
0.328	0.6536	39000	0.2432
0.2949	0.6703	40000	0.2428
0.3282	0.6871	41000	0.2413
0.3534	0.7038	42000	0.2408
0.3046	0.7206	43000	0.2402
0.3071	0.7374	44000	0.2398
0.3252	0.7541	45000	0.2385
0.339	0.7709	46000	0.2379
0.2964	0.7876	47000	0.2374
0.3086	0.8044	48000	0.2378
0.2796	0.8211	49000	0.2370
0.3071	0.8379	50000	0.2367
0.2864	0.8547	51000	0.2365
0.2841	0.8714	52000	0.2363
0.302	0.8882	53000	0.2363
0.2927	0.9049	54000	0.2364
0.3095	0.9217	55000	0.2364
0.2893	0.9384	56000	0.2364
0.3039	0.9552	57000	0.2363
0.3044	0.9720	58000	0.2364
0.2853	0.9887	59000	0.2364

Framework versions

• Transformers 4.57.3
• Pytorch 2.8.0+cu128
• Datasets 3.6.0
• Tokenizers 0.22.1