tangledgroup
/

tangled-alpha-0.9-core

Text Generation

Model card Files Files and versions

mtasic85 commited on Mar 15

Commit

1895830

·

1 Parent(s): 041c526

cpt core 4

Files changed (1) hide show

scripts/cpt_core_model_4.py +2 -2

scripts/cpt_core_model_4.py CHANGED Viewed

@@ -111,8 +111,8 @@ trainer = UnslothTrainer(
         fp16=not is_bfloat16_supported(),
         bf16=is_bfloat16_supported(),
         logging_steps=1,
-        # optim='adamw_8bit',
-        optim='adamw_torch_fused',
         weight_decay=0.01,
         lr_scheduler_type='cosine',
         seed=23,

         fp16=not is_bfloat16_supported(),
         bf16=is_bfloat16_supported(),
         logging_steps=1,
+        optim='adamw_8bit',
+        # optim='adamw_torch_fused',
         weight_decay=0.01,
         lr_scheduler_type='cosine',
         seed=23,