Spaces:

pyakhurel
/

test-finetuned

Runtime error

pyakhurel commited on Oct 13, 2023

Commit

1f94b56

1 Parent(s): ced2b52

Update app.py

Files changed (1) hide show

app.py CHANGED Viewed

@@ -18,7 +18,7 @@ bnb_config = transformers.BitsAndBytesConfig(
 model = AutoModelForCausalLM.from_pretrained(
     model_name,
-    load_in_4bit=True,
     torch_dtype=torch.bfloat16,
     quantization_config=bnb_config,
     device_map='auto'

 model = AutoModelForCausalLM.from_pretrained(
     model_name,
+    load_in_8bit=True
     torch_dtype=torch.bfloat16,
     quantization_config=bnb_config,
     device_map='auto'