Spaces:

Redmind
/

qwen2.5-7b-4bit

Runtime error

Gokulavelan commited on Mar 15

Commit

089d8a1

1 Parent(s): 38d9b9a

model change

Files changed (1) hide show

main.py CHANGED Viewed

@@ -4,7 +4,7 @@ import torch
 app = FastAPI()
-model_name = "mistralai/Mistral-7B-Instruct-v0.1"  # Change to your model
 model = AutoModelForCausalLM.from_pretrained(model_name, torch_dtype=torch.float16)
 tokenizer = AutoTokenizer.from_pretrained(model_name)

 app = FastAPI()
+model_name = "unsloth/Qwen2.5-7B-bnb-4bit"  # Change to your model
 model = AutoModelForCausalLM.from_pretrained(model_name, torch_dtype=torch.float16)
 tokenizer = AutoTokenizer.from_pretrained(model_name)