david-thrower commited on
Commit
3e4dc8c
·
verified ·
1 Parent(s): a16489c

Update app.py

Browse files

Revert quantization until I can investigate the Exceptions it is raising at generate() time ...

Files changed (1) hide show
  1. app.py +2 -2
app.py CHANGED
@@ -12,8 +12,8 @@ tokenizer = AutoTokenizer.from_pretrained(MODEL_ID)
12
 
13
  model = AutoModelForCausalLM.from_pretrained(
14
  MODEL_ID,
15
- load_in_8bit=True, # or try load_in_4bit=True
16
- device_map=DEVICE
17
  )
18
 
19
  #########
 
12
 
13
  model = AutoModelForCausalLM.from_pretrained(
14
  MODEL_ID,
15
+ # load_in_8bit=True, # or try load_in_4bit=True
16
+ # device_map=DEVICE
17
  )
18
 
19
  #########