manueldeprada
/

sampling_with_kvcache_hf_helpers

Text Generation

custom_generate

text-generation-inference

Model card Files Files and versions

manueldeprada HF Staff commited on 1 day ago

Commit

9826914

·

1 Parent(s): 97c3e33

update

Files changed (1) hide show

README.md +1 -1

README.md CHANGED Viewed

@@ -65,7 +65,7 @@ gen_out = model.generate(**inputs, custom_generate="manueldeprada/sampling_with_
 # Get detailed output with probabilities
 gen_out = model.generate(
     **inputs,
-    custom_generate="manueldeprada/sampling_with_kvcache",
     return_dict_in_generate=True,
     trust_remote_code=True
 )

 # Get detailed output with probabilities
 gen_out = model.generate(
     **inputs,
+    custom_generate="manueldeprada/sampling_with_kvcache_hf_helpers",
     return_dict_in_generate=True,
     trust_remote_code=True
 )