manueldeprada
/

sampling_with_kvcache

Text Generation

custom_generate

text-generation-inference

Model card Files Files and versions

manueldeprada HF Staff commited on 19 days ago

Commit

8f67839

·

1 Parent(s): 880ed7e

Initial commit

Files changed (1) hide show

README.md +3 -3

README.md CHANGED Viewed

@@ -64,9 +64,9 @@ gen_out = model.generate(
     return_dict_in_generate=True,
     trust_remote_code=True
 )
-print(f"Generated text: {tokenizer.batch_decode(gen_out.sequences, skip_special_tokens=True)}")
-print(f"Sampling scores: {gen_out.scores}")
-print(f"Model log probabilities: {gen_out.logps}")
 ```
 ## Algorithm

     return_dict_in_generate=True,
     trust_remote_code=True
 )
+print(f"Generated text: {tokenizer.batch_decode(gen_out['sequences'], skip_special_tokens=True)}")
+print(f"Sampling scores: {gen_out['scores']}")
+print(f"Model log probabilities: {gen_out['logps']}")
 ```
 ## Algorithm