EOS_TOKEN_ID ?
My terminators definition is
terminators = [
tokenizer.eos_token_id,
tokenizer.convert_tokens_to_ids("<|endoftext|>")
]
streamer = TextStreamer(processor, skip_prompt=True, skip_special_tokens=True)
outputs = model.generate(
**inputs,
streamer=streamer,
do_sample=True,
temperature=0.0001,
top_p=0.90,
#eos_token_id=terminators, ### I had this initially but after the model response it continues to print some junk words till token max limit
eos_token_id=[8292, 151648,151649,151643,151645,151651,151653], ### I tried getting all end token ids and tried. This also didnt work
repetition_penalty=1.35,
max_new_tokens=2000,
use_audio_in_video=True,
speaker="Ethan"
)
Can you pls assist to define what is eos_token_id to be used here ?