Baichuan doesn't work well when use AutoModelForCausalLM.from_config

#43

by baodii - opened Dec 4, 2023

Dec 4, 2023

Hi,
when I use AutoModelForCausalLM.from_config to get Baichuan model, I have found that even though the GenerationConfig.from_model_config function is called here(https://github.com/huggingface/transformers/blob/ef10dbce5cbc9a8b6a0a90b04378ca96f4023aa1/src/transformers/generation/utils.py#L1415), repetition_penalty is not set. So, when we use from_config API, we can't get right accuracy. Please check from_config api for Baichuan.

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment