axiong

/

PMC_LLaMA_13B

like 32

import transformers
import torch

tokenizer = transformers.LlamaTokenizer.from_pretrained('axiong/PMC_LLaMA_13B')
model = transformers.LlamaForCausalLM.from_pretrained('axiong/PMC_LLaMA_13B')

sentence = 'Hello, doctor' 
batch = tokenizer(
    sentence,
    return_tensors="pt", 
    add_special_tokens=False
)

with torch.no_grad():
    generated = model.generate(
        inputs = batch["input_ids"],
        max_length=200,
        do_sample=True,
        top_k=50
    )
    print('model predict: ',tokenizer.decode(generated[0]))

You are using the legacy behaviour of the <class 'transformers.models.llama.tokenization_llama.LlamaTokenizer'>. This means that tokens that come after special tokens will not be properly handled. We recommend you to read the related pull request available at https://github.com/huggingface/transformers/pull/24565
[2023-09-07 16:22:27,715] [INFO] [real_accelerator.py:158:get_accelerator] Setting ds_accelerator to cuda (auto detect)
Loading checkpoint shards: 100%|████████████████████████████████████████████████████████████████████████████████| 6/6 [02:08<00:00, 21.40s/it]/home/users/aks/miniconda3/envs/llm/lib/python3.11/site-packages/transformers/generation/utils.py:1270: UserWarning: You have modified the pretrained model configuration to control generation. This is a deprecated strategy to control generation and will be removed soon, in a future version. Please use a generation configuration file (see https://huggingface.co/docs/transformers/main_classes/text_generation )
warnings.warn(
model predict: Hello, doctor can you hear voices as well Hello, Hello, doctor can you hear me now patient No, you are a doctor, you are a doctor Hello, can I go outside and buy a sandwich? patient No, not now, you have to take your medications first. Hello, doctor can you hear me now Hello, Hello, can I leave the room now patient No, you have to wait for the medications. Hello, doctor can you hear me now patient Hello, no, you are a patient. You are ill, why are you doing this Hello, can I go outside and buy a sandwich? patient No, you need to take your medications. Hello, can I leave the room now patient No, you have to wait for the medications. patient You are a doctor, why are you doing this Hello, can I go outside and buy a sandwich? patient No, not now, take your medications first. patient You are a doctor, why