How to load and run model
#55
by
n8rob
- opened
I am not seeing any instructions in the model card for how to load and run this model. I figured out this way to do it that seems to work okay? Is this right? Or are there any special parameters or configurations I need to set?
from transformers import AutoModelForCausalLM, AutoTokenizer
model = AutoModelForCausalLM.from_pretrained("bigscience/bloom-7b1")
tokenizer = AutoTokenizer.from_pretrained("bigscience/bloom-7b1")
input_ids = tokenizer.encode("Complete this example sentence.", return_tensors="pt")
outs = model.generate(input_ids)
output = tokenizer.decode(outs.squeeze())