pengyuan commited on
Commit
59a86a0
·
verified ·
1 Parent(s): b1514a5

Update README.md

Browse files

Update the code example for using AutoProcessor, AutoModelForVision2Seq from transformers (main branch)

Files changed (1) hide show
  1. README.md +7 -5
README.md CHANGED
@@ -48,16 +48,18 @@ The model is intended to be used in enterprise applications that involve process
48
 
49
  ## Generation:
50
 
51
- Granite Vision model is supported natively `transformers>=4.48`. Below is a simple example of how to use the `granite-vision-3.1-2b-preview` model.
52
 
53
  ### Usage with `transformers`
54
 
55
  ```python
56
- from transformers import LlavaNextProcessor, LlavaNextForConditionalGeneration
 
 
57
 
58
  model_path = "ibm-granite/granite-vision-3.1-2b-preview"
59
- processor = LlavaNextProcessor.from_pretrained(model_path)
60
- model = LlavaNextForConditionalGeneration.from_pretrained(model_path, device_map="cuda:0")
61
 
62
  # prepare image and text prompt, using the appropriate prompt template
63
  url = "https://github.com/haotian-liu/LLaVA/blob/1a91fc274d7c35a9b50b3cb29c4247ae5837ce39/images/llava_v1_5_radar.jpg?raw=true"
@@ -77,7 +79,7 @@ inputs = processor.apply_chat_template(
77
  tokenize=True,
78
  return_dict=True,
79
  return_tensors="pt"
80
- ).to("cuda:0")
81
 
82
 
83
  # autoregressively complete prompt
 
48
 
49
  ## Generation:
50
 
51
+ Granite Vision model is supported natively `transformers` from the `main` branch. Below is a simple example of how to use the `granite-vision-3.1-2b-preview` model.
52
 
53
  ### Usage with `transformers`
54
 
55
  ```python
56
+ from transformers import AutoProcessor, AutoModelForVision2Seq
57
+
58
+ device = "cuda" if torch.cuda.is_available() else "cpu"
59
 
60
  model_path = "ibm-granite/granite-vision-3.1-2b-preview"
61
+ processor = AutoProcessor.from_pretrained(model_path)
62
+ model = AutoModelForVision2Seq.from_pretrained(model_path).to(device)
63
 
64
  # prepare image and text prompt, using the appropriate prompt template
65
  url = "https://github.com/haotian-liu/LLaVA/blob/1a91fc274d7c35a9b50b3cb29c4247ae5837ce39/images/llava_v1_5_radar.jpg?raw=true"
 
79
  tokenize=True,
80
  return_dict=True,
81
  return_tensors="pt"
82
+ ).to(device)
83
 
84
 
85
  # autoregressively complete prompt