Update README.md
Browse filesUpdate the code example for using AutoProcessor, AutoModelForVision2Seq from transformers (main branch)
README.md
CHANGED
@@ -48,16 +48,18 @@ The model is intended to be used in enterprise applications that involve process
|
|
48 |
|
49 |
## Generation:
|
50 |
|
51 |
-
Granite Vision model is supported natively `transformers
|
52 |
|
53 |
### Usage with `transformers`
|
54 |
|
55 |
```python
|
56 |
-
from transformers import
|
|
|
|
|
57 |
|
58 |
model_path = "ibm-granite/granite-vision-3.1-2b-preview"
|
59 |
-
processor =
|
60 |
-
model =
|
61 |
|
62 |
# prepare image and text prompt, using the appropriate prompt template
|
63 |
url = "https://github.com/haotian-liu/LLaVA/blob/1a91fc274d7c35a9b50b3cb29c4247ae5837ce39/images/llava_v1_5_radar.jpg?raw=true"
|
@@ -77,7 +79,7 @@ inputs = processor.apply_chat_template(
|
|
77 |
tokenize=True,
|
78 |
return_dict=True,
|
79 |
return_tensors="pt"
|
80 |
-
).to(
|
81 |
|
82 |
|
83 |
# autoregressively complete prompt
|
|
|
48 |
|
49 |
## Generation:
|
50 |
|
51 |
+
Granite Vision model is supported natively `transformers` from the `main` branch. Below is a simple example of how to use the `granite-vision-3.1-2b-preview` model.
|
52 |
|
53 |
### Usage with `transformers`
|
54 |
|
55 |
```python
|
56 |
+
from transformers import AutoProcessor, AutoModelForVision2Seq
|
57 |
+
|
58 |
+
device = "cuda" if torch.cuda.is_available() else "cpu"
|
59 |
|
60 |
model_path = "ibm-granite/granite-vision-3.1-2b-preview"
|
61 |
+
processor = AutoProcessor.from_pretrained(model_path)
|
62 |
+
model = AutoModelForVision2Seq.from_pretrained(model_path).to(device)
|
63 |
|
64 |
# prepare image and text prompt, using the appropriate prompt template
|
65 |
url = "https://github.com/haotian-liu/LLaVA/blob/1a91fc274d7c35a9b50b3cb29c4247ae5837ce39/images/llava_v1_5_radar.jpg?raw=true"
|
|
|
79 |
tokenize=True,
|
80 |
return_dict=True,
|
81 |
return_tensors="pt"
|
82 |
+
).to(device)
|
83 |
|
84 |
|
85 |
# autoregressively complete prompt
|