Error in preprocessing prompt inputs
I started an vllm service using
vllm serve Qwen/Qwen2.5-VL-72B-Instruct
--host 0.0.0.0
--tensor-parallel-size 2
--max-model-len 32768
--gpu-memory-utilization 0.9
--port 8001 > logs/vllm/qwen-vl.log 2>&1 &
Then I use
def get_answer_vl(path):
url = "http://localhost:8001/v1/chat/completions"
headers = {
"Content-Type": "application/json"
}
data = {
"model": "Qwen/Qwen2.5-VL-72B-Instruct",
"messages": [
{
"role": "user",
"content": [
{
"type": "image",
"image": path,
},
{"type": "text", "text": 'Please describe the image'}
]
}
],
}
response = requests.post(url, headers=headers, json=data)
if response.status_code == 200:
print(response.json()['choices'][0]['message']['content'])
return response.json()['choices'][0]['message']['content']
else:
print(f"Error: {response.status_code}, {response.text}")
and I get the following error:
ERROR 02-20 16:18:22 serving_chat.py:193] Error in preprocessing prompt inputs
ERROR 02-20 16:18:22 serving_chat.py:193] Traceback (most recent call last):
ERROR 02-20 16:18:22 serving_chat.py:193] File "/data/l00929485/.local/lib/python3.12/site-packages/vllm/entrypoints/openai/serving_chat.py", line 177, in create_chat_completion
ERROR 02-20 16:18:22 serving_chat.py:193] ) = await self._preprocess_chat(
ERROR 02-20 16:18:22 serving_chat.py:193] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR 02-20 16:18:22 serving_chat.py:193] File "/data/l00929485/.local/lib/python3.12/site-packages/vllm/entrypoints/openai/serving_engine.py", line 386, in _preprocess_chat
ERROR 02-20 16:18:22 serving_chat.py:193] conversation, mm_data_future = parse_chat_messages_futures(
ERROR 02-20 16:18:22 serving_chat.py:193] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR 02-20 16:18:22 serving_chat.py:193] File "/data/l00929485/.local/lib/python3.12/site-packages/vllm/entrypoints/chat_utils.py", line 951, in parse_chat_messages_futures
ERROR 02-20 16:18:22 serving_chat.py:193] sub_messages = _parse_chat_message_content(
ERROR 02-20 16:18:22 serving_chat.py:193] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR 02-20 16:18:22 serving_chat.py:193] File "/data/l00929485/.local/lib/python3.12/site-packages/vllm/entrypoints/chat_utils.py", line 879, in _parse_chat_message_content
ERROR 02-20 16:18:22 serving_chat.py:193] result = _parse_chat_message_content_parts(
ERROR 02-20 16:18:22 serving_chat.py:193] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR 02-20 16:18:22 serving_chat.py:193] File "/data/l00929485/.local/lib/python3.12/site-packages/vllm/entrypoints/chat_utils.py", line 781, in _parse_chat_message_content_parts
ERROR 02-20 16:18:22 serving_chat.py:193] for part in parts:
ERROR 02-20 16:18:22 serving_chat.py:193] ^^^^^
ERROR 02-20 16:18:22 serving_chat.py:193] pydantic_core._pydantic_core.ValidationError: 6 validation errors for ValidatorIterator
ERROR 02-20 16:18:22 serving_chat.py:193] 0.typed-dict.text
ERROR 02-20 16:18:22 serving_chat.py:193] Field required [type=missing, input_value={'type': 'image', 'image': 'test.png'}, input_type=dict]
ERROR 02-20 16:18:22 serving_chat.py:193] For further information visit https://errors.pydantic.dev/2.10/v/missing
ERROR 02-20 16:18:22 serving_chat.py:193] 0.typed-dict.type
ERROR 02-20 16:18:22 serving_chat.py:193] Input should be 'text' [type=literal_error, input_value='image', input_type=str]
ERROR 02-20 16:18:22 serving_chat.py:193] For further information visit https://errors.pydantic.dev/2.10/v/literal_error
ERROR 02-20 16:18:22 serving_chat.py:193] 0.typed-dict.image_url
ERROR 02-20 16:18:22 serving_chat.py:193] Field required [type=missing, input_value={'type': 'image', 'image': 'test.png'}, input_type=dict]
ERROR 02-20 16:18:22 serving_chat.py:193] For further information visit https://errors.pydantic.dev/2.10/v/missing
ERROR 02-20 16:18:22 serving_chat.py:193] 0.typed-dict.type
ERROR 02-20 16:18:22 serving_chat.py:193] Input should be 'image_url' [type=literal_error, input_value='image', input_type=str]
ERROR 02-20 16:18:22 serving_chat.py:193] For further information visit https://errors.pydantic.dev/2.10/v/literal_error
ERROR 02-20 16:18:22 serving_chat.py:193] 0.typed-dict.input_audio
ERROR 02-20 16:18:22 serving_chat.py:193] Field required [type=missing, input_value={'type': 'image', 'image': 'test.png'}, input_type=dict]
ERROR 02-20 16:18:22 serving_chat.py:193] For further information visit https://errors.pydantic.dev/2.10/v/missing
ERROR 02-20 16:18:22 serving_chat.py:193] 0.typed-dict.type
ERROR 02-20 16:18:22 serving_chat.py:193] Input should be 'input_audio' [type=literal_error, input_value='image', input_type=str]
ERROR 02-20 16:18:22 serving_chat.py:193] For further information visit https://errors.pydantic.dev/2.10/v/literal_error