Unable to call this model using lmdeploy serve api_server.
Traceback (most recent call last):
File "", line 198, in _run_module_as_main
File "", line 88, in run_code
File "E:\open-webui-0.3.35\backend\open_webui\python311\Scripts\lmdeploy.exe_main.py", line 7, in
File "E:\open-webui-0.3.35\backend\open_webui\python311\Lib\site-packages\lmdeploy\cli\entrypoint.py", line 39, in run
args.run(args)
File "E:\open-webui-0.3.35\backend\open_webui\python311\Lib\site-packages\lmdeploy\cli\serve.py", line 318, in api_server
chat_template_config = get_chat_template(args.chat_template)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "E:\open-webui-0.3.35\backend\open_webui\python311\Lib\site-packages\lmdeploy\cli\utils.py", line 75, in get_chat_template
assert chat_template in MODELS.module_dict.keys(),
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: chat template 'skywork-r1v-chat' is not registered. The builtin chat templates are: dict_keys(['base', 'llama', 'deepseek-v3', 'deepseek-r1', 'cogvlm', 'cogvlm2', 'vicuna', 'wizardlm', 'llava-v1', 'mini-gemini-vicuna', 'internlm', 'internlm2', 'internlm3', 'internvl-internlm2', 'internvl2-internlm2', 'internvl2_5', 'internlm-xcomposer2', 'internlm-xcomposer2d5', 'baichuan2', 'puyu', 'llama2', 'llama3', 'llama3_1', 'llama3_2', 'qwen', 'minicpm3', 'minicpmv-2d6', 'qwen2d5', 'qwq_preview', 'qwq', 'codellama', 'falcon', 'chatglm', 'solar', 'ultracm', 'ultralm', 'yi', 'mistral', 'mixtral', 'gemma', 'deepseek', 'internvl-zh', 'deepseek-vl', 'deepseek-vl2', 'deepseek-coder', 'yi-vl', 'dbrx', 'llava-chatml', 'internvl-zh-hermes2', 'phi-3', 'internvl2-phi3', 'chatglm3', 'glm4', 'codegeex4', 'internvl-phi3', 'molmo', 'llama4'])
We have fixed this error.
@sealical The following command works, but the think tag is missing
lmdeploy serve api_server E:\models\awq\Skywork-R1V2-38B-AWQ --backend turbomind --model-format awq --server-port 5001 --chat-template qwq --session-len 70000