Kwaipilot

/

KAT-V1-40B

like 105

Follow

Kwaipilot 133

+1

+1

Hey, great work.

I would like to know if it is supported in vLLM and if so if the model can do tool call.

Thanks

Yes, our model supports Hermes function calling format and you can use vLLM for inference.
Here is the sample vLLM command:

vllm serve --model Kwaipilot/KAT-V1-40B --tensor-parallel-size 2 --trust-remote-code --enable-auto-tool-choice --tool-call-parser hermes