Can be served?

by prudant - opened Oct 29

Oct 29

vllm, or something like that for production ready high demand scenarios?

Ihor

Knowledgator Engineering org 13 days ago

Hi @prudant , we are working on an easy way to serve these models. I`ll update you when it's ready.

3 days ago

•

@prudant , you can serve it on triton as an onnx model with a python backend ensemble. That is pretty fast. Need higher demand than that?

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment