CPU optimized version

#3
by gururaser - opened

Hi, can you share any CPU optimized version of this model?

HITsz-Text and Multimodal Generative Intelligence Group(TMG) org

Apologies for the delayed response during the holiday period.

We have not yet attempted to convert this model into a CPU-optimized version.
However, we may release an ONNX version of the model in the future for more efficient inference.

HITsz-Text and Multimodal Generative Intelligence Group(TMG) org

We have released the GGUF version: HIT-TMG/KaLM-embedding-multilingual-mini-instruct-v1.5-GGUF, which was converted using llama.cpp.
Currently, it is functioning properly when deployed via llama.cpp, but it has not yet been tested with LM Studio.
If you have any requirements, you are welcome to try this model.

YanshekWoo changed discussion status to closed

Sign up or log in to comment