GGUF version

#50
by alainlizotte - opened

Could we please have access to a GGUF version of Jina V4 to be used with Ollama ?

V3 never got converted 😞

Jina AI org
edited Jul 3

Hi @alainlizotte , GGUF version is in progress. I will notify you when it's done.

Hi @jupyterjazz any news about the GGUF version?

Jina AI org

Hi @sqliteai , my colleague is working on it along with some other tasks, so I’m not sure exactly when it will be done. Probably this week or next.

any update ?

+1 for GGUF version

Jina AI org

sorry may I ask why jina-embedding-v4-gguf cut visual model out ?
is that gguf format not support ?
I have interest in jina-embedding-v4 , since it state that "multimodal and multilingual retrieval" support, but now gguf format seems not support ?

Jina AI org

image will be back, we just need more test there. image is treated a bit different in gguf.

also we aim to figure out the smallest minimum working unit so that people can deploy it at scale without any redundancy.

thanks, I really no idea on the format issue.
since using LM Studio for gguf format .... thanks for work and I think I will wait you the full function ready before test

Thank you !

Tried it a bit on multilingual documents, mostly French.

Sadly, BGE-M3 is still giving better results for me.

But Jina V4 gave good results. Just not as much complete and accurate than BGE-M3.

Sign up or log in to comment