Create vidore_eval.md

vidore_eval.md

@@ -0,0 +1,18 @@
+# How to run the Vidore Evaluation
+
+If you want to run the vidore evaluation on the jina-embeddings-v4 model (and on the Document Retrieval Benchmark curated by Jina AI), you need to install requirements in [this fork/branch](https://github.com/jina-ai/vidore-benchmark-fork/tree/feat-add-jina-embeddings) (these changes should be merged with the source code of Vidore soon).
+
+```
+pip install vidore-benchmark[jina-v4]
+```
+
+You can run the evaluation with the following command:
+
+```
+vidore-benchmark evaluate-retriever \
+    --model-class jev4 \
+    --model-name jinaai/jina-embeddings-v4 \
+    --collection-name jinaai/document-screenshot-retrieval-benchmark-small-684831c022c53b21c313b449 \
+    --dataset-format qa \
+    --split test
+```