Spaces:
Running
Running
Commit
·
1c161e2
1
Parent(s):
b5279c6
docs: update documentation on how to add your model
Browse files
app.py
CHANGED
@@ -274,32 +274,10 @@ def main():
|
|
274 |
To submit a new model to the ViDoRe leaderboard, follow these steps:
|
275 |
|
276 |
1. **Evaluate your model**:
|
277 |
-
- Follow the evaluation
|
278 |
|
279 |
2. **Format your submission file**:
|
280 |
-
-
|
281 |
-
following structure:
|
282 |
-
```json
|
283 |
-
{
|
284 |
-
"dataset_name_1": {
|
285 |
-
"metric_1": score_1,
|
286 |
-
"metric_2": score_2,
|
287 |
-
...
|
288 |
-
},
|
289 |
-
"dataset_name_2": {
|
290 |
-
"metric_1": score_1,
|
291 |
-
"metric_2": score_2,
|
292 |
-
...
|
293 |
-
},
|
294 |
-
}
|
295 |
-
```
|
296 |
-
- The dataset names should be the same as the ViDoRe and ViDoRe 2 dataset names listed in the following
|
297 |
-
collections: [ViDoRe Benchmark](https://huggingface.co/collections/vidore/vidore-benchmark-667173f98e70a1c0fa4db00d) and [ViDoRe Benchmark 2](vidore/vidore-benchmark-v2-dev-67ae03e3924e85b36e7f53b0).
|
298 |
-
|
299 |
-
3. **Submit your model**:
|
300 |
-
- Create a public HuggingFace model repository with your model.
|
301 |
-
- Add the tag `vidore` to your model in the metadata of the model card and place the
|
302 |
-
`results.json` file at the root.
|
303 |
|
304 |
And you're done! Your model will appear on the leaderboard when you click refresh! Once the space
|
305 |
gets rebooted, it will appear on startup.
|
|
|
274 |
To submit a new model to the ViDoRe leaderboard, follow these steps:
|
275 |
|
276 |
1. **Evaluate your model**:
|
277 |
+
- Follow the evaluation procedure provided in the [ViDoRe GitHub repository](https://github.com/illuin-tech/vidore-benchmark/) that uses MTEB.
|
278 |
|
279 |
2. **Format your submission file**:
|
280 |
+
- Add the generated files to [MTEB results](https://github.com/embeddings-benchmark/results) project. Check the [Colpali results](https://github.com/embeddings-benchmark/results/tree/main/results/vidore__colpali-v1.3/1b5c8929330df1a66de441a9b5409a878f0de5b0) for an example.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
281 |
|
282 |
And you're done! Your model will appear on the leaderboard when you click refresh! Once the space
|
283 |
gets rebooted, it will appear on startup.
|