antonioloison commited on
Commit
1c161e2
·
1 Parent(s): b5279c6

docs: update documentation on how to add your model

Browse files
Files changed (1) hide show
  1. app.py +2 -24
app.py CHANGED
@@ -274,32 +274,10 @@ def main():
274
  To submit a new model to the ViDoRe leaderboard, follow these steps:
275
 
276
  1. **Evaluate your model**:
277
- - Follow the evaluation script provided in the [ViDoRe GitHub repository](https://github.com/illuin-tech/vidore-benchmark/)
278
 
279
  2. **Format your submission file**:
280
- - The submission file should automatically be generated, and named `results.json` with the
281
- following structure:
282
- ```json
283
- {
284
- "dataset_name_1": {
285
- "metric_1": score_1,
286
- "metric_2": score_2,
287
- ...
288
- },
289
- "dataset_name_2": {
290
- "metric_1": score_1,
291
- "metric_2": score_2,
292
- ...
293
- },
294
- }
295
- ```
296
- - The dataset names should be the same as the ViDoRe and ViDoRe 2 dataset names listed in the following
297
- collections: [ViDoRe Benchmark](https://huggingface.co/collections/vidore/vidore-benchmark-667173f98e70a1c0fa4db00d) and [ViDoRe Benchmark 2](vidore/vidore-benchmark-v2-dev-67ae03e3924e85b36e7f53b0).
298
-
299
- 3. **Submit your model**:
300
- - Create a public HuggingFace model repository with your model.
301
- - Add the tag `vidore` to your model in the metadata of the model card and place the
302
- `results.json` file at the root.
303
 
304
  And you're done! Your model will appear on the leaderboard when you click refresh! Once the space
305
  gets rebooted, it will appear on startup.
 
274
  To submit a new model to the ViDoRe leaderboard, follow these steps:
275
 
276
  1. **Evaluate your model**:
277
+ - Follow the evaluation procedure provided in the [ViDoRe GitHub repository](https://github.com/illuin-tech/vidore-benchmark/) that uses MTEB.
278
 
279
  2. **Format your submission file**:
280
+ - Add the generated files to [MTEB results](https://github.com/embeddings-benchmark/results) project. Check the [Colpali results](https://github.com/embeddings-benchmark/results/tree/main/results/vidore__colpali-v1.3/1b5c8929330df1a66de441a9b5409a878f0de5b0) for an example.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
281
 
282
  And you're done! Your model will appear on the leaderboard when you click refresh! Once the space
283
  gets rebooted, it will appear on startup.