Hugging Face's logo

Spaces:
ginkgo-datapoints
/
abdev-leaderboard
Running

App Files Community
2
loodvanniekerkginkgo commited on
Commit
fafd10e
·
1 Parent(s): 8dcd98f

Added link to HF sign up

Browse files
Files changed (3) hide show
  1. .gradio/certificate.pem +31 -0
  2. about.py +34 -22
  3. scratch.ipynb +475 -0
.gradio/certificate.pem ADDED
@@ -0,0 +1,31 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ -----BEGIN CERTIFICATE-----
2
+ MIIFazCCA1OgAwIBAgIRAIIQz7DSQONZRGPgu2OCiwAwDQYJKoZIhvcNAQELBQAw
3
+ TzELMAkGA1UEBhMCVVMxKTAnBgNVBAoTIEludGVybmV0IFNlY3VyaXR5IFJlc2Vh
4
+ cmNoIEdyb3VwMRUwEwYDVQQDEwxJU1JHIFJvb3QgWDEwHhcNMTUwNjA0MTEwNDM4
5
+ WhcNMzUwNjA0MTEwNDM4WjBPMQswCQYDVQQGEwJVUzEpMCcGA1UEChMgSW50ZXJu
6
+ ZXQgU2VjdXJpdHkgUmVzZWFyY2ggR3JvdXAxFTATBgNVBAMTDElTUkcgUm9vdCBY
7
+ MTCCAiIwDQYJKoZIhvcNAQEBBQADggIPADCCAgoCggIBAK3oJHP0FDfzm54rVygc
8
+ h77ct984kIxuPOZXoHj3dcKi/vVqbvYATyjb3miGbESTtrFj/RQSa78f0uoxmyF+
9
+ 0TM8ukj13Xnfs7j/EvEhmkvBioZxaUpmZmyPfjxwv60pIgbz5MDmgK7iS4+3mX6U
10
+ A5/TR5d8mUgjU+g4rk8Kb4Mu0UlXjIB0ttov0DiNewNwIRt18jA8+o+u3dpjq+sW
11
+ T8KOEUt+zwvo/7V3LvSye0rgTBIlDHCNAymg4VMk7BPZ7hm/ELNKjD+Jo2FR3qyH
12
+ B5T0Y3HsLuJvW5iB4YlcNHlsdu87kGJ55tukmi8mxdAQ4Q7e2RCOFvu396j3x+UC
13
+ B5iPNgiV5+I3lg02dZ77DnKxHZu8A/lJBdiB3QW0KtZB6awBdpUKD9jf1b0SHzUv
14
+ KBds0pjBqAlkd25HN7rOrFleaJ1/ctaJxQZBKT5ZPt0m9STJEadao0xAH0ahmbWn
15
+ OlFuhjuefXKnEgV4We0+UXgVCwOPjdAvBbI+e0ocS3MFEvzG6uBQE3xDk3SzynTn
16
+ jh8BCNAw1FtxNrQHusEwMFxIt4I7mKZ9YIqioymCzLq9gwQbooMDQaHWBfEbwrbw
17
+ qHyGO0aoSCqI3Haadr8faqU9GY/rOPNk3sgrDQoo//fb4hVC1CLQJ13hef4Y53CI
18
+ rU7m2Ys6xt0nUW7/vGT1M0NPAgMBAAGjQjBAMA4GA1UdDwEB/wQEAwIBBjAPBgNV
19
+ HRMBAf8EBTADAQH/MB0GA1UdDgQWBBR5tFnme7bl5AFzgAiIyBpY9umbbjANBgkq
20
+ hkiG9w0BAQsFAAOCAgEAVR9YqbyyqFDQDLHYGmkgJykIrGF1XIpu+ILlaS/V9lZL
21
+ ubhzEFnTIZd+50xx+7LSYK05qAvqFyFWhfFQDlnrzuBZ6brJFe+GnY+EgPbk6ZGQ
22
+ 3BebYhtF8GaV0nxvwuo77x/Py9auJ/GpsMiu/X1+mvoiBOv/2X/qkSsisRcOj/KK
23
+ NFtY2PwByVS5uCbMiogziUwthDyC3+6WVwW6LLv3xLfHTjuCvjHIInNzktHCgKQ5
24
+ ORAzI4JMPJ+GslWYHb4phowim57iaztXOoJwTdwJx4nLCgdNbOhdjsnvzqvHu7Ur
25
+ TkXWStAmzOVyyghqpZXjFaH3pO3JLF+l+/+sKAIuvtd7u+Nxe5AW0wdeRlN8NwdC
26
+ jNPElpzVmbUq4JUagEiuTDkHzsxHpFKVK7q4+63SM1N95R1NbdWhscdCb+ZAJzVc
27
+ oyi3B43njTOQ5yOf+1CceWxG1bQVs5ZufpsMljq4Ui0/1lvh+wjChP4kqKOJ2qxq
28
+ 4RgqsahDYVvTH9w7jXbyLeiNdd8XM2w9U/t7y0Ff/9yi0GE44Za4rF2LN9d11TPA
29
+ mRGunUHBcnWEvgJBQl9nJEiU0Zsnvgc/ubhPgXRR4Xq37Z0j4r7g1SgEEzwxA57d
30
+ emyPxgcYxn/eR44/KJ4EBs+lVDR3veyJm+kXQ99b21/+jh5Xos1AnX5iItreGCc=
31
+ -----END CERTIFICATE-----
about.py CHANGED
@@ -3,17 +3,18 @@ ABOUT_INTRO = f"""
3
 
4
  ### Register [here](https://datapoints.ginkgo.bio/ai-competitions/2025-abdev-competition)!
5
 
6
- We're inviting the ML/bio community to predict developability properties based on 5 of the assays in the [GDPa1 dataset](https://huggingface.co/datasets/ginkgo-datapoints/GDPa1) and assessed on a private heldout test set.
 
7
 
8
  #### What is antibody developability and why is it important?
9
 
10
- Antibodies have to be manufacturable, stable in high concentrations, and have low off-target effects.
11
  Properties such as these can often hinder the progression of an antibody to the clinic, and are collectively referred to as 'developability'.
12
- Here we show 5 of these properties and invite the community to submit and develop better predictors, which will be tested out on a heldout private set to assess model generalization.
13
 
14
- #### Prizes
15
 
16
- There are up to **$60k in prizes** up for grabs! For each of the 5 properties in the competition, there is a prize for the model with the highest performance for that property on the private test set.
17
  There is also an 'open-source' prize for the best model trained on the GDPa1 dataset (reporting cross-validation results) and assessed on the private test set where authors provide all training code and data.
18
  For each of these 6 prizes, participants have the choice between **$10k in data generation credits** with [Ginkgo Datapoints](https://datapoints.ginkgo.bio/) or a **cash prize** with a value of $2000.
19
  See the FAQ below for more details.
@@ -24,26 +25,20 @@ ABOUT_TEXT = """
24
 
25
  #### How to participate?
26
 
27
- You must submit predictions for a validation set before submitting predictions for the private test set. You could view the public validation set as the "qualifying exam" and the private test set as the "final exam".
28
-
29
- There are two options for validation sets:
30
- - **Track 1**: If you already have a developability model, you can submit your predictions for the GDPa1 public dataset.
31
- Note that for models trained on the Jain dataset (a subset of the GDPa1 dataset), this evaluation will be overoptimistic and the private test set results will likely be lower.
32
- - **Track 2**: If you don't have a model, train one using cross-validation on the GDPa1 dataset and submit your predictions under the "Cross-validation" option.
33
- If trained only on this data, this will also provide you with a more realistic estimate of your model's performance on the private test set.
 
 
 
34
 
35
- Finally, submit your predictions on the heldout private test set. This will not appear on the leaderboard, and will be used to determine the winners at the close of the competition.
36
  We may release private test set results at intermediate points during the competition.
37
 
38
- #### How to submit?
39
-
40
- 1. Create a Hugging Face account if you don't have one yet (this is used to track unique submissions and to access the GDPa1 dataset).
41
- 2. Register your team on the [Competition Registration](https://datapoints.ginkgo.bio/ai-competitions/2025-abdev-competition) page.
42
- 3. Build a model or validate your model using the [GDPa1 dataset](https://huggingface.co/datasets/ginkgo-datapoints/GDPa1).
43
- 4. Make predictions for all the antibody sequences for your properties of interest.
44
- 5. Submit a CSV file containing the `"antibody_name"` column and a column from GDPa1 matching the property names you are predicting (e.g. `"antibody_name,Titer,PR_CHO"` if your model predicts Titer and Polyreactivity).
45
- If you click the "Anonymous" checkbox, your predictions will not be displayed alongside your Hugging Face username but with a random ID.
46
-
47
  There is an example submission file on the "✉️ Submit" tab. When you are ready to submit your predictions for the private test set, download the test set sequences from the "✉️ Submit" tab and follow the same process.
48
 
49
  For the cross-validation metrics (if training only on the GDPa1 dataset), use the `"hierarchical_cluster_IgG_isotype_stratified_fold"` column to split the dataset into folds and make predictions for each of the folds.
@@ -66,6 +61,7 @@ We'd like to add some more existing models to the leaderboard. Some examples of
66
  If you would like to collaborate with others, start a discussion on the "Community" tab at the top of this page.
67
  """
68
  # Note(Lood): Let's track these FAQs in the main Google Doc and have that remain the source of truth.
 
69
  FAQS = {
70
  "Is there a fee to enter?": "No. Participation is free of charge.",
71
  "Who can participate?": "Anyone. We encourage academic labs, individuals, and especially industry teams who use developability models in production.",
@@ -113,4 +109,20 @@ FAQS = {
113
  'and a row with the sequence matching the sequence in the input file. These predictions are then evaluated in the backend using the Spearman rank correlation between predictions and experimental values, and these metrics are then added to the leaderboard. '
114
  'Predictions remain private and are not seen by other contestants.'
115
  ),
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
116
  }
 
3
 
4
  ### Register [here](https://datapoints.ginkgo.bio/ai-competitions/2025-abdev-competition)!
5
 
6
+ - Task: Predict 5 antibody developability properties based on the assays in the [GDPa1 dataset](https://huggingface.co/datasets/ginkgo-datapoints/GDPa1)
7
+ - Submissions are scored on a private held-out test set.
8
 
9
  #### What is antibody developability and why is it important?
10
 
11
+ Antibodies have to be manufacturable, stable in high concentrations, and have low off-target effects.
12
  Properties such as these can often hinder the progression of an antibody to the clinic, and are collectively referred to as 'developability'.
13
+ Here we invite the community to submit and develop better predictors, which will be tested out on a heldout private set to assess model generalization.
14
 
15
+ #### 🏆 Prizes
16
 
17
+ For each of the 5 properties in the competition, there is a prize for the model with the highest performance for that property on the private test set.
18
  There is also an 'open-source' prize for the best model trained on the GDPa1 dataset (reporting cross-validation results) and assessed on the private test set where authors provide all training code and data.
19
  For each of these 6 prizes, participants have the choice between **$10k in data generation credits** with [Ginkgo Datapoints](https://datapoints.ginkgo.bio/) or a **cash prize** with a value of $2000.
20
  See the FAQ below for more details.
 
25
 
26
  #### How to participate?
27
 
28
+ 1. **Create a Hugging Face account** [here](https://huggingface.co/join) if you don't have one yet (this is used to track unique submissions and to access the GDPa1 dataset).
29
+ 2. **Register your team** on the [Competition Registration](https://datapoints.ginkgo.bio/ai-competitions/2025-abdev-competition) page.
30
+ 3. **Build a model** or validate it on the [GDPa1](https://huggingface.co/datasets/ginkgo-datapoints/GDPa1) dataset.
31
+ 4. **Choose a validation track**. You must first upload predictions on the validation set before submitting predictions on the private test set:
32
+ - **Track 1**: If you already have a developability model, you can submit your predictions for the GDPa1 public dataset.
33
+ - **Track 2**: If you don't have a model, train one using cross-validation on the GDPa1 dataset and submit your predictions under the "Cross-validation" option.
34
+ 5. **Submit your predictions** as a CSV with `antibody_name` + one column per property you are predicting (e.g. `"antibody_name,Titer,PR_CHO"` if your model predicts Titer and Polyreactivity).
35
+ You do **not** need to predict all 5 properties — each property has its own leaderboard and prize.
36
+ If you click the "Anonymous" checkbox, your predictions will not be displayed alongside your Hugging Face username but with a random ID.
37
+ 6. **Final test submission**: Download test sequences from the "✉️ Submit" tab and upload predictions.
38
 
39
+ The validation set results will appear on the leaderboard after a few minutes. The private test set results will not appear on the leaderboard, and will be used to determine the winners at the close of the competition.
40
  We may release private test set results at intermediate points during the competition.
41
 
 
 
 
 
 
 
 
 
 
42
  There is an example submission file on the "✉️ Submit" tab. When you are ready to submit your predictions for the private test set, download the test set sequences from the "✉️ Submit" tab and follow the same process.
43
 
44
  For the cross-validation metrics (if training only on the GDPa1 dataset), use the `"hierarchical_cluster_IgG_isotype_stratified_fold"` column to split the dataset into folds and make predictions for each of the folds.
 
61
  If you would like to collaborate with others, start a discussion on the "Community" tab at the top of this page.
62
  """
63
  # Note(Lood): Let's track these FAQs in the main Google Doc and have that remain the source of truth.
64
+ # Note(Lood): Add another note of "many models are trained on different datasets, and differing train/test splits, so this is a consistent way of comparing for a heldout set"
65
  FAQS = {
66
  "Is there a fee to enter?": "No. Participation is free of charge.",
67
  "Who can participate?": "Anyone. We encourage academic labs, individuals, and especially industry teams who use developability models in production.",
 
109
  'and a row with the sequence matching the sequence in the input file. These predictions are then evaluated in the backend using the Spearman rank correlation between predictions and experimental values, and these metrics are then added to the leaderboard. '
110
  'Predictions remain private and are not seen by other contestants.'
111
  ),
112
+ "Can I submit predictions for only one property?": (
113
+ "Yes. You do not need to predict all 5 properties to participate. Each property has its own leaderboard and prize, so you may submit models for a subset of the assays if you wish."
114
+ ),
115
+ "Can I switch between Track 1 and Track 2 during the competition?": (
116
+ "Yes. You may submit to both tracks. For example, you can benchmark an existing model on the GDPa1 dataset (Track 1) and later also train and submit a cross-validation model on GDPa1 (Track 2)."
117
+ ),
118
+ "Are participants required to use the provided cross-validation splits?": (
119
+ "Yes, if submitting cross-validation results, to ensure fair comparison. The results will be calculated by taking the average Spearman correlation coefficient across all folds."
120
+ ),
121
+ "Are there any country restrictions for prize eligibility?": (
122
+ "Yes. Due to applicable laws, prizes cannot be awarded to participants from countries under U.S. sanctions. See the competition terms for details."
123
+ ),
124
+ "How are private test set submissions handled?": (
125
+ "We will use the private test set submission at the close of the competition to determine the winners. "
126
+ "If there are any intermediate releases of private test set results, these will not affect the final ranking."
127
+ ),
128
  }
scratch.ipynb ADDED
@@ -0,0 +1,475 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "cells": [
3
+ {
4
+ "cell_type": "code",
5
+ "execution_count": 1,
6
+ "id": "556cfb74",
7
+ "metadata": {},
8
+ "outputs": [
9
+ {
10
+ "name": "stderr",
11
+ "output_type": "stream",
12
+ "text": [
13
+ "/opt/conda/envs/antibody_datasets/lib/python3.13/site-packages/tqdm/auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html\n",
14
+ " from .autonotebook import tqdm as notebook_tqdm\n"
15
+ ]
16
+ }
17
+ ],
18
+ "source": [
19
+ "from datasets import load_dataset\n",
20
+ "import pandas as pd"
21
+ ]
22
+ },
23
+ {
24
+ "cell_type": "code",
25
+ "execution_count": 2,
26
+ "id": "8b8cea40",
27
+ "metadata": {},
28
+ "outputs": [
29
+ {
30
+ "name": "stderr",
31
+ "output_type": "stream",
32
+ "text": [
33
+ "Generating train split: 100%|██████████| 61/61 [00:00<00:00, 176.99 examples/s]\n"
34
+ ]
35
+ },
36
+ {
37
+ "ename": "ValueError",
38
+ "evalue": "Unknown split \"auto_submissions\". Should be one of ['train'].",
39
+ "output_type": "error",
40
+ "traceback": [
41
+ "\u001b[31m---------------------------------------------------------------------------\u001b[39m",
42
+ "\u001b[31mValueError\u001b[39m Traceback (most recent call last)",
43
+ "\u001b[36mCell\u001b[39m\u001b[36m \u001b[39m\u001b[32mIn[2]\u001b[39m\u001b[32m, line 2\u001b[39m\n\u001b[32m 1\u001b[39m \u001b[38;5;66;03m# access results dataset\u001b[39;00m\n\u001b[32m----> \u001b[39m\u001b[32m2\u001b[39m res = \u001b[43mload_dataset\u001b[49m\u001b[43m(\u001b[49m\u001b[33;43m\"\u001b[39;49m\u001b[33;43mginkgo-datapoints/abdev-bench-results\u001b[39;49m\u001b[33;43m\"\u001b[39;49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43msplit\u001b[49m\u001b[43m=\u001b[49m\u001b[33;43m\"\u001b[39;49m\u001b[33;43mauto_submissions\u001b[39;49m\u001b[33;43m\"\u001b[39;49m\u001b[43m)\u001b[49m\n\u001b[32m 3\u001b[39m \u001b[38;5;28mprint\u001b[39m(res)\n",
44
+ "\u001b[36mFile \u001b[39m\u001b[32m/opt/conda/envs/antibody_datasets/lib/python3.13/site-packages/datasets/load.py:2096\u001b[39m, in \u001b[36mload_dataset\u001b[39m\u001b[34m(path, name, data_dir, data_files, split, cache_dir, features, download_config, download_mode, verification_mode, keep_in_memory, save_infos, revision, token, streaming, num_proc, storage_options, trust_remote_code, **config_kwargs)\u001b[39m\n\u001b[32m 2092\u001b[39m \u001b[38;5;66;03m# Build dataset for splits\u001b[39;00m\n\u001b[32m 2093\u001b[39m keep_in_memory = (\n\u001b[32m 2094\u001b[39m keep_in_memory \u001b[38;5;28;01mif\u001b[39;00m keep_in_memory \u001b[38;5;129;01mis\u001b[39;00m \u001b[38;5;129;01mnot\u001b[39;00m \u001b[38;5;28;01mNone\u001b[39;00m \u001b[38;5;28;01melse\u001b[39;00m is_small_dataset(builder_instance.info.dataset_size)\n\u001b[32m 2095\u001b[39m )\n\u001b[32m-> \u001b[39m\u001b[32m2096\u001b[39m ds = \u001b[43mbuilder_instance\u001b[49m\u001b[43m.\u001b[49m\u001b[43mas_dataset\u001b[49m\u001b[43m(\u001b[49m\u001b[43msplit\u001b[49m\u001b[43m=\u001b[49m\u001b[43msplit\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mverification_mode\u001b[49m\u001b[43m=\u001b[49m\u001b[43mverification_mode\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43min_memory\u001b[49m\u001b[43m=\u001b[49m\u001b[43mkeep_in_memory\u001b[49m\u001b[43m)\u001b[49m\n\u001b[32m 2097\u001b[39m \u001b[38;5;28;01mif\u001b[39;00m save_infos:\n\u001b[32m 2098\u001b[39m builder_instance._save_infos()\n",
45
+ "\u001b[36mFile \u001b[39m\u001b[32m/opt/conda/envs/antibody_datasets/lib/python3.13/site-packages/datasets/builder.py:1127\u001b[39m, in \u001b[36mDatasetBuilder.as_dataset\u001b[39m\u001b[34m(self, split, run_post_process, verification_mode, in_memory)\u001b[39m\n\u001b[32m 1124\u001b[39m verification_mode = VerificationMode(verification_mode \u001b[38;5;129;01mor\u001b[39;00m VerificationMode.BASIC_CHECKS)\n\u001b[32m 1126\u001b[39m \u001b[38;5;66;03m# Create a dataset for each of the given splits\u001b[39;00m\n\u001b[32m-> \u001b[39m\u001b[32m1127\u001b[39m datasets = \u001b[43mmap_nested\u001b[49m\u001b[43m(\u001b[49m\n\u001b[32m 1128\u001b[39m \u001b[43m \u001b[49m\u001b[43mpartial\u001b[49m\u001b[43m(\u001b[49m\n\u001b[32m 1129\u001b[39m \u001b[43m \u001b[49m\u001b[38;5;28;43mself\u001b[39;49m\u001b[43m.\u001b[49m\u001b[43m_build_single_dataset\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 1130\u001b[39m \u001b[43m \u001b[49m\u001b[43mrun_post_process\u001b[49m\u001b[43m=\u001b[49m\u001b[43mrun_post_process\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 1131\u001b[39m \u001b[43m \u001b[49m\u001b[43mverification_mode\u001b[49m\u001b[43m=\u001b[49m\u001b[43mverification_mode\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 1132\u001b[39m \u001b[43m \u001b[49m\u001b[43min_memory\u001b[49m\u001b[43m=\u001b[49m\u001b[43min_memory\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 1133\u001b[39m \u001b[43m \u001b[49m\u001b[43m)\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 1134\u001b[39m \u001b[43m \u001b[49m\u001b[43msplit\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 1135\u001b[39m \u001b[43m \u001b[49m\u001b[43mmap_tuple\u001b[49m\u001b[43m=\u001b[49m\u001b[38;5;28;43;01mTrue\u001b[39;49;00m\u001b[43m,\u001b[49m\n\u001b[32m 1136\u001b[39m \u001b[43m \u001b[49m\u001b[43mdisable_tqdm\u001b[49m\u001b[43m=\u001b[49m\u001b[38;5;28;43;01mTrue\u001b[39;49;00m\u001b[43m,\u001b[49m\n\u001b[32m 1137\u001b[39m \u001b[43m\u001b[49m\u001b[43m)\u001b[49m\n\u001b[32m 1138\u001b[39m \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;28misinstance\u001b[39m(datasets, \u001b[38;5;28mdict\u001b[39m):\n\u001b[32m 1139\u001b[39m datasets = DatasetDict(datasets)\n",
46
+ "\u001b[36mFile \u001b[39m\u001b[32m/opt/conda/envs/antibody_datasets/lib/python3.13/site-packages/datasets/utils/py_utils.py:494\u001b[39m, in \u001b[36mmap_nested\u001b[39m\u001b[34m(function, data_struct, dict_only, map_list, map_tuple, map_numpy, num_proc, parallel_min_length, batched, batch_size, types, disable_tqdm, desc)\u001b[39m\n\u001b[32m 492\u001b[39m \u001b[38;5;28;01mif\u001b[39;00m batched:\n\u001b[32m 493\u001b[39m data_struct = [data_struct]\n\u001b[32m--> \u001b[39m\u001b[32m494\u001b[39m mapped = \u001b[43mfunction\u001b[49m\u001b[43m(\u001b[49m\u001b[43mdata_struct\u001b[49m\u001b[43m)\u001b[49m\n\u001b[32m 495\u001b[39m \u001b[38;5;28;01mif\u001b[39;00m batched:\n\u001b[32m 496\u001b[39m mapped = mapped[\u001b[32m0\u001b[39m]\n",
47
+ "\u001b[36mFile \u001b[39m\u001b[32m/opt/conda/envs/antibody_datasets/lib/python3.13/site-packages/datasets/builder.py:1157\u001b[39m, in \u001b[36mDatasetBuilder._build_single_dataset\u001b[39m\u001b[34m(self, split, run_post_process, verification_mode, in_memory)\u001b[39m\n\u001b[32m 1154\u001b[39m split = Split(split)\n\u001b[32m 1156\u001b[39m \u001b[38;5;66;03m# Build base dataset\u001b[39;00m\n\u001b[32m-> \u001b[39m\u001b[32m1157\u001b[39m ds = \u001b[38;5;28;43mself\u001b[39;49m\u001b[43m.\u001b[49m\u001b[43m_as_dataset\u001b[49m\u001b[43m(\u001b[49m\n\u001b[32m 1158\u001b[39m \u001b[43m \u001b[49m\u001b[43msplit\u001b[49m\u001b[43m=\u001b[49m\u001b[43msplit\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 1159\u001b[39m \u001b[43m \u001b[49m\u001b[43min_memory\u001b[49m\u001b[43m=\u001b[49m\u001b[43min_memory\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 1160\u001b[39m \u001b[43m\u001b[49m\u001b[43m)\u001b[49m\n\u001b[32m 1161\u001b[39m \u001b[38;5;28;01mif\u001b[39;00m run_post_process:\n\u001b[32m 1162\u001b[39m \u001b[38;5;28;01mfor\u001b[39;00m resource_file_name \u001b[38;5;129;01min\u001b[39;00m \u001b[38;5;28mself\u001b[39m._post_processing_resources(split).values():\n",
48
+ "\u001b[36mFile \u001b[39m\u001b[32m/opt/conda/envs/antibody_datasets/lib/python3.13/site-packages/datasets/builder.py:1231\u001b[39m, in \u001b[36mDatasetBuilder._as_dataset\u001b[39m\u001b[34m(self, split, in_memory)\u001b[39m\n\u001b[32m 1229\u001b[39m \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;28mself\u001b[39m._check_legacy_cache():\n\u001b[32m 1230\u001b[39m dataset_name = \u001b[38;5;28mself\u001b[39m.name\n\u001b[32m-> \u001b[39m\u001b[32m1231\u001b[39m dataset_kwargs = \u001b[43mArrowReader\u001b[49m\u001b[43m(\u001b[49m\u001b[43mcache_dir\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;28;43mself\u001b[39;49m\u001b[43m.\u001b[49m\u001b[43minfo\u001b[49m\u001b[43m)\u001b[49m\u001b[43m.\u001b[49m\u001b[43mread\u001b[49m\u001b[43m(\u001b[49m\n\u001b[32m 1232\u001b[39m \u001b[43m \u001b[49m\u001b[43mname\u001b[49m\u001b[43m=\u001b[49m\u001b[43mdataset_name\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 1233\u001b[39m \u001b[43m \u001b[49m\u001b[43minstructions\u001b[49m\u001b[43m=\u001b[49m\u001b[43msplit\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 1234\u001b[39m \u001b[43m \u001b[49m\u001b[43msplit_infos\u001b[49m\u001b[43m=\u001b[49m\u001b[38;5;28;43mself\u001b[39;49m\u001b[43m.\u001b[49m\u001b[43minfo\u001b[49m\u001b[43m.\u001b[49m\u001b[43msplits\u001b[49m\u001b[43m.\u001b[49m\u001b[43mvalues\u001b[49m\u001b[43m(\u001b[49m\u001b[43m)\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 1235\u001b[39m \u001b[43m \u001b[49m\u001b[43min_memory\u001b[49m\u001b[43m=\u001b[49m\u001b[43min_memory\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 1236\u001b[39m \u001b[43m\u001b[49m\u001b[43m)\u001b[49m\n\u001b[32m 1237\u001b[39m fingerprint = \u001b[38;5;28mself\u001b[39m._get_dataset_fingerprint(split)\n\u001b[32m 1238\u001b[39m \u001b[38;5;28;01mreturn\u001b[39;00m Dataset(fingerprint=fingerprint, **dataset_kwargs)\n",
49
+ "\u001b[36mFile \u001b[39m\u001b[32m/opt/conda/envs/antibody_datasets/lib/python3.13/site-packages/datasets/arrow_reader.py:248\u001b[39m, in \u001b[36mBaseReader.read\u001b[39m\u001b[34m(self, name, instructions, split_infos, in_memory)\u001b[39m\n\u001b[32m 227\u001b[39m \u001b[38;5;28;01mdef\u001b[39;00m\u001b[38;5;250m \u001b[39m\u001b[34mread\u001b[39m(\n\u001b[32m 228\u001b[39m \u001b[38;5;28mself\u001b[39m,\n\u001b[32m 229\u001b[39m name,\n\u001b[32m (...)\u001b[39m\u001b[32m 232\u001b[39m in_memory=\u001b[38;5;28;01mFalse\u001b[39;00m,\n\u001b[32m 233\u001b[39m ):\n\u001b[32m 234\u001b[39m \u001b[38;5;250m \u001b[39m\u001b[33;03m\"\"\"Returns Dataset instance(s).\u001b[39;00m\n\u001b[32m 235\u001b[39m \n\u001b[32m 236\u001b[39m \u001b[33;03m Args:\u001b[39;00m\n\u001b[32m (...)\u001b[39m\u001b[32m 245\u001b[39m \u001b[33;03m kwargs to build a single Dataset instance.\u001b[39;00m\n\u001b[32m 246\u001b[39m \u001b[33;03m \"\"\"\u001b[39;00m\n\u001b[32m--> \u001b[39m\u001b[32m248\u001b[39m files = \u001b[38;5;28;43mself\u001b[39;49m\u001b[43m.\u001b[49m\u001b[43mget_file_instructions\u001b[49m\u001b[43m(\u001b[49m\u001b[43mname\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43minstructions\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43msplit_infos\u001b[49m\u001b[43m)\u001b[49m\n\u001b[32m 249\u001b[39m \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;129;01mnot\u001b[39;00m files:\n\u001b[32m 250\u001b[39m msg = \u001b[33mf\u001b[39m\u001b[33m'\u001b[39m\u001b[33mInstruction \u001b[39m\u001b[33m\"\u001b[39m\u001b[38;5;132;01m{\u001b[39;00minstructions\u001b[38;5;132;01m}\u001b[39;00m\u001b[33m\"\u001b[39m\u001b[33m corresponds to no data!\u001b[39m\u001b[33m'\u001b[39m\n",
50
+ "\u001b[36mFile \u001b[39m\u001b[32m/opt/conda/envs/antibody_datasets/lib/python3.13/site-packages/datasets/arrow_reader.py:221\u001b[39m, in \u001b[36mBaseReader.get_file_instructions\u001b[39m\u001b[34m(self, name, instruction, split_infos)\u001b[39m\n\u001b[32m 219\u001b[39m \u001b[38;5;28;01mdef\u001b[39;00m\u001b[38;5;250m \u001b[39m\u001b[34mget_file_instructions\u001b[39m(\u001b[38;5;28mself\u001b[39m, name, instruction, split_infos):\n\u001b[32m 220\u001b[39m \u001b[38;5;250m \u001b[39m\u001b[33;03m\"\"\"Return list of dict {'filename': str, 'skip': int, 'take': int}\"\"\"\u001b[39;00m\n\u001b[32m--> \u001b[39m\u001b[32m221\u001b[39m file_instructions = \u001b[43mmake_file_instructions\u001b[49m\u001b[43m(\u001b[49m\n\u001b[32m 222\u001b[39m \u001b[43m \u001b[49m\u001b[43mname\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43msplit_infos\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43minstruction\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mfiletype_suffix\u001b[49m\u001b[43m=\u001b[49m\u001b[38;5;28;43mself\u001b[39;49m\u001b[43m.\u001b[49m\u001b[43m_filetype_suffix\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mprefix_path\u001b[49m\u001b[43m=\u001b[49m\u001b[38;5;28;43mself\u001b[39;49m\u001b[43m.\u001b[49m\u001b[43m_path\u001b[49m\n\u001b[32m 223\u001b[39m \u001b[43m \u001b[49m\u001b[43m)\u001b[49m\n\u001b[32m 224\u001b[39m files = file_instructions.file_instructions\n\u001b[32m 225\u001b[39m \u001b[38;5;28;01mreturn\u001b[39;00m files\n",
51
+ "\u001b[36mFile \u001b[39m\u001b[32m/opt/conda/envs/antibody_datasets/lib/python3.13/site-packages/datasets/arrow_reader.py:130\u001b[39m, in \u001b[36mmake_file_instructions\u001b[39m\u001b[34m(name, split_infos, instruction, filetype_suffix, prefix_path)\u001b[39m\n\u001b[32m 128\u001b[39m instruction = ReadInstruction.from_spec(instruction)\n\u001b[32m 129\u001b[39m \u001b[38;5;66;03m# Create the absolute instruction (per split)\u001b[39;00m\n\u001b[32m--> \u001b[39m\u001b[32m130\u001b[39m absolute_instructions = \u001b[43minstruction\u001b[49m\u001b[43m.\u001b[49m\u001b[43mto_absolute\u001b[49m\u001b[43m(\u001b[49m\u001b[43mname2len\u001b[49m\u001b[43m)\u001b[49m\n\u001b[32m 132\u001b[39m \u001b[38;5;66;03m# For each split, return the files instruction (skip/take)\u001b[39;00m\n\u001b[32m 133\u001b[39m file_instructions = []\n",
52
+ "\u001b[36mFile \u001b[39m\u001b[32m/opt/conda/envs/antibody_datasets/lib/python3.13/site-packages/datasets/arrow_reader.py:620\u001b[39m, in \u001b[36mReadInstruction.to_absolute\u001b[39m\u001b[34m(self, name2len)\u001b[39m\n\u001b[32m 608\u001b[39m \u001b[38;5;28;01mdef\u001b[39;00m\u001b[38;5;250m \u001b[39m\u001b[34mto_absolute\u001b[39m(\u001b[38;5;28mself\u001b[39m, name2len):\n\u001b[32m 609\u001b[39m \u001b[38;5;250m \u001b[39m\u001b[33;03m\"\"\"Translate instruction into a list of absolute instructions.\u001b[39;00m\n\u001b[32m 610\u001b[39m \n\u001b[32m 611\u001b[39m \u001b[33;03m Those absolute instructions are then to be added together.\u001b[39;00m\n\u001b[32m (...)\u001b[39m\u001b[32m 618\u001b[39m \u001b[33;03m list of _AbsoluteInstruction instances (corresponds to the + in spec).\u001b[39;00m\n\u001b[32m 619\u001b[39m \u001b[33;03m \"\"\"\u001b[39;00m\n\u001b[32m--> \u001b[39m\u001b[32m620\u001b[39m \u001b[38;5;28;01mreturn\u001b[39;00m [\u001b[43m_rel_to_abs_instr\u001b[49m\u001b[43m(\u001b[49m\u001b[43mrel_instr\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mname2len\u001b[49m\u001b[43m)\u001b[49m \u001b[38;5;28;01mfor\u001b[39;00m rel_instr \u001b[38;5;129;01min\u001b[39;00m \u001b[38;5;28mself\u001b[39m._relative_instructions]\n",
53
+ "\u001b[36mFile \u001b[39m\u001b[32m/opt/conda/envs/antibody_datasets/lib/python3.13/site-packages/datasets/arrow_reader.py:437\u001b[39m, in \u001b[36m_rel_to_abs_instr\u001b[39m\u001b[34m(rel_instr, name2len)\u001b[39m\n\u001b[32m 435\u001b[39m split = rel_instr.splitname\n\u001b[32m 436\u001b[39m \u001b[38;5;28;01mif\u001b[39;00m split \u001b[38;5;129;01mnot\u001b[39;00m \u001b[38;5;129;01min\u001b[39;00m name2len:\n\u001b[32m--> \u001b[39m\u001b[32m437\u001b[39m \u001b[38;5;28;01mraise\u001b[39;00m \u001b[38;5;167;01mValueError\u001b[39;00m(\u001b[33mf\u001b[39m\u001b[33m'\u001b[39m\u001b[33mUnknown split \u001b[39m\u001b[33m\"\u001b[39m\u001b[38;5;132;01m{\u001b[39;00msplit\u001b[38;5;132;01m}\u001b[39;00m\u001b[33m\"\u001b[39m\u001b[33m. Should be one of \u001b[39m\u001b[38;5;132;01m{\u001b[39;00m\u001b[38;5;28mlist\u001b[39m(name2len)\u001b[38;5;132;01m}\u001b[39;00m\u001b[33m.\u001b[39m\u001b[33m'\u001b[39m)\n\u001b[32m 438\u001b[39m num_examples = name2len[split]\n\u001b[32m 439\u001b[39m from_ = rel_instr.from_\n",
54
+ "\u001b[31mValueError\u001b[39m: Unknown split \"auto_submissions\". Should be one of ['train']."
55
+ ]
56
+ }
57
+ ],
58
+ "source": [
59
+ "# access results dataset\n",
60
+ "res = load_dataset(\"ginkgo-datapoints/abdev-bench-results\", split=\"auto_submissions\")\n",
61
+ "print(res)"
62
+ ]
63
+ },
64
+ {
65
+ "cell_type": "code",
66
+ "execution_count": 4,
67
+ "id": "2b136f33",
68
+ "metadata": {},
69
+ "outputs": [
70
+ {
71
+ "data": {
72
+ "text/html": [
73
+ "<div>\n",
74
+ "<style scoped>\n",
75
+ " .dataframe tbody tr th:only-of-type {\n",
76
+ " vertical-align: middle;\n",
77
+ " }\n",
78
+ "\n",
79
+ " .dataframe tbody tr th {\n",
80
+ " vertical-align: top;\n",
81
+ " }\n",
82
+ "\n",
83
+ " .dataframe thead th {\n",
84
+ " text-align: right;\n",
85
+ " }\n",
86
+ "</style>\n",
87
+ "<table border=\"1\" class=\"dataframe\">\n",
88
+ " <thead>\n",
89
+ " <tr style=\"text-align: right;\">\n",
90
+ " <th></th>\n",
91
+ " <th>submission_id</th>\n",
92
+ " <th>spearman</th>\n",
93
+ " <th>top_10_recall</th>\n",
94
+ " <th>dataset</th>\n",
95
+ " <th>assay</th>\n",
96
+ " <th>model</th>\n",
97
+ " <th>user</th>\n",
98
+ " <th>submission_time</th>\n",
99
+ " </tr>\n",
100
+ " </thead>\n",
101
+ " <tbody>\n",
102
+ " <tr>\n",
103
+ " <th>0</th>\n",
104
+ " <td>empty</td>\n",
105
+ " <td>0.000000</td>\n",
106
+ " <td>0.000000</td>\n",
107
+ " <td>GDPa1</td>\n",
108
+ " <td>empty</td>\n",
109
+ " <td>empty</td>\n",
110
+ " <td>anonymous</td>\n",
111
+ " <td>NaN</td>\n",
112
+ " </tr>\n",
113
+ " <tr>\n",
114
+ " <th>1</th>\n",
115
+ " <td>08a9b21d-a06f-4c44-a2c2-2d7a03c558c3</td>\n",
116
+ " <td>0.121806</td>\n",
117
+ " <td>0.125000</td>\n",
118
+ " <td>GDPa1</td>\n",
119
+ " <td>Tm2</td>\n",
120
+ " <td>anonymoussubmissions</td>\n",
121
+ " <td>anonymoussubmissions</td>\n",
122
+ " <td>2025-08-13T13:51:50.519786</td>\n",
123
+ " </tr>\n",
124
+ " <tr>\n",
125
+ " <th>2</th>\n",
126
+ " <td>08a9b21d-a06f-4c44-a2c2-2d7a03c558c3</td>\n",
127
+ " <td>0.187877</td>\n",
128
+ " <td>0.083333</td>\n",
129
+ " <td>GDPa1</td>\n",
130
+ " <td>Titer</td>\n",
131
+ " <td>anonymoussubmissions</td>\n",
132
+ " <td>anonymoussubmissions</td>\n",
133
+ " <td>2025-08-13T13:51:50.519786</td>\n",
134
+ " </tr>\n",
135
+ " <tr>\n",
136
+ " <th>3</th>\n",
137
+ " <td>134c9dfb-3b27-48ac-8f5d-c8663d8bebed</td>\n",
138
+ " <td>0.121806</td>\n",
139
+ " <td>0.125000</td>\n",
140
+ " <td>GDPa1</td>\n",
141
+ " <td>Tm2</td>\n",
142
+ " <td>anonymoussubmissions</td>\n",
143
+ " <td>anonymoussubmissions</td>\n",
144
+ " <td>2025-08-13T13:44:10.148599</td>\n",
145
+ " </tr>\n",
146
+ " <tr>\n",
147
+ " <th>4</th>\n",
148
+ " <td>134c9dfb-3b27-48ac-8f5d-c8663d8bebed</td>\n",
149
+ " <td>0.187877</td>\n",
150
+ " <td>0.083333</td>\n",
151
+ " <td>GDPa1</td>\n",
152
+ " <td>Titer</td>\n",
153
+ " <td>anonymoussubmissions</td>\n",
154
+ " <td>anonymoussubmissions</td>\n",
155
+ " <td>2025-08-13T13:44:10.148599</td>\n",
156
+ " </tr>\n",
157
+ " <tr>\n",
158
+ " <th>5</th>\n",
159
+ " <td>3763ce44-0ec5-4eec-80a8-361b2bfe4ee0</td>\n",
160
+ " <td>0.121806</td>\n",
161
+ " <td>0.125000</td>\n",
162
+ " <td>GDPa1</td>\n",
163
+ " <td>Tm2</td>\n",
164
+ " <td>test</td>\n",
165
+ " <td>test</td>\n",
166
+ " <td>2025-08-13T13:46:15.853105</td>\n",
167
+ " </tr>\n",
168
+ " <tr>\n",
169
+ " <th>6</th>\n",
170
+ " <td>3763ce44-0ec5-4eec-80a8-361b2bfe4ee0</td>\n",
171
+ " <td>0.187877</td>\n",
172
+ " <td>0.083333</td>\n",
173
+ " <td>GDPa1</td>\n",
174
+ " <td>Titer</td>\n",
175
+ " <td>test</td>\n",
176
+ " <td>test</td>\n",
177
+ " <td>2025-08-13T13:46:15.853105</td>\n",
178
+ " </tr>\n",
179
+ " <tr>\n",
180
+ " <th>7</th>\n",
181
+ " <td>378b4d52-4d96-40b6-b554-b8f5d8bc5fbd</td>\n",
182
+ " <td>0.121806</td>\n",
183
+ " <td>0.125000</td>\n",
184
+ " <td>GDPa1</td>\n",
185
+ " <td>Tm2</td>\n",
186
+ " <td>test</td>\n",
187
+ " <td>test</td>\n",
188
+ " <td>2025-08-07T19:10:24.934110</td>\n",
189
+ " </tr>\n",
190
+ " <tr>\n",
191
+ " <th>8</th>\n",
192
+ " <td>378b4d52-4d96-40b6-b554-b8f5d8bc5fbd</td>\n",
193
+ " <td>0.187877</td>\n",
194
+ " <td>0.083333</td>\n",
195
+ " <td>GDPa1</td>\n",
196
+ " <td>Titer</td>\n",
197
+ " <td>test</td>\n",
198
+ " <td>test</td>\n",
199
+ " <td>2025-08-07T19:10:24.934110</td>\n",
200
+ " </tr>\n",
201
+ " <tr>\n",
202
+ " <th>9</th>\n",
203
+ " <td>56b4ab17-560e-474b-93f1-ff81fa14fb10</td>\n",
204
+ " <td>0.121806</td>\n",
205
+ " <td>0.125000</td>\n",
206
+ " <td>GDPa1</td>\n",
207
+ " <td>Tm2</td>\n",
208
+ " <td>test</td>\n",
209
+ " <td>test</td>\n",
210
+ " <td>2025-08-12T17:49:22.380229</td>\n",
211
+ " </tr>\n",
212
+ " <tr>\n",
213
+ " <th>10</th>\n",
214
+ " <td>56b4ab17-560e-474b-93f1-ff81fa14fb10</td>\n",
215
+ " <td>0.187877</td>\n",
216
+ " <td>0.083333</td>\n",
217
+ " <td>GDPa1</td>\n",
218
+ " <td>Titer</td>\n",
219
+ " <td>test</td>\n",
220
+ " <td>test</td>\n",
221
+ " <td>2025-08-12T17:49:22.380229</td>\n",
222
+ " </tr>\n",
223
+ " <tr>\n",
224
+ " <th>11</th>\n",
225
+ " <td>7cdcae41-c32a-430a-8a39-be3f00fd315d</td>\n",
226
+ " <td>0.121806</td>\n",
227
+ " <td>0.125000</td>\n",
228
+ " <td>GDPa1</td>\n",
229
+ " <td>Tm2</td>\n",
230
+ " <td>asdfasdfasdfas</td>\n",
231
+ " <td>asdfasdfasdfas</td>\n",
232
+ " <td>2025-08-12T17:53:35.204680</td>\n",
233
+ " </tr>\n",
234
+ " <tr>\n",
235
+ " <th>12</th>\n",
236
+ " <td>7cdcae41-c32a-430a-8a39-be3f00fd315d</td>\n",
237
+ " <td>0.187877</td>\n",
238
+ " <td>0.083333</td>\n",
239
+ " <td>GDPa1</td>\n",
240
+ " <td>Titer</td>\n",
241
+ " <td>asdfasdfasdfas</td>\n",
242
+ " <td>asdfasdfasdfas</td>\n",
243
+ " <td>2025-08-12T17:53:35.204680</td>\n",
244
+ " </tr>\n",
245
+ " <tr>\n",
246
+ " <th>13</th>\n",
247
+ " <td>afe237b4-97cf-4e81-a4c1-f4d6fa76aa04</td>\n",
248
+ " <td>0.121806</td>\n",
249
+ " <td>0.125000</td>\n",
250
+ " <td>GDPa1</td>\n",
251
+ " <td>Tm2</td>\n",
252
+ " <td>anonymoussubmissions</td>\n",
253
+ " <td>anonymoussubmissions</td>\n",
254
+ " <td>2025-08-13T13:43:46.042660</td>\n",
255
+ " </tr>\n",
256
+ " <tr>\n",
257
+ " <th>14</th>\n",
258
+ " <td>afe237b4-97cf-4e81-a4c1-f4d6fa76aa04</td>\n",
259
+ " <td>0.187877</td>\n",
260
+ " <td>0.083333</td>\n",
261
+ " <td>GDPa1</td>\n",
262
+ " <td>Titer</td>\n",
263
+ " <td>anonymoussubmissions</td>\n",
264
+ " <td>anonymoussubmissions</td>\n",
265
+ " <td>2025-08-13T13:43:46.042660</td>\n",
266
+ " </tr>\n",
267
+ " <tr>\n",
268
+ " <th>15</th>\n",
269
+ " <td>b84d6c6c-36d3-42d4-84ad-a91a3758199d</td>\n",
270
+ " <td>0.121806</td>\n",
271
+ " <td>0.125000</td>\n",
272
+ " <td>GDPa1</td>\n",
273
+ " <td>Tm2</td>\n",
274
+ " <td>test</td>\n",
275
+ " <td>test</td>\n",
276
+ " <td>2025-08-13T13:41:41.024660</td>\n",
277
+ " </tr>\n",
278
+ " <tr>\n",
279
+ " <th>16</th>\n",
280
+ " <td>b84d6c6c-36d3-42d4-84ad-a91a3758199d</td>\n",
281
+ " <td>0.187877</td>\n",
282
+ " <td>0.083333</td>\n",
283
+ " <td>GDPa1</td>\n",
284
+ " <td>Titer</td>\n",
285
+ " <td>test</td>\n",
286
+ " <td>test</td>\n",
287
+ " <td>2025-08-13T13:41:41.024660</td>\n",
288
+ " </tr>\n",
289
+ " <tr>\n",
290
+ " <th>17</th>\n",
291
+ " <td>d2804c50-33a8-4465-8e27-20762adda13e</td>\n",
292
+ " <td>0.358522</td>\n",
293
+ " <td>0.208333</td>\n",
294
+ " <td>GDPa1</td>\n",
295
+ " <td>HIC</td>\n",
296
+ " <td>notmyusername_test</td>\n",
297
+ " <td>notmyusername_test</td>\n",
298
+ " <td>2025-07-24T20:56:08.953098</td>\n",
299
+ " </tr>\n",
300
+ " <tr>\n",
301
+ " <th>18</th>\n",
302
+ " <td>d2804c50-33a8-4465-8e27-20762adda13e</td>\n",
303
+ " <td>0.358522</td>\n",
304
+ " <td>0.208333</td>\n",
305
+ " <td>GDPa1</td>\n",
306
+ " <td>HIC</td>\n",
307
+ " <td>notmyusername_test</td>\n",
308
+ " <td>notmyusername_test</td>\n",
309
+ " <td>2025-07-24T20:56:08.953098</td>\n",
310
+ " </tr>\n",
311
+ " <tr>\n",
312
+ " <th>19</th>\n",
313
+ " <td>d82e3ef6-e54c-4854-b8b7-bee28f04791e</td>\n",
314
+ " <td>0.121806</td>\n",
315
+ " <td>0.125000</td>\n",
316
+ " <td>GDPa1</td>\n",
317
+ " <td>Tm2</td>\n",
318
+ " <td>test</td>\n",
319
+ " <td>test</td>\n",
320
+ " <td>2025-08-12T17:47:17.935587</td>\n",
321
+ " </tr>\n",
322
+ " <tr>\n",
323
+ " <th>20</th>\n",
324
+ " <td>d82e3ef6-e54c-4854-b8b7-bee28f04791e</td>\n",
325
+ " <td>0.187877</td>\n",
326
+ " <td>0.083333</td>\n",
327
+ " <td>GDPa1</td>\n",
328
+ " <td>Titer</td>\n",
329
+ " <td>test</td>\n",
330
+ " <td>test</td>\n",
331
+ " <td>2025-08-12T17:47:17.935587</td>\n",
332
+ " </tr>\n",
333
+ " <tr>\n",
334
+ " <th>21</th>\n",
335
+ " <td>fa4d1c11-770d-40a8-b846-12e219b4b87a</td>\n",
336
+ " <td>0.121806</td>\n",
337
+ " <td>0.125000</td>\n",
338
+ " <td>GDPa1</td>\n",
339
+ " <td>Tm2</td>\n",
340
+ " <td>test</td>\n",
341
+ " <td>test</td>\n",
342
+ " <td>2025-08-12T17:49:20.145092</td>\n",
343
+ " </tr>\n",
344
+ " <tr>\n",
345
+ " <th>22</th>\n",
346
+ " <td>fa4d1c11-770d-40a8-b846-12e219b4b87a</td>\n",
347
+ " <td>0.187877</td>\n",
348
+ " <td>0.083333</td>\n",
349
+ " <td>GDPa1</td>\n",
350
+ " <td>Titer</td>\n",
351
+ " <td>test</td>\n",
352
+ " <td>test</td>\n",
353
+ " <td>2025-08-12T17:49:20.145092</td>\n",
354
+ " </tr>\n",
355
+ " </tbody>\n",
356
+ "</table>\n",
357
+ "</div>"
358
+ ],
359
+ "text/plain": [
360
+ " submission_id spearman top_10_recall dataset \\\n",
361
+ "0 empty 0.000000 0.000000 GDPa1 \n",
362
+ "1 08a9b21d-a06f-4c44-a2c2-2d7a03c558c3 0.121806 0.125000 GDPa1 \n",
363
+ "2 08a9b21d-a06f-4c44-a2c2-2d7a03c558c3 0.187877 0.083333 GDPa1 \n",
364
+ "3 134c9dfb-3b27-48ac-8f5d-c8663d8bebed 0.121806 0.125000 GDPa1 \n",
365
+ "4 134c9dfb-3b27-48ac-8f5d-c8663d8bebed 0.187877 0.083333 GDPa1 \n",
366
+ "5 3763ce44-0ec5-4eec-80a8-361b2bfe4ee0 0.121806 0.125000 GDPa1 \n",
367
+ "6 3763ce44-0ec5-4eec-80a8-361b2bfe4ee0 0.187877 0.083333 GDPa1 \n",
368
+ "7 378b4d52-4d96-40b6-b554-b8f5d8bc5fbd 0.121806 0.125000 GDPa1 \n",
369
+ "8 378b4d52-4d96-40b6-b554-b8f5d8bc5fbd 0.187877 0.083333 GDPa1 \n",
370
+ "9 56b4ab17-560e-474b-93f1-ff81fa14fb10 0.121806 0.125000 GDPa1 \n",
371
+ "10 56b4ab17-560e-474b-93f1-ff81fa14fb10 0.187877 0.083333 GDPa1 \n",
372
+ "11 7cdcae41-c32a-430a-8a39-be3f00fd315d 0.121806 0.125000 GDPa1 \n",
373
+ "12 7cdcae41-c32a-430a-8a39-be3f00fd315d 0.187877 0.083333 GDPa1 \n",
374
+ "13 afe237b4-97cf-4e81-a4c1-f4d6fa76aa04 0.121806 0.125000 GDPa1 \n",
375
+ "14 afe237b4-97cf-4e81-a4c1-f4d6fa76aa04 0.187877 0.083333 GDPa1 \n",
376
+ "15 b84d6c6c-36d3-42d4-84ad-a91a3758199d 0.121806 0.125000 GDPa1 \n",
377
+ "16 b84d6c6c-36d3-42d4-84ad-a91a3758199d 0.187877 0.083333 GDPa1 \n",
378
+ "17 d2804c50-33a8-4465-8e27-20762adda13e 0.358522 0.208333 GDPa1 \n",
379
+ "18 d2804c50-33a8-4465-8e27-20762adda13e 0.358522 0.208333 GDPa1 \n",
380
+ "19 d82e3ef6-e54c-4854-b8b7-bee28f04791e 0.121806 0.125000 GDPa1 \n",
381
+ "20 d82e3ef6-e54c-4854-b8b7-bee28f04791e 0.187877 0.083333 GDPa1 \n",
382
+ "21 fa4d1c11-770d-40a8-b846-12e219b4b87a 0.121806 0.125000 GDPa1 \n",
383
+ "22 fa4d1c11-770d-40a8-b846-12e219b4b87a 0.187877 0.083333 GDPa1 \n",
384
+ "\n",
385
+ " assay model user \\\n",
386
+ "0 empty empty anonymous \n",
387
+ "1 Tm2 anonymoussubmissions anonymoussubmissions \n",
388
+ "2 Titer anonymoussubmissions anonymoussubmissions \n",
389
+ "3 Tm2 anonymoussubmissions anonymoussubmissions \n",
390
+ "4 Titer anonymoussubmissions anonymoussubmissions \n",
391
+ "5 Tm2 test test \n",
392
+ "6 Titer test test \n",
393
+ "7 Tm2 test test \n",
394
+ "8 Titer test test \n",
395
+ "9 Tm2 test test \n",
396
+ "10 Titer test test \n",
397
+ "11 Tm2 asdfasdfasdfas asdfasdfasdfas \n",
398
+ "12 Titer asdfasdfasdfas asdfasdfasdfas \n",
399
+ "13 Tm2 anonymoussubmissions anonymoussubmissions \n",
400
+ "14 Titer anonymoussubmissions anonymoussubmissions \n",
401
+ "15 Tm2 test test \n",
402
+ "16 Titer test test \n",
403
+ "17 HIC notmyusername_test notmyusername_test \n",
404
+ "18 HIC notmyusername_test notmyusername_test \n",
405
+ "19 Tm2 test test \n",
406
+ "20 Titer test test \n",
407
+ "21 Tm2 test test \n",
408
+ "22 Titer test test \n",
409
+ "\n",
410
+ " submission_time \n",
411
+ "0 NaN \n",
412
+ "1 2025-08-13T13:51:50.519786 \n",
413
+ "2 2025-08-13T13:51:50.519786 \n",
414
+ "3 2025-08-13T13:44:10.148599 \n",
415
+ "4 2025-08-13T13:44:10.148599 \n",
416
+ "5 2025-08-13T13:46:15.853105 \n",
417
+ "6 2025-08-13T13:46:15.853105 \n",
418
+ "7 2025-08-07T19:10:24.934110 \n",
419
+ "8 2025-08-07T19:10:24.934110 \n",
420
+ "9 2025-08-12T17:49:22.380229 \n",
421
+ "10 2025-08-12T17:49:22.380229 \n",
422
+ "11 2025-08-12T17:53:35.204680 \n",
423
+ "12 2025-08-12T17:53:35.204680 \n",
424
+ "13 2025-08-13T13:43:46.042660 \n",
425
+ "14 2025-08-13T13:43:46.042660 \n",
426
+ "15 2025-08-13T13:41:41.024660 \n",
427
+ "16 2025-08-13T13:41:41.024660 \n",
428
+ "17 2025-07-24T20:56:08.953098 \n",
429
+ "18 2025-07-24T20:56:08.953098 \n",
430
+ "19 2025-08-12T17:47:17.935587 \n",
431
+ "20 2025-08-12T17:47:17.935587 \n",
432
+ "21 2025-08-12T17:49:20.145092 \n",
433
+ "22 2025-08-12T17:49:20.145092 "
434
+ ]
435
+ },
436
+ "execution_count": 4,
437
+ "metadata": {},
438
+ "output_type": "execute_result"
439
+ }
440
+ ],
441
+ "source": [
442
+ "pd.read_csv(\"hf://datasets/ginkgo-datapoints/abdev-bench-results/auto_submissions/metrics_all.csv\")"
443
+ ]
444
+ },
445
+ {
446
+ "cell_type": "code",
447
+ "execution_count": null,
448
+ "id": "6c7e0ad6",
449
+ "metadata": {},
450
+ "outputs": [],
451
+ "source": []
452
+ }
453
+ ],
454
+ "metadata": {
455
+ "kernelspec": {
456
+ "display_name": "antibody_datasets",
457
+ "language": "python",
458
+ "name": "python3"
459
+ },
460
+ "language_info": {
461
+ "codemirror_mode": {
462
+ "name": "ipython",
463
+ "version": 3
464
+ },
465
+ "file_extension": ".py",
466
+ "mimetype": "text/x-python",
467
+ "name": "python",
468
+ "nbconvert_exporter": "python",
469
+ "pygments_lexer": "ipython3",
470
+ "version": "3.13.3"
471
+ }
472
+ },
473
+ "nbformat": 4,
474
+ "nbformat_minor": 5
475
+ }