update readme

README.md

@@ -19,7 +19,9 @@ Please see the linked model cards below for more details.
   - [clic/clusters/v1.9.0](clic/clusters/v1.9.0/README.md)
   - [clic/clusters/v2.0.0](clic/clusters/v2.0.0/README.md)
   - [clic/clusters/v2.1.0](clic/clusters/v2.1.0/pyg-clic_20241106_104416_929167/README.md)
+  - [clic/clusters/v2.2.0](clic/clusters/v2.2.0/pyg-clic_20250106_193536_269746/README.md)
   - [cms/v2.1.0](cms/v2.1.0/pyg-cms_20241101_090645_682892/README.md)
+  - [cms/v2.2.0](cms/v2.2.0/pyg-cms_20241212_101648_120237/README.md)
 
 
 ## Papers

cms/v2.2.0/pyg-cms_20241212_101648_120237/README.md

@@ -0,0 +1,189 @@
+# Model Card for mlpf-cms-v2.2.0
+
+This model reconstructs particles in a detector, based on the tracks and calorimeter clusters recorded by the detector.
+The primary difference with respect to v2.2.0 is the inclusion of the sqrt(pt) weight term in the pT and energy regression loss.
+Additionally, the model has been scaled down to ~5M parameters (previously ~100M) for more efficient inference.
+
+## Model Details
+
+The performance is measured with respect to generator-level jets and MET computed from Pythia particles, i.e. the truth-level jets and MET.
+
+<details>
+  <summary>Jet performance</summary>
+  
+  <img src="plots_checkpoint-05-3.498507/cms_pf_qcd/jet_response_iqr_over_med_pt.png" alt="ttbar jet resolution" width="300"/>
+  <img src="plots_checkpoint-05-3.498507/cms_pf_ttbar/jet_response_iqr_over_med_pt.png" alt="qq jet resolution" width="300"/>
+  <img src="plots_checkpoint-05-3.498507/cms_pf_ztt/jet_response_iqr_over_med_pt.png" alt="ttbar jet resolution" width="300"/>
+
+</details>
+
+<details>
+  <summary>MET performance</summary>
+  
+  <img src="plots_checkpoint-05-3.498507/cms_pf_qcd/met_response_iqr_over_med.png" alt="ttbar MET resolution" width="300"/>
+  <img src="plots_checkpoint-05-3.498507/cms_pf_ttbar/met_response_iqr_over_med.png" alt="qq MET resolution" width="300"/>
+  <img src="plots_checkpoint-05-3.498507/cms_pf_ztt/met_response_iqr_over_med.png" alt="ttbar MET resolution" width="300"/>
+
+</details>
+
+### Model Description
+
+- **Developed by:** CMS MLPF Team 
+- **Model type:** transformer
+- **License:** Apache License
+
+### Model Sources
+
+- **Repository:** https://github.com/jpata/particleflow/releases/tag/v2.2.0
+
+## Uses
+### Direct Use
+
+This model may be used to study the physics and computational performance on ML-based reconstruction in simulation within the CMS collaboration.
+
+### Out-of-Scope Use
+
+This model is not intended for physics measurements on real data or for use outside the CMS collaboration.
+
+## Bias, Risks, and Limitations
+
+The model has only been trained on simulation data and has not been validated against real data.
+The model has not been peer reviewed or published in a peer-reviewed journal.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+```
+#get the code
+git clone https://github.com/jpata/particleflow
+cd particleflow
+git checkout v2.2.0
+
+#get the models
+git clone https://huggingface.co/jpata/particleflow models
+```
+
+## Training Details
+Trained on 1x A100 for 5 epochs over ~2 days.
+
+### Training Data
+The following datasets were used:
+```
+18G     /scratch/persistent/joosep/tensorflow_datasets/cms_pf_qcd/1/2.5.0
+18G     /scratch/persistent/joosep/tensorflow_datasets/cms_pf_qcd/2/2.5.0
+18G     /scratch/persistent/joosep/tensorflow_datasets/cms_pf_qcd/3/2.5.0
+18G     /scratch/persistent/joosep/tensorflow_datasets/cms_pf_qcd/4/2.5.0
+18G     /scratch/persistent/joosep/tensorflow_datasets/cms_pf_qcd/5/2.5.0
+18G     /scratch/persistent/joosep/tensorflow_datasets/cms_pf_qcd/6/2.5.0
+18G     /scratch/persistent/joosep/tensorflow_datasets/cms_pf_qcd/7/2.5.0
+18G     /scratch/persistent/joosep/tensorflow_datasets/cms_pf_qcd/8/2.5.0
+18G     /scratch/persistent/joosep/tensorflow_datasets/cms_pf_qcd/9/2.5.0
+18G     /scratch/persistent/joosep/tensorflow_datasets/cms_pf_qcd/10/2.5.0
+8.4G    /scratch/persistent/joosep/tensorflow_datasets/cms_pf_qcd_nopu/1/2.5.0
+8.4G    /scratch/persistent/joosep/tensorflow_datasets/cms_pf_qcd_nopu/2/2.5.0
+8.4G    /scratch/persistent/joosep/tensorflow_datasets/cms_pf_qcd_nopu/3/2.5.0
+8.4G    /scratch/persistent/joosep/tensorflow_datasets/cms_pf_qcd_nopu/4/2.5.0
+8.4G    /scratch/persistent/joosep/tensorflow_datasets/cms_pf_qcd_nopu/5/2.5.0
+8.4G    /scratch/persistent/joosep/tensorflow_datasets/cms_pf_qcd_nopu/6/2.5.0
+8.4G    /scratch/persistent/joosep/tensorflow_datasets/cms_pf_qcd_nopu/7/2.5.0
+8.4G    /scratch/persistent/joosep/tensorflow_datasets/cms_pf_qcd_nopu/8/2.5.0
+8.4G    /scratch/persistent/joosep/tensorflow_datasets/cms_pf_qcd_nopu/9/2.5.0
+8.4G    /scratch/persistent/joosep/tensorflow_datasets/cms_pf_qcd_nopu/10/2.5.0
+18G     /scratch/persistent/joosep/tensorflow_datasets/cms_pf_ttbar/1/2.5.0
+18G     /scratch/persistent/joosep/tensorflow_datasets/cms_pf_ttbar/2/2.5.0
+18G     /scratch/persistent/joosep/tensorflow_datasets/cms_pf_ttbar/3/2.5.0
+18G     /scratch/persistent/joosep/tensorflow_datasets/cms_pf_ttbar/4/2.5.0
+18G     /scratch/persistent/joosep/tensorflow_datasets/cms_pf_ttbar/5/2.5.0
+18G     /scratch/persistent/joosep/tensorflow_datasets/cms_pf_ttbar/6/2.5.0
+18G     /scratch/persistent/joosep/tensorflow_datasets/cms_pf_ttbar/7/2.5.0
+18G     /scratch/persistent/joosep/tensorflow_datasets/cms_pf_ttbar/8/2.5.0
+18G     /scratch/persistent/joosep/tensorflow_datasets/cms_pf_ttbar/9/2.5.0
+19G     /scratch/persistent/joosep/tensorflow_datasets/cms_pf_ttbar/10/2.5.0
+8.6G    /scratch/persistent/joosep/tensorflow_datasets/cms_pf_ttbar_nopu/1/2.5.0
+8.6G    /scratch/persistent/joosep/tensorflow_datasets/cms_pf_ttbar_nopu/2/2.5.0
+8.6G    /scratch/persistent/joosep/tensorflow_datasets/cms_pf_ttbar_nopu/3/2.5.0
+8.6G    /scratch/persistent/joosep/tensorflow_datasets/cms_pf_ttbar_nopu/4/2.5.0
+8.6G    /scratch/persistent/joosep/tensorflow_datasets/cms_pf_ttbar_nopu/5/2.5.0
+8.6G    /scratch/persistent/joosep/tensorflow_datasets/cms_pf_ttbar_nopu/6/2.5.0
+8.6G    /scratch/persistent/joosep/tensorflow_datasets/cms_pf_ttbar_nopu/7/2.5.0
+8.6G    /scratch/persistent/joosep/tensorflow_datasets/cms_pf_ttbar_nopu/8/2.5.0
+8.6G    /scratch/persistent/joosep/tensorflow_datasets/cms_pf_ttbar_nopu/9/2.5.0
+8.6G    /scratch/persistent/joosep/tensorflow_datasets/cms_pf_ttbar_nopu/10/2.5.0
+18G     /scratch/persistent/joosep/tensorflow_datasets/cms_pf_ztt/1/2.5.0
+18G     /scratch/persistent/joosep/tensorflow_datasets/cms_pf_ztt/2/2.5.0
+18G     /scratch/persistent/joosep/tensorflow_datasets/cms_pf_ztt/3/2.5.0
+18G     /scratch/persistent/joosep/tensorflow_datasets/cms_pf_ztt/4/2.5.0
+18G     /scratch/persistent/joosep/tensorflow_datasets/cms_pf_ztt/5/2.5.0
+18G     /scratch/persistent/joosep/tensorflow_datasets/cms_pf_ztt/6/2.5.0
+18G     /scratch/persistent/joosep/tensorflow_datasets/cms_pf_ztt/7/2.5.0
+18G     /scratch/persistent/joosep/tensorflow_datasets/cms_pf_ztt/8/2.5.0
+18G     /scratch/persistent/joosep/tensorflow_datasets/cms_pf_ztt/9/2.5.0
+18G     /scratch/persistent/joosep/tensorflow_datasets/cms_pf_ztt/10/2.5.0
+5.8G    /scratch/persistent/joosep/tensorflow_datasets/cms_pf_ztt_nopu/1/2.5.0
+5.8G    /scratch/persistent/joosep/tensorflow_datasets/cms_pf_ztt_nopu/2/2.5.0
+5.7G    /scratch/persistent/joosep/tensorflow_datasets/cms_pf_ztt_nopu/3/2.5.0
+5.8G    /scratch/persistent/joosep/tensorflow_datasets/cms_pf_ztt_nopu/4/2.5.0
+5.7G    /scratch/persistent/joosep/tensorflow_datasets/cms_pf_ztt_nopu/5/2.5.0
+5.7G    /scratch/persistent/joosep/tensorflow_datasets/cms_pf_ztt_nopu/6/2.5.0
+5.7G    /scratch/persistent/joosep/tensorflow_datasets/cms_pf_ztt_nopu/7/2.5.0
+5.7G    /scratch/persistent/joosep/tensorflow_datasets/cms_pf_ztt_nopu/8/2.5.0
+5.7G    /scratch/persistent/joosep/tensorflow_datasets/cms_pf_ztt_nopu/9/2.5.0
+5.8G    /scratch/persistent/joosep/tensorflow_datasets/cms_pf_ztt_nopu/10/2.5.0
+```
+
+## Training Procedure 
+
+```bash
+#!/bin/bash
+#SBATCH --partition gpu
+#SBATCH --gres gpu:a100:1
+#SBATCH --mem-per-gpu 300G
+#SBATCH -o logs/slurm-%x-%j-%N.out
+
+IMG=/home/software/singularity/pytorch.simg:2024-12-03
+cd ~/particleflow
+
+ulimit -n 100000
+singularity exec -B /scratch/persistent --nv \
+    --env PYTHONPATH=`pwd` \
+    --env KERAS_BACKEND=torch \
+    $IMG python3 mlpf/pipeline.py --gpus 1 \
+    --data-dir /scratch/persistent/joosep/tensorflow_datasets --config parameters/pytorch/pyg-cms.yaml \
+    --train --conv-type attention \
+    --gpu-batch-multiplier 5 --checkpoint-freq 1 --num-workers 8 --prefetch-factor 50 --comet --ntest 1000 --test-datasets cms_pf_qcd_nopu
+```
+
+## Evaluation
+```bash
+#!/bin/bash
+#SBATCH --partition gpu
+#SBATCH --gres gpu:mig:1
+#SBATCH --mem-per-gpu 100G
+#SBATCH -o logs/slurm-%x-%j-%N.out
+
+IMG=/home/software/singularity/pytorch.simg:2024-08-18
+cd ~/particleflow
+
+WEIGHTS=experiments/pyg-cms_20241212_101648_120237/checkpoints/checkpoint-05-3.498507.pth
+DATASET=$1
+env
+singularity exec -B /local -B /scratch/persistent --nv \
+     --env PYTHONPATH=`pwd` \
+     --env KERAS_BACKEND=torch \
+     $IMG python mlpf/pipeline.py --gpus 1 \
+     --data-dir /scratch/persistent/joosep/tensorflow_datasets --config parameters/pytorch/pyg-cms.yaml \
+     --test --make-plots --gpu-batch-multiplier 2 --load $WEIGHTS --ntest 10000 --dtype bfloat16 --num-workers 1 --prefetch-factor 10 --test-datasets $DATASET
+```
+## Citation
+
+## Glossary
+
+- PF: particle flow reconstruction
+- MLPF: machine learning for particle flow
+- CMS: Compact Muon Solenoid
+
+## Model Card Contact
+
+Joosep Pata, [email protected]
+