Upload 9 files

README.md

@@ -1,173 +1,3 @@
----
-license: mit
-datasets:
-- hrishivish23/MPM-Verse-MaterialSim-Small
-- hrishivish23/MPM-Verse-MaterialSim-Large
-language:
-- en
-metrics:
-- accuracy
-pipeline_tag: graph-ml
-tags:
-- physics
-- scientific-ml
-- lagrangian-dynamics
-- neural-operator
-- neural-operator-transformer
-- graph-neural-networks
-- graph-transformer
-- sequence-to-sequence
-- autoregressive
-- temporal-dynamics
----
-
-# 📌 PhysicsEngine: Reduced-Order Neural Operators for Lagrangian Dynamics
-
-**By [Hrishikesh Viswanath](https://huggingface.co/hrishivish23), Yue Chang, Julius Berner, Peter Yichen Chen, Aniket Bera**  
-
-![Physics Simulation](https://hrishikeshvish.github.io/projects/giorom_data/giorom_pipeline_plasticine.png)
-
----
-
-## 📝 Model Overview  
-**GIOROM** is a **Reduced-Order Neural Operator Transformer** designed for **Lagrangian dynamics simulations on highly sparse graphs**. The model enables hybrid **Eulerian-Lagrangian learning** by:  
-
-- **Projecting Lagrangian inputs onto uniform grids** with a **Graph-Interaction-Operator**.  
-- **Predicting acceleration from sparse velocity inputs** using past time windows with a **Neural Operator Transformer**.  
-- **Learning physics from sparse inputs (n ≪ N)** while allowing reconstruction at arbitrarily dense resolutions via an **Integral Transform Model**.
-- **Dataset Compatibility**: This model is compatible with [`MPM-Verse-MaterialSim-Small/Sand3DNCLAWSmall`](https://huggingface.co/datasets/hrishivish23/MPM-Verse-MaterialSim-Small/tree/main/Sand3DNCLAWSmall), 
-
-⚠ **Note:** While the model can infer using an integral transform, **this repository only provides weights for the time-stepper model that predicts acceleration.**
-
----
-
-## 📊 Available Model Variants  
-Each variant corresponds to a specific dataset, showcasing the reduction in particle count (n: reduced-order, N: full-order).  
-
-| Model Name                     | n (Reduced) | N (Full) |
-|---------------------------------|------------|---------|
-| `giorom-3d-t-sand3d-long`       | 3.0K       | 32K     |
-| `giorom-3d-t-water3d`           | 1.7K       | 55K     |
-| `giorom-3d-t-elasticity`        | 2.6K       | 78K     |
-| `giorom-3d-t-plasticine`        | 1.1K       | 5K      |
-| `giorom-2d-t-water`             | 0.12K      | 1K      |
-| `giorom-2d-t-sand`              | 0.3K       | 2K      |
-| `giorom-2d-t-jelly`             | 0.2K       | 1.9K    |
-| `giorom-2d-t-multimaterial`     | 0.25K      | 2K      |
-
----
-
-## 💡 How It Works  
-
-### 🔹 Input Representation  
-The model predicts **acceleration** from past velocity inputs:  
-
-- **Input Shape:** `[n, D, W]`  
-  - `n`: Number of particles (reduced-order, n ≪ N)  
-  - `D`: Dimension (2D or 3D)  
-  - `W`: Time window (past velocity states)  
-
-- **Projected to a uniform latent space** of size `[c^D, D]` where:  
-  - `c ∈ {8, 16, 32}`  
-  - `n - δn ≤ c^D ≤ n + δn`  
-
-This allows the model to generalize physics across different resolutions and discretizations.  
-
-### 🔹 Prediction & Reconstruction  
-- The model **learns physical dynamics** on the sparse input representation.  
-- The **integral transform model** reconstructs dense outputs at arbitrary resolutions (not included in this repo).  
-- Enables **highly efficient, scalable simulations** without requiring full-resolution training.
-
----
-
-## 🚀 Usage Guide  
-### 1️⃣ Install Dependencies
-```bash
-pip install transformers huggingface_hub torch
-```
-```
-git clone https://github.com/HrishikeshVish/GIOROM/
-cd GIOROM
-```
-
-
-### 2️⃣ Load a Model
-
-
-
-```python
-from models.giorom3d_T import PhysicsEngine
-from models.config import TimeStepperConfig
-
-time_stepper_config = TimeStepperConfig()
-
-simulator = PhysicsEngine(time_stepper_config)
-repo_id = "hrishivish23/giorom-3d-t-sand3d"
-time_stepper_config = time_stepper_config.from_pretrained(repo_id)
-simulator = simulator.from_pretrained(repo_id, config=time_stepper_config)
-```
-
-### 3️⃣ Run Inference
-```python
-import torch
-
-```
-
----
-
-## 📂 Model Weights and Checkpoints  
-| Model Name                     | Model ID |
-|---------------------------------|-------------|
-| `giorom-3d-t-sand3d-long`            | [`hrishivish23/giorom-3d-t-sand3d-long`](https://huggingface.co/hrishivish23/giorom-3d-t-sand3d-long) |
-| `giorom-3d-t-water3d`           | [`hrishivish23/giorom-3d-t-water3d`](https://huggingface.co/hrishivish23/giorom-3d-t-water3d) |
-
-
----
-
-## 📚 Training Details  
-### 🔧 Hyperparameters  
-- **Graph Interaction Operator** layers: **4**
-- **Transformer Heads**: **4**
-- **Embedding Dimension:** **128**  
-- **Latent Grid Sizes:** `{8×8, 16×16, 32×32}`  
-- **Learning Rate:** `1e-4`  
-- **Optimizer:** `Adamax`  
-- **Loss Function:** `MSE + Physics Regularization (Loss computed on Euler integrated outputs)`  
-- **Training Steps:** `1M+ steps`  
-
-### 🖥️ Hardware  
-- **Trained on:** NVIDIA RTX 3050
-- **Batch Size:** `2`  
-
----
-
-## 📜 Citation  
-If you use this model, please cite:
-```bibtex
-@article{viswanath2024reduced,
-  title={Reduced-Order Neural Operators: Learning Lagrangian Dynamics on Highly Sparse Graphs},
-  author={Viswanath, Hrishikesh and Chang, Yue and Berner, Julius and Chen, Peter Yichen and Bera, Aniket},
-  journal={arXiv preprint arXiv:2407.03925},
-  year={2024}
-}
-```
-
----
-
-## 💬 Contact  
-For questions or collaborations:  
-- 🧑‍💻 Author: [Hrishikesh Viswanath](https://hrishikeshvish.github.io)  
-- 📧 Email: [email protected] 
-- 💬 Hugging Face Discussion: [Model Page](https://huggingface.co/hrishivish23/giorom-3d-t-sand3d-long/discussions)  
-
----
-
-## 🔗 Related Work  
-- **Neural Operators for PDEs**: Fourier Neural Operators, Graph Neural Operators  
-- **Lagrangian Methods**: Material Point Methods, SPH, NCLAW, CROM, LiCROM
-- **Physics-Based ML**: PINNs, GNS, MeshGraphNet
-
----
-
-### 🔹 Summary  
-This model is ideal for **fast and scalable physics simulations** where full-resolution computation is infeasible. The reduced-order approach allows **efficient learning on sparse inputs**, with the ability to **reconstruct dense outputs using an integral transform model (not included in this repo).**
+---
+license: mit
+---

config.json

@@ -1,9 +1,10 @@
 {
+  "_name_or_path": "/scratch/gilbreth/hviswan/GIOROM/saved_models/checkpoint-220000",
   "architectures": [
     "PhysicsEngine"
   ],
   "dim": 3,
-  "gno_radius": 0.165,
+  "gno_radius": 0.125,
   "heads": 3,
   "hidden_size": 128,
   "in_gno_mlp_hidden_layers": [

gitattributes

@@ -0,0 +1,35 @@
+*.7z filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.bz2 filter=lfs diff=lfs merge=lfs -text
+*.ckpt filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.gz filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.mlmodel filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.npy filter=lfs diff=lfs merge=lfs -text
+*.npz filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pickle filter=lfs diff=lfs merge=lfs -text
+*.pkl filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.rar filter=lfs diff=lfs merge=lfs -text
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text
+*.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tar filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tgz filter=lfs diff=lfs merge=lfs -text
+*.wasm filter=lfs diff=lfs merge=lfs -text
+*.xz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.zst filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text

model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:cbae34bf29e1d2db971aa80ea138ae658c5725a051106ee6157754d8917b3851
+oid sha256:a5988486eae82102d817268d65bf13e2c75d158ced00daf4994b1d476e602a52
 size 9568276

optimizer.pt

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:59ec1eb13b63796ad34b3fef55cef5056298b124dd60cca8d2e20ae55308cf75
-size 19246970
+oid sha256:1eb9785441eb6839ef6303b6d399c399b97af4f9cab8dabb543ae2a7969a29e1
+size 19244538

rng_state.pth

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:4083aa1322a231a785325c26497b14a004df92984d3c40f4ec4ece66ac2e9b09
+oid sha256:389769e79f51e0631cb8227739225e60ff3aa16e83c5a4096cd66c07328b42af
 size 14244

scheduler.pt

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:60e429bed308f3741637c88d0bd73b051342308591193d374d219af1d4ac4d5d
+oid sha256:56f191a7c9a54597391a1acc630adbf0d4ab538406bf0edc4f28d4e9cfd8f5f8
 size 1000

trainer_state.json

@@ -1,33 +1,33 @@
 {
   "best_metric": null,
   "best_model_checkpoint": null,
-  "epoch": 15.923566878980893,
+  "epoch": 11.67728237791932,
   "eval_steps": 500,
-  "global_step": 300000,
+  "global_step": 220000,
   "is_hyper_param_search": false,
   "is_local_process_zero": true,
   "is_world_process_zero": true,
   "log_history": [
     {
-      "avg_loss": 0.17167335909317713,
-      "grad_norm": 0.0,
-      "learning_rate": 7.585775750096955e-05,
-      "loss": 0.1428,
-      "lr": 7.585779243476592e-05,
-      "step": 299999
+      "avg_loss": 0.29662682761985026,
+      "grad_norm": 32.271949768066406,
+      "learning_rate": 6.674212038333604e-05,
+      "loss": 0.06960329413414001,
+      "lr": 6.668070762644054e-05,
+      "step": 219999
     }
   ],
   "logging_steps": 1500,
   "max_steps": 37680000,
   "num_input_tokens_seen": 0,
   "num_train_epochs": 2000,
-  "save_steps": 100000,
+  "save_steps": 110000,
   "stateful_callbacks": {
     "TrainerControl": {
       "args": {
         "should_epoch_stop": false,
         "should_evaluate": false,
-        "should_log": true,
+        "should_log": false,
         "should_save": true,
         "should_training_stop": false
       },

training_args.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:1d6b918707d435205378f09b443e0d4952a640bd1a9256371c045e60abfaaf21
+oid sha256:3b2d5fb1239f85339ec2dac852116c36477743f93287a6bbff77e3c3c2e6e1d0
 size 5240