Experiments with new architecture that enables latent space reasoning
Aman Gupta PRO
amang1802
AI & ML interests
None yet
Organizations
Small model pretraining experiments
Synthetic Data rewrite (model checkpoints)
Models trained with synthetic data generated using various synthetic rewrite methods
WildeWeb Research
Soft Skills Data curation
Smol-Math
Smol-Math pretraining and post-training projects
PPO experiments
Using PPO with simpler reward functions
Synthetic Data rewrite research (training and eval datasets)
Researching methods for synthetic rewrites for CPT datasets and evaluating them in their ability to improve knowledge memorization
-
amang1802/synthetic_data_unconditioned_L3.1_70B
Viewer • Updated • 10.2k • 6 -
amang1802/synthetic_data_unconditioned_L3.1_70B_deduped
Viewer • Updated • 10.2k • 9 -
amang1802/synthetic_data_unconditioned_L3.1_405B_Instruct
Viewer • Updated • 10.2k • 8 -
amang1802/synthetic_data_unconditioned_L3.1_405B_Instruct_deduped
Viewer • Updated • 10.2k • 13
ThinkTransformer experiments
Experiments with new architecture that enables latent space reasoning
Smol-Math
Smol-Math pretraining and post-training projects
Small model pretraining experiments
PPO experiments
Using PPO with simpler reward functions
Synthetic Data rewrite (model checkpoints)
Models trained with synthetic data generated using various synthetic rewrite methods
Synthetic Data rewrite research (training and eval datasets)
Researching methods for synthetic rewrites for CPT datasets and evaluating them in their ability to improve knowledge memorization
-
amang1802/synthetic_data_unconditioned_L3.1_70B
Viewer • Updated • 10.2k • 6 -
amang1802/synthetic_data_unconditioned_L3.1_70B_deduped
Viewer • Updated • 10.2k • 9 -
amang1802/synthetic_data_unconditioned_L3.1_405B_Instruct
Viewer • Updated • 10.2k • 8 -
amang1802/synthetic_data_unconditioned_L3.1_405B_Instruct_deduped
Viewer • Updated • 10.2k • 13
WildeWeb Research
Soft Skills Data curation