EleutherAI/deep-ignorance-unfiltered

Text Generation • 7B • Updated 16 days ago • 154

Note Fully Trained — Unfiltered Baseline Model - Pretraining Filtering: None - Annealing Filtering: None - Results Location: Main Paper

EleutherAI/deep-ignorance-e2e-strong-filter

Text Generation • 7B • Updated 26 days ago • 48

Note Fully Trained - Pretraining Filtering: Strong Filter - Annealing Filtering: Strong Filter - Results Location: Main Paper (Strong Filter)

EleutherAI/deep-ignorance-strong-filter-pt-weak-filter-anneal

Text Generation • 7B • Updated 16 days ago • 83

Note Fully Trained - Pretraining Filtering: Strong Filter - Annealing Filtering: Weak Filter - Results Location: Main Paper (Weak Filter)

EleutherAI/deep-ignorance-e2e-weak-filter

Text Generation • 7B • Updated 16 days ago • 67

Note Fully Trained - Pretraining Filtering: Weak Filter - Annealing Filtering: Weak Filter - Results Location: Appendix

EleutherAI/deep-ignorance-weak-filter-pt-strong-filter-anneal

Text Generation • 7B • Updated 15 days ago • 74

Note Fully Trained - Pretraining Filtering: Weak Filter - Annealing Filtering: Strong Filter

EleutherAI/deep-ignorance-pretraining-stage-unfiltered

Text Generation • 7B • Updated 8 days ago • 422

Note Pretrained model that has not undergone annealing or any data filtering. - Pretraining Filtering: None - Results Location: Not Included

EleutherAI/deep-ignorance-pretraining-stage-strong-filter

Text Generation • 7B • Updated 8 days ago • 14

Note Pretrained model that has not undergone annealing. - Pretraining Filtering: Strong Filter - Results Location: Not Included

EleutherAI/deep-ignorance-pretraining-stage-weak-filter

Text Generation • 7B • Updated 8 days ago • 123

Note Pretrained model which has not undergone annealing. - Pretraining Filtering: Weak Filter - Results Location: Not Included

EleutherAI/deep-ignorance-pretraining-stage-extra-weak-filter

Updated about 23 hours ago • 8

Note Pretrained model that has not undergone annealing. - Pretraining Filtering: Extra Weak Filter - Results Location: Not Included

EleutherAI/deep-ignorance-e2e-strong-filter-cb-lat

Text Generation • 7B • Updated 4 days ago • 14

Note Fully Trained with Circuit Breaking & Latent Adversarial Training - Pretraining Filtering: Strong Filter - Annealing Filtering: Strong Filter - Post-training: Circuit Breaking + Latent Adversarial Training - Results Location: Main Paper (Strong Filter + CB + LAT)

EleutherAI/deep-ignorance-strong-filter-pt-weak-filter-anneal-cb-lat

Text Generation • 7B • Updated 4 days ago • 9

Note Fully Trained with Circuit Breaking & Latent Adversarial Training - Pretraining Filtering: Strong Filter - Annealing Filtering: Weak Filter - Post-training: Circuit Breaking + Latent Adversarial Training - Results Location: Main Paper (Weak Filter + CB + LAT)

EleutherAI/deep-ignorance-unfiltered-cb

Text Generation • 7B • Updated 4 days ago • 10

Note Fully Trained — Unfiltered Baseline Model with Circuit Breaking - Pretraining Filtering: None - Annealing Filtering: None - Post-training: Circuit Breaking - Results Location: Main Paper (CB)

EleutherAI/deep-ignorance-unfiltered-cb-lat

Text Generation • 7B • Updated 4 days ago • 10

Note Fully Trained — Unfiltered Baseline Model with Circuit Breaking & Latent Adversarial Training - Pretraining Filtering: None - Annealing Filtering: None - Post-training: Circuit Breaking + Latent Adversarial Training - Results Location: Main Paper (CB + LAT)

EleutherAI/deep-ignorance-e2e-strong-filter-cb

Text Generation • 7B • Updated 4 days ago • 10

Note Fully Trained with Circuit Breaking - Pretraining Filtering: Strong Filter - Annealing Filtering: Strong Filter - Post-training: Circuit Breaking - Results Location: Main Paper (Strong Filter + CB)

EleutherAI/deep-ignorance-strong-filter-pt-weak-filter-anneal-cb

Text Generation • 7B • Updated 4 days ago • 9

Note Fully Trained with Circuit Breaking - Pretraining Filtering: Strong Filter - Annealing Filtering: Weak Filter - Post-training: Circuit Breaking - Results Location: Main Paper (Weak Filter + CB)

EleutherAI/deep-ignorance-e2e-extra-weak-filter

Text Generation • 7B • Updated 3 days ago • 93

Note Fully Trained - Pretraining Filtering: Extra Weak Filter - Annealing Filtering: Extra Weak Filter - Results Location: Not Included

EleutherAI/deep-ignorance-e2e-strong-filter-weak-knowledge-corrupted

Text Generation • 7B • Updated Jun 3 • 72

Note Fully Trained - Pretraining Filtering: Strong Filter - Annealing Filtering: Strong Filter - Post-training: Weak Knowledge Corruption via Synthetic Document Fine-Tuning - Results Location: Main Paper & Appendix

EleutherAI/deep-ignorance-e2e-strong-filter-strong-knowledge-corrupted

Text Generation • 7B • Updated 2 days ago • 102

Note Fully Trained - Pretraining Filtering: Strong Filter - Annealing Filtering: Strong Filter - Post-training: Strong Knowledge Corruption via Synthetic Document Fine-Tuning - Results Location: Main Paper & Appendix

Note All prompts from WMDP-Bio that can be evaluated using a cloze-style prompt.

Note WMDP-Bio, where data is broken down by topic category and whether it contains likely shortcuts.

Note General knowledge multiple-choice and cloze-style prompts that are used to ensure that models are familiar with the MCQA test benchmarks, like WMDP and MMLU.

Note The original annealing dataset for training the LLMs. This dataset is not filtered.

Note The original pretraining dataset for training the LLMs. This dataset is not filtered.