Jaehun
/

PrismNLI-0.4B

@@ -1,13 +1,40 @@
 ---
 library_name: transformers
-tags: []
 ---
-# Model Card for Model ID
-<!-- Provide a quick summary of what the model is/does. -->
 ## Model Details
@@ -33,167 +60,30 @@ This is the model card of a 🤗 transformers model that has been pushed on the
 - **Paper [optional]:** [More Information Needed]
 - **Demo [optional]:** [More Information Needed]
-## Uses
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-### Direct Use
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
-[More Information Needed]
-### Downstream Use [optional]
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
-[More Information Needed]
-### Out-of-Scope Use
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
-[More Information Needed]
-## Bias, Risks, and Limitations
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
-[More Information Needed]
-### Recommendations
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
-## How to Get Started with the Model
-Use the code below to get started with the model.
-[More Information Needed]
-## Training Details
-### Training Data
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
-[More Information Needed]
-### Training Procedure
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-#### Preprocessing [optional]
-[More Information Needed]
-#### Training Hyperparameters
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
-#### Speeds, Sizes, Times [optional]
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-[More Information Needed]
-## Evaluation
-<!-- This section describes the evaluation protocols and provides the results. -->
-### Testing Data, Factors & Metrics
-#### Testing Data
-<!-- This should link to a Dataset Card if possible. -->
-[More Information Needed]
-#### Factors
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-[More Information Needed]
-#### Metrics
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-[More Information Needed]
-### Results
-[More Information Needed]
-#### Summary
-## Model Examination [optional]
-<!-- Relevant interpretability work for the model goes here -->
-[More Information Needed]
-## Environmental Impact
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
-## Technical Specifications [optional]
-### Model Architecture and Objective
-[More Information Needed]
-### Compute Infrastructure
-[More Information Needed]
-#### Hardware
-[More Information Needed]
-#### Software
-[More Information Needed]
-## Citation [optional]
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
-**BibTeX:**
-[More Information Needed]
-**APA:**
-[More Information Needed]
-## Glossary [optional]
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
-[More Information Needed]
-## More Information [optional]
-[More Information Needed]
-## Model Card Authors [optional]
-[More Information Needed]
-## Model Card Contact
-[More Information Needed]

 ---
 library_name: transformers
+tags:
+- NLI
+- entailment
+- zeroshot
+license: cc
+datasets:
+- Jaehun/PrismNLI
+language:
+- en
+base_model:
+- microsoft/deberta-v3-large
 ---
+# PrismNLI-0.4B
+[Paper](https://arxiv.org/abs/2505.20161) [Project Page](https://nvlabs.github.io/prismatic-synthesis/)
+PrismNLI-0.4B is a compact yet powerful expert model, purpose-built for natural language inference (NLI) and zero-shot classification.
+**Despite its small size, it delivers state-of-the-art performance on 8 NLI benchmarks**, making it a go-to solution for high-accuracy, low-latency NLI applications.
+PrismNLI-0.4B is fine-tuned from [deberta-v3-large](https://huggingface.co/microsoft/deberta-v3-large)
+on our high-quality dataset [PrismNLI](https://huggingface.co/datasets/Jaehun/PrismNLI), curated specifically to improve generalization of the trained model.
+For further details, please refer to our [paper](https://arxiv.org/abs/2505.20161).
+This version of the model goes beyond the original from our [paper](https://arxiv.org/abs/2505.20161), to produce a single, robust NLI model ready for off-the-shelf deployment.
+The enhancement includes:
+- Instead of starting from scratch, we start from [deberta-v3-large-zeroshot-v2.0](https://huggingface.co/MoritzLaurer/deberta-v3-large-zeroshot-v2.0), a checkpoint of
+deberta-v3-lage trained on diverse classification data.
+- Following prior works on entailment models, we reformulate the traditional 3-way NLI classification—`entailment`, `neutral`, and `contradiction`—into a binary setup:
+ `entailment` vs. `not-entailment`. This simplification enables the model to act as a **universal classifier**, by asking a single, intuitive question: *Is this hypothesis true, given the premise?*
+# Training Data
+The model has been fine-tuned on 515K NLI datapoints from [PrismNLI](https://huggingface.co/datasets/Jaehun/PrismNLI), a synthetic dataset to improve generalization of
+NLI models. The dataset has been generated by [Qwen2.5-72B-Instruct](https://huggingface.co/Qwen/Qwen2.5-72B-Instruct) via our algorithm, Prismatic Synthesis that scales synthetic data while improving the diversity of generated samples.
 ## Model Details
 - **Paper [optional]:** [More Information Needed]
 - **Demo [optional]:** [More Information Needed]
+## Citation
+If you find this model useful, please consider citing us!
+```bibtex
+@misc{prismatic-synthesis,
+      title={Prismatic Synthesis: Gradient-based Data Diversification Boosts Generalization in LLM Reasoning},
+      author={Jaehun Jung and Seungju Han and Ximing Lu and Skyler Hallinan and David Acuna and Shrimai Prabhumoye and Mostafa Patwary and Mohammad Shoeybi and Bryan Catanzaro and Yejin Choi},
+      year={2025},
+      eprint={2505.20161},
+      archivePrefix={arXiv},
+      primaryClass={cs.LG},
+      url={https://arxiv.org/abs/2505.20161},
+}
+```
+## License/Terms of Use:
+Governing Terms: This model is licensed under the Creative Commons Attribution 4.0 International License (CC BY 4.0)
+available at https://creativecommons.org/licenses/by/4.0/legalcode.
+This model is trained with synthetic data generated from Qwen2.5-72B-Instruct.
+If this dataset is used to create, train, fine tune, or otherwise improve an AI model,
+which is distributed or made available, such AI model may be subject to redistribution
+and use requirements in the [Qwen License Agreement](https://huggingface.co/Qwen/Qwen2.5-72B-Instruct/blob/main/LICENSE).