Update README.md

README.md

@@ -1,3 +1,154 @@
----
-license: mit
----
+---
+license: mit
+datasets:
+- nyu-mll/glue
+- google-research-datasets/paws-x
+- tasksource/pit
+- AlekseyKorshuk/quora-question-pairs
+language:
+- en
+metrics:
+- accuracy
+- f1
+base_model:
+- google-bert/bert-base-cased
+library_name: transformers
+---
+
+# Model Card for Fine-Tuned BERT for Paraphrase Detection
+
+### Model Description
+This is a fine-tuned version of **BERT-base** for **paraphrase detection**, trained on four benchmark datasets: **MRPC, QQP, PAWS-X, and PIT**. The model is designed for applications such as **duplicate content detection, question answering, and semantic similarity analysis**. It offers strong recall capabilities, making it effective in identifying paraphrases even in complex sentence structures.
+
+- **Developed by:** Viswadarshan R R  
+- **Model Type:** Transformer-based Sentence Pair Classifier  
+- **Language:** English  
+- **Finetuned from:** `bert-base-cased`
+
+### Model Sources
+
+- **Repository:** [Hugging Face Model Hub](https://huggingface.co/viswadarshan06/pd-bert/)  
+- **Research Paper:** _Comparative Insights into Modern Architectures for Paraphrase Detection_ (Accepted at ICCIDS 2025)  
+- **Demo:** (To be added upon deployment)  
+
+## Uses
+
+### Direct Use
+- Identifying **duplicate questions** in customer support and FAQs.  
+- Improving **semantic search** in retrieval-based systems.  
+- Enhancing **document deduplication** and text similarity applications.  
+
+### Downstream Use
+This model can be further fine-tuned on domain-specific paraphrase datasets for industries such as **healthcare, legal, and finance**.
+
+### Out-of-Scope Use
+- The model is **monolingual** and trained only on **English datasets**, requiring additional fine-tuning for multilingual tasks.  
+- May struggle with **idiomatic expressions** or complex figurative language.  
+
+## Bias, Risks, and Limitations
+
+### Known Limitations
+- **Higher recall but lower precision**: The model tends to over-identify paraphrases, leading to increased false positives.  
+- **Contextual ambiguity**: May misinterpret sentences that require deep contextual reasoning.  
+
+### Recommendations
+Users can mitigate the **false positive rate** by applying post-processing techniques or confidence threshold tuning.  
+
+## How to Get Started with the Model
+
+To use the model, install **transformers** and load the fine-tuned model as follows:
+
+```python
+from transformers import AutoTokenizer, AutoModelForSequenceClassification
+
+# Load the tokenizer and model
+model_path = "viswadarshan06/pd-bert"
+tokenizer = AutoTokenizer.from_pretrained(model_path)
+model = AutoModelForSequenceClassification.from_pretrained(model_path)
+
+# Encode sentence pairs
+inputs = tokenizer("The car is fast.", "The vehicle moves quickly.", return_tensors="pt", padding=True, truncation=True)
+
+# Get predictions
+outputs = model(**inputs)
+logits = outputs.logits
+predicted_class = logits.argmax().item()
+print("Paraphrase" if predicted_class == 1 else "Not a Paraphrase")
+```
+
+## Training Details  
+
+This model was trained using a combination of four datasets:
+
+- **MRPC**: News-based paraphrases.  
+- **QQP**: Duplicate question detection.  
+- **PAWS-X**: Adversarial paraphrases for robustness testing.  
+- **PIT**: Short-text paraphrase dataset.  
+
+### Training Procedure
+
+- **Tokenizer**: BERT Tokenizer  
+- **Batch Size**: 16  
+- **Optimizer**: AdamW  
+- **Loss Function**: Cross-entropy  
+
+#### Training Hyperparameters
+- **Learning Rate**: 2e-5  
+- **Sequence Length**:  
+  - MRPC: 256  
+  - QQP: 336  
+  - PIT: 64  
+  - PAWS-X: 256  
+
+#### Speeds, Sizes, Times  
+
+- **GPU Used**: NVIDIA A100  
+- **Total Training Time**: ~6 hours  
+- **Compute Units Used**: 80  
+
+### Testing Data, Factors & Metrics  
+
+#### Testing Data  
+The model was tested on combined test sets and evaluated using:
+- Accuracy  
+- Precision  
+- Recall  
+- F1-Score  
+- Runtime  
+
+### Results  
+
+## **BERT Model Evaluation Metrics**  
+| Model   | Dataset     | Accuracy (%) | Precision (%) | Recall (%) | F1-Score (%) | Runtime (sec) |  
+|---------|------------|-------------|--------------|------------|-------------|---------------|  
+| BERT | MRPC Validation | 88.24 | 88.37 | 95.34 | 91.72 | 1.41 |  
+| BERT | MRPC Test | 84.87 | 85.84 | 92.50 | 89.04 | 5.77 |  
+| BERT | QQP Validation | 87.92 | 81.44 | 86.86 | 84.06 | 43.24 |  
+| BERT | QQP Test | 88.14 | 82.49 | 86.56 | 84.47 | 43.51 |  
+| BERT | PAWS-X Validation | 91.90 | 87.57 | 94.67 | 90.98 | 6.73 |  
+| BERT | PAWS-X Test | 92.60 | 88.69 | 95.92 | 92.16 | 6.82 |  
+| BERT | PIT Validation | 77.38 | 72.41 | 58.57 | 64.76 | 4.34 |  
+| BERT | PIT Test | 86.16 | 64.11 | 76.57 | 69.79 | 0.98 |  
+
+### Summary  
+This **BERT-based Paraphrase Detection Model** demonstrates strong **recall capabilities**, making it highly effective at **identifying paraphrases** across varied linguistic structures. While it tends to overpredict paraphrases, it remains a **strong baseline** for **semantic similarity tasks** and can be fine-tuned further for **domain-specific applications**.  
+
+### **Citation**  
+
+If you use this model, please cite:  
+
+```bibtex
+@inproceedings{viswadarshan2025paraphrase,
+   title={Comparative Insights into Modern Architectures for Paraphrase Detection},
+   author={Viswadarshan R R, Viswaa Selvam S, Felcia Lilian J, Mahalakshmi S},
+   booktitle={International Conference on Computational Intelligence, Data Science, and Security (ICCIDS)},
+   year={2025},
+   publisher={IFIP AICT Series by Springer}
+}
+```  
+
+## Model Card Contact  
+
+📧 Email: [email protected]  
+
+🔗 GitHub: [Viswadarshan R R](https://github.com/viswadarshan-024)