README.md

---
language:
- ar
tags:
- sentence-transformers
- sentence-similarity
- feature-extraction
- matryoshka
- arabic
- natural-language-inference
- bert
- nli
- arabert
datasets:
- Omartificial-Intelligence-Space/Arabic-NLi-Pair-Class
metrics:
- cosine_accuracy
- cosine_f1
- accuracy
- f1
library_name: sentence-transformers
pipeline_tag: sentence-similarity
base_model: aubmindlab/bert-base-arabertv02
license: apache-2.0
model-index:
- name: Arabic BERT NLI Matryoshka
  results:
  - task:
      type: natural-language-inference
      name: Natural Language Inference
    dataset:
      type: Omartificial-Intelligence-Space/Arabic-NLi-Pair-Class
      name: Arabic NLI Pair Classification
    metrics:
    - type: accuracy
      value: 0.8125
      name: Best Accuracy (128 dim)
    - type: f1
      value: 0.8142
      name: Best F1 (256 dim)
---

# DIMI Embedding model

<div align="center">

<img src="https://cdn-uploads.huggingface.co/production/uploads/65fb3ac20cfe262da2bb0fcc/uOuEn0LNhSVEBbOLwfFUu.jpeg" width="300"/>

*State-of-the-art Arabic Sentence Embeddings*

</div>


## Model Description

This model is a **Matryoshka representation learning** version of AraBERT specifically fine-tuned for Arabic Natural Language Inference (NLI) tasks. It generates embeddings that can be truncated to different dimensions (768, 512, 256, 128, 64) while maintaining strong performance across all sizes.

The model is based on `aubmindlab/bert-base-arabertv02` and trained using the Matryoshka Representation Learning approach, which allows for flexible embedding dimensions without retraining.

## Key Features

- 🔄 **Flexible Dimensions**: Single model supports embeddings of size 768, 512, 256, 128, and 64
- 🚀 **High Performance**: Consistently outperforms base model across all dimensions
- 📊 **Arabic NLI Optimized**: Specifically trained on Arabic Natural Language Inference tasks
- ⚡ **Efficient**: Smaller dimensions offer faster inference with minimal performance loss
- 🎯 **Binary Classification**: Optimized for entailment vs contradiction classification

## Performance Results

Our model shows significant improvements over the base AraBERT model across all embedding dimensions:

| Dimension | Matryoshka Accuracy | Base Accuracy | Matryoshka F1 | Base F1 | Improvement |
|-----------|---------------------|---------------|---------------|---------|-------------|
| 768       | 80.3%              | 56.8%         | 81.15%        | 41.94%  | +39.21%     |
| 512       | 80.6%              | 56.9%         | 81.36%        | 44.32%  | +37.05%     |
| 256       | 80.95%             | 55.65%        | 81.42%        | 38.7%   | +42.72%     |
| 128       | 81.25%             | 56.7%         | 81.37%        | 40.6%   | +40.77%     |
| 64        | 81.0%              | 55.8%         | 80.51%        | 37.92%  | +42.59%     |

## Quick Start

### Installation

```bash
pip install sentence-transformers torch
```

### Basic Usage

```python
from sentence_transformers import SentenceTransformer

# Load the model
model = SentenceTransformer('AhmedZaky1/arabic-bert-nli-matryoshka')

# Example sentences
sentences = [
    "الطقس جميل اليوم",
    "إنه يوم مشمس وجميل", 
    "أحب قراءة الكتب"
]

# Generate embeddings (default: full 768 dimensions)
embeddings = model.encode(sentences)
print(f"Full embeddings shape: {embeddings.shape}")

# Use different dimensions by truncating
embeddings_256 = embeddings[:, :256]  # Use first 256 dimensions
embeddings_128 = embeddings[:, :128]  # Use first 128 dimensions
embeddings_64 = embeddings[:, :64]    # Use first 64 dimensions

print(f"256-dim embeddings shape: {embeddings_256.shape}")
```

### Similarity Computation

```python
from sentence_transformers import util

# Compute similarity between sentences
sentence1 = "القطة تجلس على السجادة"
sentence2 = "الكلب يلعب في الحديقة"

embeddings = model.encode([sentence1, sentence2])
similarity = util.cos_sim(embeddings[0], embeddings[1])
print(f"Similarity: {similarity.item():.4f}")
```

### NLI Classification

```python
def classify_nli_pair(premise, hypothesis, threshold=0.6):
    """
    Classify Natural Language Inference relationship

    Args:
        premise: The premise sentence
        hypothesis: The hypothesis sentence  
        threshold: Similarity threshold for classification

    Returns:
        str: 'entailment' if similarity > threshold, else 'contradiction'
    """
    embeddings = model.encode([premise, hypothesis])
    similarity = util.cos_sim(embeddings[0], embeddings[1]).item()

    return 'entailment' if similarity > threshold else 'contradiction'

# Example usage
premise = "الرجل يقرأ كتاباً في المكتبة"
hypothesis = "شخص يقرأ في مكان هادئ"

result = classify_nli_pair(premise, hypothesis)
print(f"Relationship: {result}")
```

### Choosing the Right Dimension

- **768 dimensions**: Maximum accuracy for critical applications
- **512 dimensions**: Best balance of performance and efficiency  
- **256 dimensions**: Good performance with 3x faster inference
- **128 dimensions**: Suitable for real-time applications
- **64 dimensions**: Ultra-fast inference for large-scale processing

## Training Details

### Dataset
- **Training Data**: Arabic-NLI-Pair-Class dataset from Omartificial-Intelligence-Space
- **Language**: Modern Standard Arabic (MSA)
- **Task Type**: Binary classification (entailment vs contradiction)

### Training Configuration
- **Base Model**: aubmindlab/bert-base-arabertv02
- **Max Sequence Length**: 75 tokens
- **Batch Size**: 64
- **Epochs**: 5
- **Matryoshka Dimensions**: [768, 512, 256, 128, 64]
- **Loss Function**: MatryoshkaLoss with CosineSimilarityLoss
- **Optimization**: AdamW with automatic mixed precision (AMP)

## Use Cases

1. **Arabic Text Similarity**: Measure semantic similarity between Arabic texts
2. **Natural Language Inference**: Determine logical relationships between Arabic sentences
3. **Information Retrieval**: Find relevant Arabic documents based on queries
4. **Semantic Search**: Build Arabic search engines with semantic understanding
5. **Text Classification**: Use embeddings as features for downstream Arabic NLP tasks

## Citation

If you use this model in your research, please cite:

```bibtex
@model{arabic-bert-nli-matryoshka,
  title={Arabic BERT NLI Matryoshka Embeddings},
  author={Ahmed Mouad},
  year={2025},
  url={https://huggingface.co/AhmedZaky1/arabic-bert-nli-matryoshka}
}
```

## Acknowledgments

- **AraBERT Team**: For the excellent base model (aubmindlab/bert-base-arabertv02)
- **Sentence Transformers**: For the robust training framework
- **Matryoshka Representation Learning**: For the innovative approach to nested embeddings
- **Arabic NLI Dataset**: Omartificial-Intelligence-Space for the training data

## License

This model is released under the Apache 2.0 License.

---

**Model Version**: 1.0  
**Last Updated**: May 2025  
**Framework**: sentence-transformers  
**Language**: Arabic (العربية)