metadata
language: en
license: apache-2.0
tags:
- sentence-transformers
- text-embedding
- rag
- scion-architecture
datasets:
- custom
metrics:
- retrieval
SCION Questions Embedding Model
This model is fine-tuned from sentence-transformers/all-MiniLM-L6-v2 on a dataset of questions about SCION internet architecture paired with relevant document passages.
Model description
The model was fine-tuned to optimize for retrieval performance in RAG applications related to SCION internet architecture.
Intended uses & limitations
This model is specifically trained for retrieving relevant passages from a corpus of SCION Internet Architecture related documentation, specifications and research papers.
Training procedure
The model was trained using sentence-transformers with MultipleNegativesRankingLoss on query-document pairs.
Performance
| Metric | Base Model | Fine-tuned | Improvement |
|---|---|---|---|
| ndcg@10 | 0.6009 | 0.7928 | +31.92% |
| mrr | 0.5476 | 0.7475 | +36.52% |
| hits@1 | 0.4395 | 0.6457 | +46.94% |
| hits@3 | 0.6211 | 0.8327 | +34.08% |
| hits@10 | 0.7686 | 0.9323 | +21.30% |