|
|
--- |
|
|
base_model: meta-llama/Llama-3.2-1B |
|
|
library_name: peft |
|
|
license: apache-2.0 |
|
|
datasets: |
|
|
- trumancai/revela_training_corpus |
|
|
language: |
|
|
- en |
|
|
tags: |
|
|
- retrieval |
|
|
--- |
|
|
|
|
|
# Model Summary |
|
|
|
|
|
Revela is a self-supervised bi-encoder retrieval model, trained on raw text with an in-batch attention mechanism. This version, `Revela-1b` was trained on a corpus of 320K batches in the size of 16 by chunking Wikipedia. See the paper for more details. |
|
|
|
|
|
- **Repository:** [TRUMANCFY/Revela](https://github.com/TRUMANCFY/Revela) |
|
|
<!-- - **Paper:** --> |
|
|
- **Training Dataset:** [trumancai/revela_training_corpus](https://huggingface.co/datasets/trumancai/revela_training_corpus) |
|
|
|
|
|
# Other Links |
|
|
| Binary | Description | |
|
|
|:-------|:------------| |
|
|
| [trumancai/Revela-code-3b](https://huggingface.co/trumancai/Revela-code-3b) | 3 B-parameter code-retriever. | |
|
|
| [trumancai/Revela-code-1b](https://huggingface.co/trumancai/Revela-code-1b) | 1 B-parameter code-retriever. | |
|
|
| [trumancai/Revela-code-500M](https://huggingface.co/trumancai/Revela-code-500M) | 500 M-parameter code-retriever. | |
|
|
| [trumancai/Revela-3b](https://huggingface.co/trumancai/Revela-3b) | 3 B-parameter Wikipedia retriever. | |
|
|
| [trumancai/Revela-1b](https://huggingface.co/trumancai/Revela-1b) | 1 B-parameter Wikipedia retriever. | |
|
|
| [trumancai/Revela-500M](https://huggingface.co/trumancai/Revela-500M) | 500 M-parameter Wikipedia retriever. | |
|
|
| [trumancai/revela_code_training_corpus](https://huggingface.co/datasets/trumancai/revela_code_training_corpus) | Code training corpus. | |
|
|
| [trumancai/revela_training_corpus](https://huggingface.co/datasets/trumancai/revela_training_corpus) | Wikipedia training corpus. | |
|
|
|
|
|
|
|
|
# Usage |
|
|
We can evaluate the trained models with customized [mteb](https://github.com/TRUMANCFY/mteb). |
|
|
|
|
|
|
|
|
```python |
|
|
from mteb.model_meta import ModelMeta |
|
|
from mteb.models.repllama_models import RepLLaMAWrapper, _loader |
|
|
|
|
|
revela_llama_3b = ModelMeta( |
|
|
loader=_loader( |
|
|
RepLLaMAWrapper, |
|
|
base_model_name_or_path="meta-llama/Llama-3.2-3B", |
|
|
peft_model_name_or_path="trumancai/Revela-3b", |
|
|
device_map="auto", |
|
|
torch_dtype=torch.bfloat16, |
|
|
), |
|
|
name="trumancai/Revela-3b", |
|
|
languages=["eng_Latn"], |
|
|
open_source=True, |
|
|
revision="2b31c92f23acc46762587ea37cb55032da788561", # base-peft revision |
|
|
release_date="2025-04-13", |
|
|
) |
|
|
revela_llama_3b_model = revela_llama_3b.loader() |
|
|
|
|
|
evaluation = mteb.MTEB(tasks=["SciFact", "NFCorpus"]) |
|
|
evaluation.run(model=revela_llama_3b_model, output_folder="results/Revela-3b") |
|
|
``` |
|
|
|
|
|
|
|
|
# License |
|
|
|
|
|
# Citation |
|
|
|
