metadata
base_model: meta-llama/Llama-3.2-1B
library_name: peft
license: apache-2.0
datasets:
- trumancai/revela_training_corpus
language:
- en
tags:
- retrieval
Model Summary
Revela is a self-supervised bi-encoder retrieval model, trained on raw text with an in-batch attention mechanism. This version, Revela-1b was trained on a corpus of 320K batches in the size of 16 by chunking Wikipedia. See the paper for more details.
- Repository: TRUMANCFY/Revela
- Training Dataset: trumancai/revela_training_corpus
Other Links
| Binary | Description |
|---|---|
| trumancai/Revela-code-3b | 3 B-parameter code-retriever. |
| trumancai/Revela-code-1b | 1 B-parameter code-retriever. |
| trumancai/Revela-code-500M | 500 M-parameter code-retriever. |
| trumancai/Revela-3b | 3 B-parameter Wikipedia retriever. |
| trumancai/Revela-1b | 1 B-parameter Wikipedia retriever. |
| trumancai/Revela-500M | 500 M-parameter Wikipedia retriever. |
| trumancai/revela_code_training_corpus | Code training corpus. |
| trumancai/revela_training_corpus | Wikipedia training corpus. |
Usage
We can evaluate the trained models with customized mteb.
from mteb.model_meta import ModelMeta
from mteb.models.repllama_models import RepLLaMAWrapper, _loader
revela_llama_3b = ModelMeta(
loader=_loader(
RepLLaMAWrapper,
base_model_name_or_path="meta-llama/Llama-3.2-3B",
peft_model_name_or_path="trumancai/Revela-3b",
device_map="auto",
torch_dtype=torch.bfloat16,
),
name="trumancai/Revela-3b",
languages=["eng_Latn"],
open_source=True,
revision="2b31c92f23acc46762587ea37cb55032da788561", # base-peft revision
release_date="2025-04-13",
)
revela_llama_3b_model = revela_llama_3b.loader()
evaluation = mteb.MTEB(tasks=["SciFact", "NFCorpus"])
evaluation.run(model=revela_llama_3b_model, output_folder="results/Revela-3b")