FreshStack: Building Realistic Benchmarks for Evaluating Retrieval on Technical Documents
Paper
•
2504.13128
•
Published
•
5
BEIR
university
AI & ML interests
BEIR (Benchmarking IR) consists of a homogenous benchmark for diverse sentence or passage level IR tasks. It provides a common and easy framework for the cross-domain evaluation of your retrieval models.
Recent Activity
View all activity
BeIR's activity
Qrel file missing
1
#3 opened 26 days ago
by
Aabylay
nthakurÂ
authored
3
papers
28 days ago
Post
1653
Last year, I curated & generated a few multilingual SFT and DPO datasets by translating English SFT/DPO datasets into 9-10 languages using the
mistralai/Mistral-7B-Instruct-v0.2 model.
I hope it helps the community for pretraining/instruction tuning multilingual LLMs! I added a small diagram to briefly describe which datasets are added and their sources.
Happy to collaborate in either using these datasets for instruction FT, or wishes to extend translated versions of newer SFT/DPO english datasets!
nthakur/multilingual-sft-and-dpo-datasets-67eaf56fe3feca5a57cf7d74
I hope it helps the community for pretraining/instruction tuning multilingual LLMs! I added a small diagram to briefly describe which datasets are added and their sources.
Happy to collaborate in either using these datasets for instruction FT, or wishes to extend translated versions of newer SFT/DPO english datasets!
nthakur/multilingual-sft-and-dpo-datasets-67eaf56fe3feca5a57cf7d74
nthakurÂ
authored
a
paper
4 months ago
nthakurÂ
authored
a
paper
7 months ago
nthakurÂ
authored
a
paper
10 months ago
Post
3552
🦢 The SWIM-IR dataset contains 29 million text-retrieval training pairs across 27 diverse languages. It is one of the largest synthetic multilingual datasets generated using PaLM 2 on Wikipedia! 🔥🔥
SWIM-IR dataset contains three subsets :
- Cross-lingual:
- Monolingual:
- Indic Cross-lingual:
Check it out:
https://huggingface.co/collections/nthakur/swim-ir-dataset-662ddaecfc20896bf14dd9b7
SWIM-IR dataset contains three subsets :
- Cross-lingual:
nthakur/swim-ir-cross-lingual- Monolingual:
nthakur/swim-ir-monolingual- Indic Cross-lingual:
nthakur/indic-swim-ir-cross-lingualCheck it out:
https://huggingface.co/collections/nthakur/swim-ir-dataset-662ddaecfc20896bf14dd9b7
nthakurÂ
authored
9
papers
over 1 year ago
Resources for Brewing BEIR: Reproducible Reference Models and an Official Leaderboard
Paper
•
2306.07471
•
Published
NoMIRACL: Knowing When You Don't Know for Robust Multilingual Retrieval-Augmented Generation
Paper
•
2312.11361
•
Published
•
1
HAGRID: A Human-LLM Collaborative Dataset for Generative Information-Seeking with Attribution
Paper
•
2307.16883
•
Published
Augmented SBERT: Data Augmentation Method for Improving Bi-Encoders for Pairwise Sentence Scoring Tasks
Paper
•
2010.08240
•
Published
Evaluating Embedding APIs for Information Retrieval
Paper
•
2305.06300
•
Published
•
1
GPL: Generative Pseudo Labeling for Unsupervised Domain Adaptation of Dense Retrieval
Paper
•
2112.07577
•
Published
Making a MIRACL: Multilingual Information Retrieval Across a Continuum of Languages
Paper
•
2210.09984
•
Published
•
2
BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Models
Paper
•
2104.08663
•
Published
•
3
Leveraging LLMs for Synthesizing Training Data Across Many Languages in Multilingual Dense Retrieval
Paper
•
2311.05800
•
Published
•
4
nreimersÂ
authored
a
paper
about 2 years ago
nreimersÂ
updated
a
model
over 3 years ago