LLM Hallucination Leaderboard
🚀
Visualize data with an interactive interface
Vectara
company
AI & ML interests
retrieval augmented generation, grounded generation, large language models, LLMs, question answering, chatbot
Recent Activity
View all activity
vectara's activity
forrest-vectara
updated
a
Space
1 day ago
forrest-vectara
published
a
Space
2 days ago
forrest-vectara
updated
a
Space
2 days ago
forrest-vectara
updated
a
dataset
3 days ago
forrest-vectara
published
a
dataset
3 days ago
Search and filtering do not work
➕
👍
2
3
#10 opened 2 months ago
by
harbachonak
ofermend
updated
3
Spaces
5 days ago
ofermend
updated
6
Spaces
6 days ago
UCSF Ortho Demo
🐨
Ask questions about UCSF Orthopedics
HMC Demo
🐨
Ask questions about Harvard Management
2
Justice Harvard
🐨
Teacher Assistant for Justice Harvard using vectara-agentic
6
Hacker News chat
🐨
chatbot with HN data using vectara-agentic
7
Legal Assistant
🐨
Legal Assistant using vectara-agentic
CFPB Assistant
🐨
CFPB Assistant using vectara-agentic
Post
347
Excited to share open-rag-eval (https://github.com/vectara/open-rag-eval) a new open source project to help scale RAG evaluation. The key benefit: it does not require golden answers so much more scalable.
Would love any thoughts or feedback (or even better - if you want to contribute a PR that would be great).
Would love any thoughts or feedback (or even better - if you want to contribute a PR that would be great).
stsui96
updated
a
dataset
18 days ago
Post
1600
Last year, I curated & generated a few multilingual SFT and DPO datasets by translating English SFT/DPO datasets into 9-10 languages using the
mistralai/Mistral-7B-Instruct-v0.2 model.
I hope it helps the community for pretraining/instruction tuning multilingual LLMs! I added a small diagram to briefly describe which datasets are added and their sources.
Happy to collaborate in either using these datasets for instruction FT, or wishes to extend translated versions of newer SFT/DPO english datasets!
nthakur/multilingual-sft-and-dpo-datasets-67eaf56fe3feca5a57cf7d74
I hope it helps the community for pretraining/instruction tuning multilingual LLMs! I added a small diagram to briefly describe which datasets are added and their sources.
Happy to collaborate in either using these datasets for instruction FT, or wishes to extend translated versions of newer SFT/DPO english datasets!
nthakur/multilingual-sft-and-dpo-datasets-67eaf56fe3feca5a57cf7d74
clefourrier
posted
an
update
2 months ago
Post
2450
Gemma3 family is out! Reading the tech report, and this section was really interesting to me from a methods/scientific fairness pov.
Instead of doing over-hyped comparisons, they clearly state that **results are reported in a setup which is advantageous to their models**.
(Which everybody does, but people usually don't say)
For a tech report, it makes a lot of sense to report model performance when used optimally!
On leaderboards on the other hand, comparison will be apples to apples, but in a potentially unoptimal way for a given model family (like some user interact sub-optimally with models)
Also contains a cool section (6) on training data memorization rate too! Important to see if your model will output the training data it has seen as such: always an issue for privacy/copyright/... but also very much for evaluation!
Because if your model knows its evals by heart, you're not testing for generalization.
Instead of doing over-hyped comparisons, they clearly state that **results are reported in a setup which is advantageous to their models**.
(Which everybody does, but people usually don't say)
For a tech report, it makes a lot of sense to report model performance when used optimally!
On leaderboards on the other hand, comparison will be apples to apples, but in a potentially unoptimal way for a given model family (like some user interact sub-optimally with models)
Also contains a cool section (6) on training data memorization rate too! Important to see if your model will output the training data it has seen as such: always an issue for privacy/copyright/... but also very much for evaluation!
Because if your model knows its evals by heart, you're not testing for generalization.
nthakur
authored
a
paper
3 months ago