open-llm-leaderboard/open_llm_leaderboard

Resource: Understanding the new benchmarks

pinned

2

#796 opened 7 months ago by

rombodawg

💬 Discussion thread: Model contamination techniques 💬

pinned

34

#472 opened about 1 year ago by

clefourrier

💎 Resources and community initiatives around the Leaderboard! 💎

pinned

#174 opened over 1 year ago by

clefourrier

🚩 Report: Not working

4

#1082 opened about 11 hours ago by

ehristoforu

Unable to submit byroneverson/Yi-1.5-34B-Chat-abliterated

#1080 opened 8 days ago by

CombinHorizon

why evaluation is taking long time for large llm?

#1079 opened 9 days ago by

Imran1

Qwen2.5-32B merge eval error

2

#1078 opened 10 days ago by

ehristoforu

All Orca Mini v* 70B Models Evaluation Failing

6

#1076 opened 13 days ago by

pankajmathur

Model results missing

1

#1075 opened 15 days ago by

maldv

The quota has been exceeded.

5

#1074 opened 16 days ago by

lunahr

Suggestion

#1073 opened 16 days ago by

zelk12

fix-gptq

1

#1072 opened 16 days ago by

alozowski

Bug in Parameters count for GPTQ quantized model

4

#1068 opened 20 days ago by

Qubitium

Resubmitting a model to use `chat_template` doesn't re-evaluate, but does change `chat_template` column

1

#1066 opened 22 days ago by

xzuyn

failed models, check logs

#1062 opened 23 days ago by

CombinHorizon

docker-container-test

#1047 opened about 1 month ago by

alozowski

Normalization for MuSR object placement

1

#1040 opened about 2 months ago by

recojt

update ranking number when editing filters

3

#1036 opened about 2 months ago by

clem

Requested feature: searchable tags for official supported languages for each model.

1

#1035 opened about 2 months ago by

Reza2kn

Proposal for new column

3

#1032 opened about 2 months ago by

Yuma42

I can't replicate results.

5

#1016 opened 3 months ago by

Pretergeek

merged models marked as non merged in the leaderboard

3

#993 opened 3 months ago by

fblgit

Feature Request: change request file format to disambiguate chat and non-chat models?

4

#954 opened 4 months ago by

CombinHorizon

Are Qwen models pretrained or continuously pretrained?

7

#941 opened 4 months ago by

djstrong

Feature request: Add toggle to only show models with linked dataset

1

#763 opened 8 months ago by

ThiloteE

Feature request: Hide models with insufficient model card from default view in leaderboard

4

#762 opened 8 months ago by

ThiloteE

Discussion: naming pattern to converge on to better identify fine-tunes

17

#761 opened 8 months ago by

ThiloteE

Crowd-Source Hardware for the LeaderBoard?

4

#570 opened about 1 year ago by

ibivibiv

Feature request: Using weights hash to identify duplicates

1

#422 opened about 1 year ago by

mrfakename

Tool: Adding evaluation results to model cards

47

#370 opened about 1 year ago by

Weyaxi

Feature suggestion: average of selected (rather than all) columns

4

#368 opened about 1 year ago by

Minus0

Tool: Open LLM Leaderboard Model Renamer

31

#310 opened over 1 year ago by

Weyaxi