cmd = [
"lighteval",
"vllm",
"model_name=h-d-h/smollm3-sft",
"leaderboard|gsm8k|0", # Try leaderboard suite format
"--push-to-hub",
"--results-org", "h-d-h"
#"lighteval|gsm8k|0|0"
]

The command above is part of a script created to run the command with hf jobs. Actual CLI was something like:

hf jobs uv run --env-file .env --flavor a100-large --secrets MY_BIGGEST_SECRET ./lighteval.py

a smol course org

Hi.

There's an issue merging:

Cannot load dataset for 9 : h-d-h/details_h-d-h__smollm3-sft_private Dataset 'h-d-h/details_h-d-h__smollm3-sft_private' doesn't exist on the Hub or cannot be accessed.

Is the dataset public?

Oops! Try now.

Cannot merge
This branch has merge conflicts in the following files:
  • submissions.json

Sign up or log in to comment