whisper-leaderboard

#31
by Steveeeeeeen HF staff - opened

What does this PR do?

This PR adds a dedicated Whisper Model Leaderboard tab to compare different Whisper model implementations across various backends and hardware configurations.

Key Changes:

  • Added new environment variables for Whisper-specific repository and paths
  • Extended load_all_info_from_dataset_hub() to handle both original and Whisper evaluation results
  • Created a new "🔄 Whisper Model Leaderboard" tab with:
    • Additional columns for Backend and Device information
    • Interactive filters to compare results by backend type and hardware
    • Same evaluation metrics (WER, RTFx) as the main leaderboard

Why?

This dedicated view will help users:

  • Compare performance across different Whisper implementations
  • Understand speed/accuracy tradeoffs on different hardware
  • Make informed decisions about which backend to use for their specific needs

Testing

  • Verified that both original and Whisper leaderboards load correctly
  • Tested filtering functionality with multiple backend/device combinations
  • Confirmed that all WER and RTFx metrics display properly

Screenshots

Capture d’écran 2024-12-02 à 10.00.45.png

Steveeeeeeen changed pull request status to closed
Steveeeeeeen changed pull request status to open
Ready to merge
This branch is ready to get merged automatically.

Sign up or log in to comment