How to calculate the PER and FER for this model?
Hi, I tried calculating PER and FER locally using the TIMIT dataset, but I have a question.
The TIMIT dataset uses TIMIT Phonetic Transcription, while this model uses the International Phonetic Alphabet, right?
Then how should I calculate PER and FER?
Since TIMIT uses TIMIT Phonetic Transcription, and the model uses IPA, would the PER and FER values still reflect the correct accuracy?
How did you calculate the PER and FER based on the TIMTI dataset?
Great question! You’ll have to convert the TIMIT phonemes to IPA. There are a bunch of libraries for this, but we found that they all contain subtle errors for edge cases so we made our own. You can see how we apply it to TIMIT here. Our full eval script is also open-source. If you just want to compare this model to other models, we’ve made a leaderboard for this.