Intron AfriSpeech-200 Automatic Speech Recognition Challenge
Can you create an automatic speech recognition (ASR) model for African accents, for use by doctors?
Prize
$5 000 USD
Time
2 months to go
Participants
11 active · 193 enrolled
Advanced
Automatic Speech Recognition
Health
Media
Leaderboard WER
Help · 28 Feb 2023, 12:45 · 2

Can you confirm whether the WER displayed on the leaderboard applies solely to the Dev dataset or if it includes both the Dev and Test datasets (the latter of which is not accessible)?

Discussion 2 answers

Here's what was shared previously to address this issue:

the dev_meta table and the dev/validation dataset contain 3227 samples/audios, the test_meta/test dataset that will be uploaded later contains 5064 samples, while the submission file should contain all IDs (8091). The easiest solution is to concatenate your dev_meta predictions with, a " " space for the test_meta transcripts.

28 Feb 2023, 14:09
Upvotes 0

Let's take your LB WER as an example:

wer_all=0.492613365 is the WER for all submission ids (8091), the WER for spaces is 1 (for 5064 test samples)

so to calculate the WER for only the dev_metadata predictions, it will be :

wer_dev = 3227*wer_all/(8091-5061*wer_all) = 3227/(8091/0.4926 - 5064/1) = 0.284

your model is not that overfitted hh

28 Feb 2023, 14:33
Upvotes 0