Can you confirm whether the WER displayed on the leaderboard applies solely to the Dev dataset or if it includes both the Dev and Test datasets (the latter of which is not accessible)?
Here's what was shared previously to address this issue:
the dev_meta table and the dev/validation dataset contain 3227 samples/audios, the test_meta/test dataset that will be uploaded later contains 5064 samples, while the submission file should contain all IDs (8091). The easiest solution is to concatenate your dev_meta predictions with, a " " space for the test_meta transcripts.
Here's what was shared previously to address this issue:
the dev_meta table and the dev/validation dataset contain 3227 samples/audios, the test_meta/test dataset that will be uploaded later contains 5064 samples, while the submission file should contain all IDs (8091). The easiest solution is to concatenate your dev_meta predictions with, a " " space for the test_meta transcripts.
Let's take your LB WER as an example:
wer_all=0.492613365 is the WER for all submission ids (8091), the WER for spaces is 1 (for 5064 test samples)
so to calculate the WER for only the dev_metadata predictions, it will be :
wer_dev = 3227*wer_all/(8091-5061*wer_all) = 3227/(8091/0.4926 - 5064/1) = 0.284
your model is not that overfitted hh