Hi Zindians,

A participant flagged that a subset of the “exit rating” videos in the dataset had been incorrectly labelled. The mislabeled samples could introduce noise and unfairness in the evaluation.

To keep the challenge fair and consistent for everyone, we’ve made the following updates:

• We are removing all exit-rating videos from the evaluation.
• The challenge will now be scored only on entry ratings.
• An updated sample submission file (with exit-rating IDs removed) is now available on the Data page.

All previous submissions will be automatically rescored using the corrected reference set. No action is required on your part - but you’re welcome to resubmit using the updated sample submission. The rescore is currently running and I’ll update this post when all past submissions have been rescored.

These changes ensure that:

• Mislabeled videos no longer affect your score
• Everyone is evaluated on a better quality subset
• The competition remains fair, reliable, and leaderboard-focused
• Models aren't penalised for errors caused by faulty annotations

We appreciate your understanding and patience - and we hope the cleaner evaluation will help you build even better models.

If you have any questions, feel free to drop them in the chat.

Happy modelling!