Since the final outcome depends on unseen private data, limiting evaluation to only two submissions per participant does not seem like the fairest approach. The public leaderboard clearly was not a perfect reflection of private leaderboard performance, as many of the eventual top performers had public F1 scores below 0.9. This means that participants with stronger public scores may also have had alternative submissions that would have performed better on the private data, but those submissions were excluded simply because only 2 highest public-scoring entry was chosen per participant.

As a result, some competitors may have been unfairly disadvantaged, not because they lacked better models, but because their potentially stronger private-board submissions were never evaluated. A more transparent and equitable approach would have been to run all submissions against the private dataset, just as was done for the public evaluation, choosing the best submission made by the participant as evaluated against the private data. That would have produced a more accurate final ranking and a leaderboard that better reflected true model performance.