Hi, could the organizers please clarify the rule about probability thresholds?
The competition page says that setting a probability threshold is strictly forbidden and that TargetF1 should be based on the default threshold of 0.5.
Does this requirement apply to every submission made during the public leaderboard phase, including experimental submissions, or only to the final two submissions selected for private evaluation?
For example, would it be allowed to submit temporary threshold variants to study model calibration, provided that the final selected and reproducible model uses:
TargetF1 = (predicted_probability >= 0.5)
Could you also clarify whether threshold-adjusted submissions are detected automatically by the platform, checked during code review, or reviewed in some other way?
This matters because, if non-compliant threshold tuning can improve public leaderboard scores without being detected until the end, participants may end up chasing leaderboard results that cannot be reproduced under the official rules. It would be helpful to know whether the current leaderboard should be assumed to reflect only submissions using the required 0.5 threshold.
An explicit clarification would help everyone experiment fairly and avoid unintentionally violating the rules.
i agree with this question, pls. answer, organizers! i have been very hesitant to do any type of class weighting as it appears to be prohibitied by the rules - but there is no way to tell if leaderboard submissions are using it!