Could someone please provide some clarity around the code review process and the determination of the final LB for this challenge? It says on the Info page that "[t]he top 20 users on the leaderboard will be requested to submit their inference script, model weights, and number of parameters for the Lelapa team to confirm that your inference script runs and that your reported number of paramesters is correct. Based on your private leaderboard score, combined with the factor of number parameters reduced by will be your final score. The top 5 users will receive an email requesting full code for the Lelapa team to code review. Prizes will go to the top 3 reproducible solutions. Final score will be calculated by: zindi_score = (PrivateLB_score + (1-(size/PARAM_SIZE))*PrivateLB_score )/2".

I interpreted that to mean that the full code review will only happen after the initial review (i.e., inference script, model weights, and number of parameters) so that the top 5 can be identified based on the final score that also takes the number of parameters into account. If the number of parameters is not taken into account for a bigger group of participants, someone with a smaller model but a private LB score slightly outside the top few places could be negatively impacted. However, I received the standard full code review email this morning with no mention of the additional request for this challenge.

Related to this, our best submission (from those that we selected for the comp) was automatically selected based on the private LB score only. Should the number of parameters (and thus the final score according to the formula) not be considered used to pick the best submission instead?

Can the process for this challenge please be clarified?