This really hurts me 🫤. The rules were changed after the competition. The client could have used the top 30 solutions that seemed more appropriate since He only wants image models. But changing the scores seems intolerable to me. The leaderboard should be the leaderboard. The private top 3 score seems artificial to me, just to get them ahead. I'm going to leave the platform for a while.

I don't really understand what the end of this competition rule is. My model uses image encode with timm and combines it with start, end, genotype and stage values ​​and uses lgbm for the final prediction. Does Zindi only want a solution using the image without using other values ​​(start, end, genotype, stage)?, or must use CNN?, or something else. Please explain so that we can anticipate problems related to the rules that may recur in future competitions. thank you for your response.

Hey @AJoel could the winning solutions be open sourced so that we can learn from them? Atleast that will bring tranparency to the below issue which I think was the main judging point?

Correctness of Preprocessing: Image merging must be performed accurately, ensuring proper alignment of the left and right image segments and stacking the images vertically.

if zindi selected less sentitive to outliers metric for evaluation small test sample, this couldn't happened.

Dear Zindians,

We would like to address a few of the concerns raised in this discussion thread, in the interest of transparency and constructive communication with the community. In addition, we outline some platform changes that we are planning to ensure better transparency in future. First let me say, we really appreciate all your messages and engagement, even if some of it is critical of the platform! That shows us that you all really care, and as always we will try our best to be open and honest with you.

Rules changes: We would like to clarify that we made no rules changes during or after this challenge. As always, the rules of Zindi challenges state that the solutions must be useful to the client according to the terms and rules laid out in the challenge. Everyone whose ranking went down in this challenge were deemed to have submitted solutions that would not help the client solve the intended challenge. We encourage all participants to read the rules of every challenge carefully, and remember that we are always going to prioritise solutions that are of real use to clients rather than simply the highest scoring submissions on the leaderboard. We always try to communicate these criteria as clearly as possible.

Reasoning for selecting the winners: Regarding the question of  @sys_ts__ and others, winners were selected on the basis of usefulness to the partner organisation, as per this rule:

Zindi is committed to providing solutions of value to our clients and partners. To this end, we reserve the right to disqualify your submission on the grounds of usability or value. This includes but is not limited to the use of data leaks or any other practices that we deem to compromise the inherent value of your solution.

In this case, most submissions in the top 30 failed in this regard, in one of two ways:

1. some models used plant number as a feature, which does not hold predictive value.
2. many models pre-processed images incorrectly, leading to meaningless predictions of plant volume.

The above rule supercedes all others in all Zindi competitions, so we encourage you to always think carefully about the usefulness of your models in a real-world situation.

Manually-adjusted scoring on the leaderboard: Many of you have noticed that the top 3 ranks have been manually adjusted. This was done to ensure that the best solutions ranked above all of the disqualified solutions. This is a limitation of the platform - manually adjusting scores is currently the only way to adjust rankings. We acknowledge that this is a poor solution, and we are taking steps to address this - please see below.

Selection of error metric: The error metric used was specifically requested by the client. We agree that the metric chosen was not ideal for this challenge, and as always we will learn from this challenge when choosing error metrics in future.

Future changes to address these issues: Your concerns around the handling of this challenge leaderboard are valid and have been heard. To avoid such issues in future, we will be rolling out the following features soon:

• We will create a visible change log for each challenge, to ensure transparency regarding any changes made during the challenge period
• We will create a function to allow reranking of the final leaderboard, without changing private leaderboard scoring
• We will create a way to indicate any manual rank changes made on the leaderboard, to ensure transparency regarding any changes made after the challenge period

As always, we appreciate your concerns and assure you all that we are always trying our best to make this platform the best it can be for all of you as well as for all of our partners. If you have any comments or questions, please do share them.

Happy hacking!