Makerere Fall Armyworm Crop Challenge
Can you determine if maize crops have been affected by the fall armyworm pest?
Prize
$1 000 USD
Time
Ended 8 months ago
Participants
156 active · 671 enrolled
Helping
Uganda
Good for beginners
Classification
Computer Vision
Agriculture
A few things to note...
Platform · 28 Apr 2022, 10:44 · 5

Hello Zindians

A few clarifications on this challenge:

Tie breaker.

The private leaderboard will be used in the final rankings.

If two solutions earn identical scores on the leaderboard, the tiebreaker will be the date and time in which the submission was made (the earlier solution will win).

Perfect score

Note that there is a 20:80 split between the public and private test sets. So one can have a perfect score in the public leaderboard and a terrible score in the private or vice versa.

Zindi maintains a public leaderboard and a private leaderboard for each competition. The Public Leaderboard includes approximately 20% of the test dataset. While the competition is open, the Public Leaderboard will rank the submitted solutions by the accuracy score they achieve. Upon close of the competition, the Private Leaderboard, which covers the other 80% of the test dataset, will be made public and will constitute the final ranking for the competition.

Predictions

Note that the expected output from your models is probabilities. You should not post-process the probabilities by setting thresholds, rounding off or changing the probabilities in any way.

If the error metric requires probabilities to be submitted, do not set thresholds (or round your probabilities) to improve your place on the leaderboard. In order to ensure that the client receives the best solution Zindi will need the raw probabilities. This will allow the clients to set thresholds to their own needs.

Resource restriction

Use only the free colab GPU resources or an equivalent GPU with same specifications as the NVIDIA Tesla K80

To make this challenge accessible to all, there are restrictions on run time. You are allowed a maximum of 7 hours’ train time and 2 hours’ inference time on the whole test set, with a maximum 1 minute inference per image.

We encourage you to use Google Colab which allows you access to a NVIDIA Tesla K80. If you choose to use a different GPU, it may not exceed the specs of an NVIDIA Tesla K80.

Discussion 5 answers

Thanks for the clarification. Quick follow-up

Your submission file should look like this (numbers to show format only):

Image_ID             Target
ID_D9ONL553           0.13
ID_263YTILY           0.87

Does this mean that if I have a probability prediction of say 0.9895 for a given image, I can round this to 0.99 or do I need to truncate it to 0.98?

Also, the sample image IDs don't have image extensions in the above sample, however, the backend grading seems to require it. Could you clarify this, please?

28 Apr 2022, 10:52
Upvotes 0

@DoubleAgent you dont have to round off or truncate the probabilities. What should go to the submission file are the raw probabilities from your model as they are.

Include the image extensions in the Image_id column of your submission files.Use the provided sample_submission file for guidance.

Will the private leader board be open to everyone or only for a few selected people?

28 Apr 2022, 12:13
Upvotes 0

@RiekertCodes At the end of the competition, the private leaderboard will be open to everyone

@Brainiac Just like @DoubleAgent said. What if the generated submission file looks like the following below without any sort of rounding off method being applied:

Assuming submission file looks like this:

Image_ID             Target
ID_D9ONL553           0.1364
ID_263YTILY           0.8745

ID_5673HDJO           1.0000

ID_298GBNS            0.8745


I am laying my emphasis on the row with Image_ID: ID_563HDJO with perfect "100% score".

Should we need to retrain our model for any of this Image_ID not to have this perfect score 100% knowingfully that no rounding off method being applied???

What's the Zindi take on this?

Thanks.
29 Apr 2022, 06:18
Upvotes 1