Hello Zindians
A few clarifications on this challenge:
Tie breaker.
The private leaderboard will be used in the final rankings.
If two solutions earn identical scores on the leaderboard, the tiebreaker will be the date and time in which the submission was made (the earlier solution will win).
Perfect score
Note that there is a 20:80 split between the public and private test sets. So one can have a perfect score in the public leaderboard and a terrible score in the private or vice versa.
Zindi maintains a public leaderboard and a private leaderboard for each competition. The Public Leaderboard includes approximately 20% of the test dataset. While the competition is open, the Public Leaderboard will rank the submitted solutions by the accuracy score they achieve. Upon close of the competition, the Private Leaderboard, which covers the other 80% of the test dataset, will be made public and will constitute the final ranking for the competition.
Predictions
Note that the expected output from your models is probabilities. You should not post-process the probabilities by setting thresholds, rounding off or changing the probabilities in any way.
If the error metric requires probabilities to be submitted, do not set thresholds (or round your probabilities) to improve your place on the leaderboard. In order to ensure that the client receives the best solution Zindi will need the raw probabilities. This will allow the clients to set thresholds to their own needs.
Resource restriction
Use only the free colab GPU resources or an equivalent GPU with same specifications as the NVIDIA Tesla K80
To make this challenge accessible to all, there are restrictions on run time. You are allowed a maximum of 7 hours’ train time and 2 hours’ inference time on the whole test set, with a maximum 1 minute inference per image.
We encourage you to use Google Colab which allows you access to a NVIDIA Tesla K80. If you choose to use a different GPU, it may not exceed the specs of an NVIDIA Tesla K80.
Thanks for the clarification. Quick follow-up
Your submission file should look like this (numbers to show format only):
Does this mean that if I have a probability prediction of say 0.9895 for a given image, I can round this to 0.99 or do I need to truncate it to 0.98?
Also, the sample image IDs don't have image extensions in the above sample, however, the backend grading seems to require it. Could you clarify this, please?
@DoubleAgent you dont have to round off or truncate the probabilities. What should go to the submission file are the raw probabilities from your model as they are.
Include the image extensions in the Image_id column of your submission files.Use the provided sample_submission file for guidance.
Will the private leader board be open to everyone or only for a few selected people?
@RiekertCodes At the end of the competition, the private leaderboard will be open to everyone
@Brainiac Just like @DoubleAgent said. What if the generated submission file looks like the following below without any sort of rounding off method being applied:
Assuming submission file looks like this: