Task Mate Kenyan Sign Language Classification Challenge
Can you classify words in Kenyan Sign Language?
Prize
$5 000 USD
Time
Ended 6 months ago
Participants
177 active · 503 enrolled
Why I am getting bad score on leaderboard despite of getting pretty good training accuracy.
Help · 31 Dec 2021, 17:11 · 5
Okay so I got training accuracy of around 99.98 percent which pretty good (neat to 100 percent). But when make my submission in given probability format I am getting score of around 2.197 on leaderboard. It will be appreciated if you could suggest me anthing, Or if you can guide me towards right my way
Well, first of all, Log loss is the metric stated by Zindi to be used here. Secondly, even though you chose to use "Accuracy" as your metric, still i could see an overffitted score of 99.98% of yours. I think the way forward is for you to set an "EarlyStopping" and at the same time use another metric entirely to monitor your chosen metric (say "Accuracy") while training your model.
Hope this helps.
i have this log loss function
metrics.log_loss(targets,outputs) where the outputs are the predicted ones and the targets the real values but i get an error which says that i should provide labels explicitly using labels parameter what might be the problem???
One important thing to check is the order of the classes during submission. The classes has to be in the same order as found in the sample submission. I remembered falling into this trap in my very early submissions.
How did you solve this problem?? Can you share a code snippet?
I think Training accuracy for small dataset like that is not that important - you overfit it easily. Look at your validation accuracy.