Primary competition visual

Task Mate Kenyan Sign Language Classification Challenge

Helping Kenya
$5 000 USD
Completed (~4 years ago)
Classification
Computer Vision
588 joined
177 active
Starti
Nov 07, 21
Closei
Feb 27, 22
Reveali
Feb 27, 22
Why I am getting bad score on leaderboard despite of getting pretty good training accuracy.
Help Ā· 31 Dec 2021, 17:11 Ā· 5

Okay so I got training accuracy of around 99.98 percent which pretty good (neat to 100 percent). But when make my submission in given probability format I am getting score of around 2.197 on leaderboard. It will be appreciated if you could suggest me anthing, Or if you can guide me towards right my way

Discussion 5 answers
User avatar
MICADEE
LAHASCOM

Well, first of all, Log loss is the metric stated by Zindi to be used here. Secondly, even though you chose to use "Accuracy" as your metric, still i could see an overffitted score of 99.98% of yours. I think the way forward is for you to set an "EarlyStopping" and at the same time use another metric entirely to monitor your chosen metric (say "Accuracy") while training your model.

Hope this helps.

User avatar
Koleshjr
Multimedia university of kenya

i have this log loss function

metrics.log_loss(targets,outputs) where the outputs are the predicted ones and the targets the real values but i get an error which says that i should provide labels explicitly using labels parameter what might be the problem???

User avatar
HungryLearner

One important thing to check is the order of the classes during submission. The classes has to be in the same order as found in the sample submission. I remembered falling into this trap in my very early submissions.

User avatar
Koleshjr
Multimedia university of kenya

How did you solve this problem?? Can you share a code snippet?

I think Training accuracy for small dataset like that is not that important - you overfit it easily. Look at your validation accuracy.

3 Jan 2022, 12:40
Upvotes 0