SFC PAYGo Solar Credit Repayment Competition
Helping Africa
$5 000 USD
Completed (over 4 years ago)
Prediction
1060 joined
275 active
Start
Jun 06, 21
Close
Aug 29, 21
Reveal
Aug 29, 21
CV vs LB
Data · 22 Jul 2021, 19:15 · 7
Trust we are all doing well and keeping safe. Please i want to know if your CV score is far from LB, I'm using KFold and my CV score is 701, when i submitted i got 666. Please if there's a different validation technique i can use to get close scores. Thank you. You can as well share your validation scores also.
this is quite interesting, I believe ts the same for most of us that the local scores are far from the LB,
But your scores gave me another mind blow, that the LB shows better score than CV.
My scores are quite the inverse, both tts and kfold give 620s locally, but the LB hits 680s. :))
Hello,
If you are having optimistic CV scores, you could be dealing with a leakage problem rather than the CV score represents true performance.
In your data preparation, if information about target is somehow leaked, that wouldn't be the case at test time, this can be the cause.
I'll say check your data preparation again and identify a distribution in your CV that isn't the case at test time. Your CV is more reliable when it's distribution represents how its expected at test time.
Yes exactly. That's just the normal way one should look at this. Thus, we shouldn't be surprised if unexpected happens in the final standing of Private Leaderboard scores at the end of the challenge.
Thank you Damola, Doe it means i'm on track? Please what's your cv score like.
Hello,
My CV score is around 687 currently, I think we should just check that the CV & LB are moving in agreement.
Also, ensure there is no leakage or bias in your CV evaluation. Till we see private LB, to know what's really on track.
Thank you for your contribution
Cheers! All the best.