Sea Turtle Rescue: Error Detection Challenge
'/%3e%3c/defs%3e%3cpath fill='%23fff' d='M-120-80h240V80h-240z'/%3e%3cpath d='M-120-80h240v48h-240z'/%3e%3cpath fill='%23060' d='M-120 32h240v48h-240z'/%3e%3cg id='b'%3e%3cuse xlink:href='%23a' stroke='%23000'/%3e%3cuse xlink:href='%23a' fill='%23fff'/%3e%3c/g%3e%3cuse xlink:href='%23b' transform='scale(-1 1)'/%3e%3cpath fill='%23b00' d='M-120-24v48h101c3 8 13 24 19 24s16-16 19-24h101v-48H19C16-32 6-48 0-48s-16 16-19 24z'/%3e%3cpath id='c' d='M19 24c3-8 5-16 5-24s-2-16-5-24c-3 8-5 16-5 24s2 16 5 24'/%3e%3cuse xlink:href='%23c' transform='scale(-1 1)'/%3e%3cg fill='%23fff'%3e%3cellipse rx='4' ry='6'/%3e%3cpath id='d' d='M1 5.85s4 8 4 21-4 21-4 21z'/%3e%3cuse xlink:href='%23d' transform='scale(-1)'/%3e%3cuse xlink:href='%23d' transform='scale(-1 1)'/%3e%3cuse xlink:href='%23d' transform='scale(1 -1)'/%3e%3c/g%3e%3c/svg%3e)
Helping Kenya
$1 950 USD
Challenge completed over 6 years ago
Classification
Anomaly Detection
321 joined
51 active
Start
Nov 30, 18
Close
Apr 28, 19
Reveal
Apr 29, 19
Next Topic: Cross-Validation
Help · 13 Mar 2019, 10:05 · 4
Did anyone experience huge discrepancy between local cross-validation score and leaderboard score?
I'm doing 5 folds k-fold crossvalidation, here's what I've been getting in terms of CV vs LB scores: 0.029 -> 0.081
0.027 -> 0.067
0,049 -> 0,058
I think you mean 0.019 not 0.049 ?
No typo there: 0.049-something. Correctly building the target variable is (IMO shouldn't be) the first step in solving the problem.
Finding a proper cross-validation process is the next part of the problem. Hint: K-fold cross-validation might not be the fittest here due to the temporal dimension in the dataset.
One thing i forgot is that there might be no correlation between CV and LB score which might make it difficult to build a model that generalises well
Short Message is that its hard to trust your local CV since Zindi competitions always have 50/50 on Public leaderboard, if its was say 30% on Public Leaderboard then you would highly trust your Local CV. Take the message for competition purpose only. thanks