I keep having local rmse between 9.3 but the submission score is giving me 12. I have tried to check every possible error from my end but can't find any. Please zindi, help check the scoring again. Other people have raised similar issues earlier too.
Can you do a sense check please and open the file you submitted and see if the Target values are wildly off what you would assume?
We have noted that the scores shown on the leaderboard and under the submissions tab do not show exponents. What might be happening is your submission 9.3 is actually 9.345678123e10. At first glance, it looks great but the e10 probably makes it one of your worst submissions.
Dev is looking into how to show exponentials for scores.
Also, please note that your local score will always be different to the Zindi score as Zindi keeps the test target on our backend.
This issue was raised in this post https://zindi.africa/competitions/datadrive2030-early-learning-predictors-challenge/discussions/16389
I also still believe, something is not right. I do not believe the best scores have not yet crossed the rmse-8 territories. The scores change marginally no matter the count of submissions made. Top submissions at 9.2 since the competition began, no matter how much the guys try..??
We either expect a big shake up when competition closes or we have a suspect competition. We wait..
some of us are using wrong validation scheme and some are overfitting their model on local CV.