Is there a way to find an equivalence between leaderboard & cv ? Leaderboard is giving 475 , CV is giving 500 000 even on smaller folds than the test's length. how is that possible? I'm using mean_squared_error and r² as metrics.
still getting 20 seconds of difference between lb/cv scores. I just wanna make sure, is it normal in case train/test are a bit different, or do i need to make a better split to have coherent cv/lb scores?
Hey DrFad, i'm using a tree model. I gave up on trying to create a perfect split to match what's the in test set. I'm trusting my cv's direction now. ( Also strangely some features disturb the difference even more )
'/%3e%3c/defs%3e%3cpath fill='%23fff' d='M-120-80h240V80h-240z'/%3e%3cpath d='M-120-80h240v48h-240z'/%3e%3cpath fill='%23060' d='M-120 32h240v48h-240z'/%3e%3cg id='b'%3e%3cuse xlink:href='%23a' stroke='%23000'/%3e%3cuse xlink:href='%23a' fill='%23fff'/%3e%3c/g%3e%3cuse xlink:href='%23b' transform='scale(-1 1)'/%3e%3cpath fill='%23b00' d='M-120-24v48h101c3 8 13 24 19 24s16-16 19-24h101v-48H19C16-32 6-48 0-48s-16 16-19 24z'/%3e%3cpath id='c' d='M19 24c3-8 5-16 5-24s-2-16-5-24c-3 8-5 16-5 24s2 16 5 24'/%3e%3cuse xlink:href='%23c' transform='scale(-1 1)'/%3e%3cg fill='%23fff'%3e%3cellipse rx='4' ry='6'/%3e%3cpath id='d' d='M1 5.85s4 8 4 21-4 21-4 21z'/%3e%3cuse xlink:href='%23d' transform='scale(-1)'/%3e%3cuse xlink:href='%23d' transform='scale(-1 1)'/%3e%3cuse xlink:href='%23d' transform='scale(1 -1)'/%3e%3c/g%3e%3c/svg%3e)
"The error metric for this competition is the Root Mean Squared Error", so may be you should calc root from 500000?
Yeah forgot to add the sqrt lol. Thanks a lot!
still getting 20 seconds of difference between lb/cv scores. I just wanna make sure, is it normal in case train/test are a bit different, or do i need to make a better split to have coherent cv/lb scores?
It depends. Are you using a linear model or a tree model?
Hey DrFad, i'm using a tree model. I gave up on trying to create a perfect split to match what's the in test set. I'm trusting my cv's direction now. ( Also strangely some features disturb the difference even more )
Wise choice in trusting your cv score.
Followed the advice of the more experimented data scientists ;)