Our solution is just a single xgboost with some feature engineering and a custom loss function (Huber loss). This was our best rmse in cv (470) and worse in public lb (423) but best in private (106).
Totally agree - it's mostly not always worth focusing on the public LB. CV never lie. Also had my best scoring single model (XGB) achieve 100+ on private but very poorly on public (357)
Yeah, got duped by the public LB scores. My earlier work was far simpler, but scored 300s. So I preferred my later work that scored sub 200.
I don't entirely understand how, if you look at the top ten, you have private scores of 100 with public scores varying from 100 to 500. That implies that the 400+ public score solutions did great on 80% of the test set and poorly on the public 20% of the test set. While the <150 public scores did great on 100% of the test set?
I totally gree.Trust your CV.I must say I was surprised at the private leaderboard score after having a CV score of 435
Totally agree - it's mostly not always worth focusing on the public LB. CV never lie. Also had my best scoring single model (XGB) achieve 100+ on private but very poorly on public (357)
Yeah, got duped by the public LB scores. My earlier work was far simpler, but scored 300s. So I preferred my later work that scored sub 200.
I don't entirely understand how, if you look at the top ten, you have private scores of 100 with public scores varying from 100 to 500. That implies that the 400+ public score solutions did great on 80% of the test set and poorly on the public 20% of the test set. While the <150 public scores did great on 100% of the test set?