Catboost and Feed forward neural networks get you in the 0.027-0.028 range that is for me this was the case. Eventually Catboost trumped it. Make sure to run any of the NN algos on Google colab to use its higher GPU bandwidth. Otherwise it is very very taxing.
Lightgbm in dart mode had the best results according to cross validation. However, catboost had the best results according to public and private leaderboard. I should have put more weight to catboost in my ensemble to score higher. At first, I thought that my catboost model were overfitting to public leaderboard, but it wasn't the case.
So far Catboost.
Catboost and Feed forward neural networks get you in the 0.027-0.028 range that is for me this was the case. Eventually Catboost trumped it. Make sure to run any of the NN algos on Google colab to use its higher GPU bandwidth. Otherwise it is very very taxing.
You can also use kaggke privit notebook with GPU, it works well. The data is available there.
LightGBM ended up working the best for me, but it required a lot more tweaking than Catboost did.
Can you share your code. Would like to see your approach and learn from it.
5-fold LGB gave me the best result
Lightgbm in dart mode had the best results according to cross validation. However, catboost had the best results according to public and private leaderboard. I should have put more weight to catboost in my ensemble to score higher. At first, I thought that my catboost model were overfitting to public leaderboard, but it wasn't the case.
could your share please your params? for me catboost and xgb perform worse.
@lcfstat, just asking-how do you get to see how your model is peforming from the private leaderboard?
@ssshch { 'boosting_type': 'dart', "max_depth":-1, "num_leaves":32, 'learning_rate': 0.1,"min_child_samples": 20, 'feature_fraction': 0.8,"bagging_freq":1,'bagging_fraction': 0.9,"lambda_l1":1,"lambda_l2":1}
However, I think you should be carefull with the number of iterations because dart mode does not support early stopping rounds.
@elijah-a-w Once the competision has finished, all participants can see their private leaderboard scores in the submissions menu.
My approach was sum of lgbm multiclass and binary problems.
one hot of categorical features, tfidf and tuning lambda_l2 increased my score.(+/- 40 place on public and privat)