CV score using LGBMRegressor and Catboost Regressor gives 3.67 and 3.62 respectively, but after stacking and ensembling score LB score is 4.18... how were you able to get 3.0 on the LB? what was your CV score?
I think the difference is on your features engineering. Algorithms doesn't matter than much, so you need to create new features that can give more signals to your model. Think of combining certain features and see if that can increase the score.
After doing that you can also do hyperparameter turning to get the best possible parameters of your model you can use Optuna which is great. I hope this can be useful for you.
What was your approach to feature engineering? I did feature engineering which improved my CV score on train test split from 3.28 to 2.85 on LGBMREGRESSOR and 2.79 on CatBoostRegressor... What was your CV score?
'%3e%3cpath fill='%23001489' d='M0 0v6h9V0z'/%3e%3cpath fill='%23e03c31' d='M0 0v3h9V0z'/%3e%3cg stroke='%23fff' stroke-width='2'%3e%3cpath id='d' d='m0 0 4.5 3L0 6m4.5-3H9'/%3e%3cuse xlink:href='%23b' stroke='%23ffb81c' clip-path='url(%23c)'/%3e%3c/g%3e%3cuse xlink:href='%23d' fill='none' stroke='%23007749' stroke-width='1.2'/%3e%3c/g%3e%3c/svg%3e)
I think the difference is on your features engineering. Algorithms doesn't matter than much, so you need to create new features that can give more signals to your model. Think of combining certain features and see if that can increase the score.
After doing that you can also do hyperparameter turning to get the best possible parameters of your model you can use Optuna which is great. I hope this can be useful for you.
What was your approach to feature engineering? I did feature engineering which improved my CV score on train test split from 3.28 to 2.85 on LGBMREGRESSOR and 2.79 on CatBoostRegressor... What was your CV score?