I have a serious concern. Due to the nature of the data split, we will not be able to get correct estimate of the performance of the model. In simple words the leaderboard claim that the RMSE of the predicted values using cheaper devices to predict PM2_5 will be around 8 is not at all close to reality.

For the same day in the data we have some hour in train, and some hour in test set.

A simple example is even if you aggregate the pm2_5 each day you are basically using future values to predict the past, and the model will easily exploit this information to give you good scores. This will be a powerful feature in your model's feature importances but this information will not be helpful in real usecase of Airqo. The data split should have been past-future to give a correct estimate.

At the end of the competition everyone's models will be using future data knowingly or unknowingly and hence will be bad estimates of the model performance in reality. Generally for any competition where a time component is involved, we should think about how will we use the model in production. In this case I will want my model to run for future days using past data, so the competition data split should simulate it.

Good Luck to everyone !