Dear @AntonioDeDomenico ,

I have a genuine concern regarding the test data. We are suppose to predict the impact of fault on the data rate. But I have few concerns regarding the same:

1. All the KPI values for that row are Nan. We may use different techniques to fill those values (ex: mean, ffill etc). But we are not providing the model with KPI values specific to that row. In addition, If we impute those rows using mean for example....we are implicitly saying that there is NO impact on the specific KPI when the fault occured as there are many rows where fault < 0.

2. There are some files where the sample period (1 Hr) is not uniform. Ex: Test - 'B0017-32_1.csv.csv'. There is approx. 9 hr gap between Non-fault and fault instance. Which again suggest we can't simply use FFill or Mean as circumstances might have changed.

I think due to the above issues I am currently getting different f1_score values on the local validation and leaderborad validation (CV : 0.8 and LB : 0.4 - 0.6). Without addressing the Nan values I think the trained model may not be useful.

Possible solution:

Keep all the KPI values in the test set and REMOVE the data_rate column (It also helps in addressing any leaks). This way we know the characteristic of the NE when the fault occured and will be able to generalize well.

I may misundestood the objective. Please correct me if I am wrong.

Thank you for your patience.