This is one competition that was quite tough and a little frustrating to me atleast.. I must say, your discoveries and solutions were nothing short of genius.Your team did very well and this is no mean feat. Congratulations and thank you for always sharing.

Thanks for leading the way by sharing, and with a great approach too.

With a mix of data sources merged and very limited rows, I found carefully treating the data types (ensure categorical features like id_facility were actually set to categorical) and null filling to be important (e.g. observe_total had missing values but the values used to calculate the total were available so filling with the sum of attentive, concentrated etc. helped clean it up).

Survey/assessment data often has ordinal nature to it, so encoding those logically (strongly disagree = 1, strongly agree = 5) seemed to help on my end.

Summary stat features by enumerator, facility, ward were definitely helpful as you show here (believe too much of this caused overfitting on my part). Other features for me were things like number of null values in each row, binary feature to indicate if child spoke more than 1 language, difference from capetown longitude, binary feature to indicate if text is present in open-response features, etc.

In the spirit of sharing, here is a simplified version of my approach: https://github.com/fitzpk/zindi_repo/blob/main/early_learning_predictors/catboost_starter.py

Did feature selection make an impactful difference for your model? Also a max_depth=16, limited rows and no regularization parameters to help prevent overfitting? Love it. Definitely a takeaway for me.

Hoping we see some of the other top performers share like you!

Hey all, thanks for sharing! My solution is quite similiar, though I did not do so much feature engineering. It ranked 6th and the link for the github repo is below. I'd agree with the conclusions brought up by the others, dropping columns with too many nulls, categoricals, ordinal rankings etc. and would add that tuning hat less of an impact than thought, probably by the nature of highly noisy data.

Regarding the sensitivity pipeline: its tricky, I think one could come close with counterfactual explanations (i.e. like DiCE) when one sets the constraints to the 'important' features. Though there is no reason that the global importance features are the best explaining individual importancy of features. And if changes are done to one globally important feature, there is no guarantee one actually reaches a different outcome class as desired by the specification.

Link to repo: https://github.com/danielseussler/zindi-earlylearning-py

Great Job Reacher and Koleshjr.