How did that approach perform?

so poor that I didn't make a submission. couldn't even detect buildings in most cases.

For me , the bda tries in localization but overfits quite easily. The classifcation part is where the problem is at even though the f1 validation scores suggest otherwise. For example:

lofF1 is 0.7892465047263282, clfF1 is 0.03334201235710739, oaF1 is 0.2601133600678736, sub class F1 score is [0.74111625 0.00988438 0.06209535 0.74323876]

where the lofF1 is the localization F1, clfF1 Is the classification f1. oaF1 is the harmonized f1 , the array represents individual classes f1

You can see the classsification f1 is not good but the localization f1 its trying

in my unet case, both localization and classification F1 were quite high on xview data > 0.85 for loc and >0.74 for classification. used xview test for validation and the rest for training (did not touch the holdout set). but using the model on Malawi dataset proved to be a challenge. couldn't detect buildings for the most part. Trying an object detection approach now.

My best sub is a tuned version of the starter notebook. even then can't really rely on cv there, as a low validation mae only translates to LB improvements up to some point. maybe strong augmentations are required.

Sample inference result:

Real Image

Predicted localization map

predicted classification map

I still feel there is hope in this approach, and I think I have a bug somewhere but the reported results in their paper indicates that we should get better scores using this approach. It is infact the SOTA model in change detection tasks. Anyways doing last experiments, if it does not show any improvement I will have to change to object deteection. One question @nymfree how would you have approached counting the buildings? Currently I am "Labelling connected regions of an integer array" using skimage.label Do you have any other idea of going about it? Thanks

using skimage.label is fine. in my unet approach I had the model predict 4 masks - 0ne for each category. I wonder if doing it like that might help with your classification accuracy.

Okay thanks