CGIAR Eyes on the Ground Challenge
Helping Africa
$10 000 USD
Completed (over 2 years ago)
Prediction
869 joined
137 active
Start
Jul 21, 23
Close
Nov 03, 23
Reveal
Nov 03, 23
LB & "type of damage" column
Help · 22 Sep 2023, 07:32 · 8
@Zindi it's clear now that we're not allowed to use the "type of damage" column. But how we could know if we can trust the LB to monitor our progress and see if our model behaves well in comprasion with others or we need to change our approach. At least if the top rankers can confirm that their approaches don't include the "type of damage" column to see if we're far away or not :)
There has been no official announcement on whether we can use the "type of damage" column or not. The competition rules do not mention this and do not prohibit it. I think using it is natural :)
It is. You can follow up this discussion https://zindi.africa/competitions/cgiar-eyes-on-the-ground-challenge/discussions/17792
Thank you. I'll check it out.
I fear now that people are using this information (type of damage) to get better results in LB. I would suggest the competition hosts remove this column from the datasets and reset the LB.
But people will still have it :( at least should reset the LB and provide new dfs with new image ids
Yeah, I agree with resetting the LB. But for the new ids, I think it might be a bit more challenging, since it will involve renaming all the images as well ( if we don't want people to easily rematch the new df to the old one)
With 1 month to go, It would be great to know where @Zindi stands. Because this might be messy once the competition ends and they have to go through the Top10-20 solutions, to eventually find out that most of them use the DR tag.
My best guess is that it is intentionnal,
when you look at the image where the type of damage isn't DR, you can't still see some drought damages and yet the extent is labelled 0.