GeoAI Aquaculture Pond Identification Challenge by FAO and ITU
Hi all,
Based on the latest information, we should expect all entries to have 4,5, or 6 continuous months with valid data. For the other months, all values should be -9999, This is not the case.
Several months have mixed entries, where the Sentinel 1 and Sentinel 2 values are inconsistent. In all cases, the Sentinel 2 values appear to be missing, leaving us with less data than expected (e.g. 2 valid months instead of 4,5, or 6).
Key Findings:
- Total monthly observations: 1030 rows × 12 months = 12,360 month‑row entries.
- Inconsistent entries (mix of present/missing features): 320 (≈2.6% of total).
- Breakdown of inconsistencies:In every inconsistent case, the 10 non‑VH/VV features are uniform (all missing). The VV and VH features are always consistent with each other (both present) and have the opposite status to the other ten features.
- Monthly distribution of inconsistencies:
Month 01: 1 row (0.1%) Month 02: 38 rows (3.7%) Month 03: 3 rows (0.3%) Month 04: 1 row (0.1%) Month 05: 1 row (0.1%) Month 06: 75 rows (7.3%) Month 07: 1 row (0.1%) Month 08: 9 rows (0.9%) Month 09: 3 rows (0.3%) Month 10: 181 rows (17.6%) Month 11: 7 rows (0.7%) Month 12: 0 rows (0.0%)
For some reason, this impact mostly the month of October.
If the intent was to provide valid contiunuous blocks of data for both Sentinel 1 and 2 for 4, 5, or 6 months, then the Test set should be corrected. If the intent was to provide a mix of valid results, then it should be documented in the description of the project.
Let me know if you can reproduce this finding.
Thanks for raising this - you are correct. Across the test set, 320 month-row entries (2.6%) have S1 data (VH/VV) present but S2 data missing, affecting 273 of the 1,030 test rows. In every case, the S1 radar bands (VH/VV) are present while the S2 optical bands are missing - never the reverse - which is consistent with cloud cover blocking optical acquisition, and the issue is concentrated in October (181 cases), June (75), and February (38). For affected rows, the number of fully-consistent months drops to 2–5 instead of the intended 4–6.
This reflects genuine data availability conditions in the source satellite imagery.
We appreciate you raising this - it's a valid observation and exactly the kind of real-world complexity this challenge is designed to test.
Hi! When you say "This reflects genuine data availability conditions in the source satellite imagery", does this mean the competition is masking this data to simulate availability conditions in the real world, or are the sensor reading in the test data the only readings that were available? Is there any insight you can give us on how/why the test data is masked the way it is? Was it due to real world conditions preventing the collection of data during certain periods of time?