Primary competition visual

Barbados Lands and Surveys Plot Automation Challenge

Helping Barbados
$10 000 USD
Completed (5 months ago)
Computer Vision
Geospatial Data
Optical Character Recognition
895 joined
179 active
Starti
Aug 01, 25
Closei
Oct 19, 25
Reveali
Oct 20, 25
User avatar
Mohamed_abdelrazik
Score Against Random Labeling
Data · 17 Oct 2025, 15:10 · 19

After nearly three months of hard work, experimenting with different algorithms, testing model variations, and carefully examining every aspect of the dataset, I’ve reached a very frustrating realization. It seems that the competition submissions are being validated against incorrect human-labeled data — in some cases, the errors appear to be completely random.For example, image BYLS-107, which also appears more than 50 times across the training set, has a completely wrong mask. This is not a borderline labeling issue — it’s simply incorrect. What’s more concerning is that there has been no official acknowledgment or clarification from the organizers about data validation steps, or whether similar issues exist in the public or private test sets.As participants, we’ve all invested significant time fine-tuning our pipelines, optimizing architectures, and exploring creative approaches to improve generalization. But if the evaluation is based on mislabeled or inconsistent ground truth, then the outcome is largely a matter of luck — not modeling skill or data understanding. Models that happen to mimic the labeling errors might rank higher, while those that genuinely learn meaningful patterns could perform worse.This undermines the spirit of the competition and the credibility of its results. I genuinely hope the organizers can address this issue, provide clarification on data quality and verification, and let us know whether the same labeling problems exist in the test data used for scoring.Because without reliable ground truth, machine learning and computer vision approaches become meaningless — no model can learn from noise.

Discussion 19 answers
User avatar
CodeJoe

Mohammed most of them are not wrong. It is the same land with different site plans likely due to valuation purposes (I can lease a land from my land even though it is the same land- valuation is done differently for the leased land meaning different site plan.). The recent ones are most likely the ones found in the images. Amy also confirmed that there is no mistake in the test set so let's take her for a word.

Good luck big man!

17 Oct 2025, 15:21
Upvotes 0
User avatar
Mohamed_abdelrazik

did you check the original polygon for BYLS-107 and plot it with the original image you will see the mask is not aligned it's almost 90 degree rotation . base don my understanding the polygon and mask should be overlapped

User avatar
CodeJoe

I did for almost all and was confused at first. But when I visualized it on the map it actually made sense. All those polygons fit in the Bare land perfectly. I then realized most of the polygons were from different site plans.

There might be one or two I overlooked but it is not really significant to drop your score to low.

User avatar
CodeJoe

Have you tried flipping the y-axis again before visualizing?

User avatar
Joseph_gitau
African center for data science and analytics

it's not all about flipping.

I did inference on all training set images and there are those samples you can never get right even though my model is ok.

User avatar
Mohamed_abdelrazik

i totally agree with you because i did the same thing i infere on training data and the mask perfectly fit the polygons but evaluating the iou between predicted masks and ground truth gives you low iou

User avatar
Joseph_gitau
African center for data science and analytics

Also this second plot shows how real polygon masks are imperfect if you cross check with actual images you will note all the different irregularites present.

User avatar
Mohamed_abdelrazik

thank you @Joseph_gitau for you info and confirmation for the same point i mentioned may i ask you if you don't mind plot BYLS-107 specially because this one has totally wrong mask just confirmation thank you again

User avatar
CodeJoe

I see, this is really different.

User avatar
CodeJoe

Well if Amy says everything is correct, we just have to take her words for it. I mean like people will not be in 97and 96 polygon scores

User avatar
Joseph_gitau
African center for data science and analytics

They would be even with wrong polygons

User avatar
Brainiac

I think the issue stems from using the image pixels to get the gt polygons, note that the polygons provided are in geo coords, so there will be some small disrepancies - as it is not a 1 to 1 mapping(pixel poly vs geo coords poly)- this could explain the 0.95 iou, but never 1. but if you use bearings and distances to get the gt polygons, most of the polygons are correct, though there is still a number of images with wrong polgons.

This are all my assumptions and made up theories 😂

User avatar
Joseph_gitau
African center for data science and analytics

This is what I have

User avatar
CodeJoe

Exactly, this was my point @Brainiac.

User avatar
Brainiac

This is where domain knowledge could have worked wonders

User avatar
Mohamed_abdelrazik

Hi @Brainiac i spent all the day working on bearing and distances to get the gt masks i use affine transfoermation and georefernce points but same issue i know this is the tight time but can you explain a methodology or piepline to use bearing and distance to get exact gt polygons for images with correct shapes , thanks

User avatar
Brainiac

Hi @Mohamed_abdelrazik — totally get where you’re coming from. I went down the same rabbit hole with cadastral maps; it took me a while (papers, articles, YouTube deep dives) to really grasp how bearings/distances tie back to ground truth. It’s a new domain, so one day isn’t realistic—don’t be hard on yourself.

Here’s the pipeline I use for bearings + distances:

  1. Locate the EN point in the image Pick the parcel’s reference point “EN” in the image as the origin for the walk.
  2. Walk clockwise from EN with bearings & distances Read each boundary leg in order (clockwise). For each leg, use its bearing and distance to get the next vertex. Keep stepping around the parcel until you return to EN.
  3. Get the polygon coords from the provided function Feed the ordered vertex list (starting at EN, clockwise, closed) into the provided bearings_to_en_coords function to generate the mask/polygon with the correct shape.

Example image:

import math
from typing import List, Tuple
import numpy as np
import matplotlib.pyplot as plt

Bearing = Tuple[float, float, float]  # (deg, minutes, distance)
EN = Tuple[float, float]  # (E, N)

def bearings_to_en_coords(
    bearings: List[Bearing],
    start_en: EN,
    use_back: bool = True,
    shift: int = 0,
    reverse: bool = False,
    bowditch: bool = True,
    close: bool = True,
) -> List[EN]:
    if reverse:
        bearings = list(reversed(bearings))
    if shift:
        k = shift % len(bearings)
        bearings = bearings[k:] + bearings[:k]

    de, dn, Ls = [], [], []
    for d, m, L in bearings:
        az = (d + m / 60.0) % 360.0
        if use_back:
            az = (az + 180.0) % 360.0
        r = math.radians(az)
        de.append(L * math.sin(r))
        dn.append(L * math.cos(r))
        Ls.append(L)

    de = np.asarray(de, float)
    dn = np.asarray(dn, float)
    Ls = np.asarray(Ls, float)

    if bowditch and Ls.sum() > 0:
        ce, cn = de.sum(), dn.sum()
        w = Ls / Ls.sum()
        de -= ce * w
        dn -= cn * w

    x = np.r_[0.0, de.cumsum()]
    y = np.r_[0.0, dn.cumsum()]
    E0, N0 = start_en
    pts = [(E0 + float(xi), N0 + float(yi)) for xi, yi in zip(x, y)]
    if close and pts[-1] != pts[0]:
        pts.append(pts[0])
    return pts
def plot_coords(coords: List[EN], title: str = "Converted EN Polygon"):
    if not coords:
        raise ValueError("Empty coords.")
    xs = [p[0] for p in coords]
    ys = [p[1] for p in coords]
    plt.figure(figsize=(6, 6))
    plt.plot(xs, ys, "-o")
    ax = plt.gca()
    ax.set_aspect("equal", adjustable="box")
    ax.grid(True, ls="--", alpha=0.3)
    ax.set_xlabel("Easting")
    ax.set_ylabel("Northing")
    ax.set_title(title)
    plt.tight_layout()
    plt.show()
USE_BACK = True
bearings = [
    (45, 27, 6.10),
    (10, 0, 4.69),
    (6, 43, 3.96),
    (96, 42, 24.56),
    (135, 37, 20.67),
    (225, 42, 28.32),
    (315, 29, 30.51),]
start_en = (40836.55, 74893.59)
coords_en = bearings_to_en_coords(
    bearings, start_en, use_back=USE_BACK, bowditch=True, close=True
)
plot_coords(
    coords_en, title=f"Converted EN ({'Back' if USE_BACK else 'Forward'} bearings)"
)



User avatar
CodeJoe

@Brainiac Thank you Big man🙇

User avatar
Mohamed_abdelrazik

@Brainiac THank you so much for your time and help bro