Solution Overview

First, I want to thank the Barbados Lands and Surveys Department for this incredible real-world challenge, Zindi Africa for hosting the competition, and all the participants who made this such an exciting competition.

My approach combined Vision-Language Models (VLMs) with deep learning segmentation to extract both geometries and metadata from analog survey plans. The key innovation was using VLMs not just for OCR, but as reasoning engines to solve spatial alignment problems.

The solution has two parallel pipelines:

1. Segmentation Pipeline:

Geometry Alignment: The training data already contains geographic coordinate shapes of land parcels, but they need to be aligned to pixel space. I used Qwen3-VL-30B (32B) by providing it with both the full survey plan map and an image of the geo coords polygon shape—the model intelligently found the corresponding pixel locations of the parcel boundaries

Model: Unet++ with EfficientNet-B5 encoder

Novel Approach: Surveyor bias model - learned per-surveyor embeddings to capture individual naming conventions and geometry patterns

Inference: 8-way TTA with IoU-based ensemble selection (picked polygons with highest consistency across augmentations)

2. Text Extraction Pipeline

Model: Fine-tuned Qwen3-VL-8B using Unsloth LoRA

Image Patchification: Split each image into 7 overlapping crops (1024×1024) to improve VLM focus

Automated Label Correction: Used VLM to audit and fix noisy training labels—the model takes in the raw labels and image, then returns corrected full names, addresses, and other metadata details before fine-tuning

Key Techniques

• 8-way TTA with IoU-based ensemble selection (not simple averaging)

• Patchifying images into 7 overlapping crops focused VLM attention on relevant regions

• Douglas-Peucker smoothing on extracted polygons reduced noise while preserving shape

• Mixed precision training (16-bit) for efficient 2048×2048 segmentation

• Surveyor bias embeddings captured domain-specific patterns (middle name conventions, preferred land mappings...)

Git Code Repo - If you found this solution helpful, please consider giving it a ⭐ on GitHub!