Our solution (Team Neural Beans with @100i and me) is conceptually simple - just a single object detection model. We fine-tuned a DINO model with a Swin Transformer-base backbone using the MMDetection library.

There are two main pre-trained versions of this model in mmdet: a version with a ResNet50 backbone and 4 scales of feature maps (DINO-4scale-R-50), and a more performant version with a Swin-Large backbone and 5 scales of feature maps (DINO-5scale-Swin-L). The DINO-5scale-Swin-L model is too big and slow given the resource restrictions of this challenge. We performed experiments with different backbones (including ConvNext (Tiny and Small), Swin (Small, Base and Large), and SwinV2 (Base)), 4 and 5 features scales, and different image sizes. Our best combination uses a Swin-Base backbone, 4 features scales and square 640x640 images.

Our training pipeline includes random horizontal and vertical flips, colour variations (using mmdet's YOLOXHSVRandomAug augmentation), and different image scales. Experiments with mosaic and mixup didn't improve the model. We utilised Exponential Moving Average (EMA) of weights during training which improved validation performance.

The model was trained for 12 epochs with a learning rate of 0.0001, with linear warmup over the first epoch, and cosine annealing beginning after the 6th epoch. The model was trained with mixed precision to reduce GPU memory usage.

Resources:

• Code repo: https://github.com/stefan027/zindi-competitions/tree/main/amini_cocoa_contamination
• Kaggle training and inference notebook: https://www.kaggle.com/code/stefan87/01-cocoa-contamination-dino-swin-b-4-scale/notebook?scriptVersionId=237234089
• Kaggle inference-only notebook: https://www.kaggle.com/code/stefan87/04-cocoa-contamination-inference
• Colab inference-only notebook: https://colab.research.google.com/drive/1K1mABa9Xe4NBGjOb9hF2DFhGN9yqCu23?usp=sharing