-This was a very interesting and exciting challenge - huge thanks to @Zindi and the hosts led by @Nevenka for presenting such a research rich area. It was my first time diving into a biology-related machine learning application.

-I also encountered MPEG for the first time of course, I’d heard of MP4, but not MPEG in the context of genomic data. It was fascinating to see how powerful the MPEG-G format is for data compression and transfer - over 200gb fastq data compressed to only abot 20gb MPEG-G files!

-Another first for me in this competition was exploring federated learning which opened up a new world of possibilities for privacy-preserving machine learning. This is exactly what makes Zindi such a great platform for emerging African talent in ML and AI - you get to explore cutting-edge ideas in real-world applications.

From my submissions and experiments:

1. The public leaderboard felt a bit unstable - my local cross-validation performance averaged around a logloss of 0.03, while the public leaderboard often showed much smaller (and inconsistent) scores -sometimes in the 0.0000x range!

2. Neural networks performed better than tree models on the private leaderboard reveal compared to best tree model (Xgboost)- 0.015 vs 0.018.

3. I tested DNA-BERT, an LLM-based approach for genomic sequences; however, its predictive performance was notably poor Public-0.37 and Private-0.28, even though feature extraction was notably multiple times faster than tree model feature extraction which took me a whooping 6hrs to generate only about 94 features!

5. Federated learning using XGBoost shows promise, outperforming centralized models on the private leaderboard (0.0152 vs 0.0018 on public).

👉 Check out my solution on this repo: https://github.com/JuliusFx131/cyclic-federated-learning - where I demonstrate cyclic federated learning, a setup that doesn’t require a central server like traditional frameworks such as Flower or PySyft. You can run the notebooks directly on kaggle now-thanks to @MuhammadQasimShabbeer who took the pain of uploading over 200gb sized fastq files on kaggle!

6. K-mer frequencies (di- and tri-nucleotides) emerged as the most important features - aligning with the idea that genomic composition biases can act as biomarkers for microbial community origins.

Overall, it was a challenging but deeply rewarding experience to all participants. Cheers to @Amy_Bray and the whole @Zindi team for patience to run very heavy feature extraction pipelines over many hours.