Many thanks to all hosts and Zindi team for such an interesting challenge. Congrats and thanks to all participants!

Despite the simple formulation the task is really difficult because some classes have close semantics and small number of training examples. My final solution is an ensemble of 6 MT5 (L, XL) models trained on different sequence lengths from 64 to 256 tokens. Each of 6 models is 5-fold self-ensemble. I wasn’t aware of any other models pretrained on Chichewa language so from the beginning concentrated on MT5. My cross-validation setup is based on 5-fold stratified split. CV score of my best ensemble was 0.7005 (pretty close to the private LB 0.7097) but its public score was only 0.6419 so choosing final submissions is this competition was a bit tricky.

One interesting conclusion from my experiments is that good model can be trained on relatively short sequence (even though almost all texts are quite long). In particular one of my best single models was trained on 64 tokens. Models trained on 384 and 512 tokens were not better. Also I trained some models on different ranges of tokens like [0:256), [256:512), etc. All ranges except the 1st one gave much lower CV score.