Hello Zindians,

This is the 2nd place solution for the advanced challenge of UmojaHack 2022.

So I tried different approaches but many seemed to fail (I think the baseline was not bad after all)

Key Points:

• For my validation strategy,I first split the signal in intervals (step =0.1) and split my data according to these intervals (stratification). I also tried grouping by ProteinId (Stratifiedgroupkfold) (seemed logical) but my cv and lb decreased so I dropped it.
• I tried to turn the problem into a classification problem (predict interval) but this method also failed.
• I tried introducing 1d CNNs after the embedding layers. This approach didn't improve the performance but I think it helps when blending.
• I made the architecture bigger (larger embeddings, more LSTMs ...)
• I didn't use only MSE loss, I also used Huber loss with different deltas. With huber loss we change the loss from MSE to other losses when the loss gets bigger than delta. This makes the model more robust to outliers and gives it more freedom to experiment more (double edged) since he will be penalized less.
• I used protbert instead of the embedding layers (for feature extraction) but It was so slow and I couldn't train for many epochs so I didn't use it.

Link : https://github.com/Mo5mami/UmojaHack-Africa-2022-African-Snake-Antivenom-Binding-Challenge