Hi! I was thinking that for those interested. It would be nice to share ideas on what has been working and what does not. Now that the validation seems closer to the leaderboard I'm really interested in learning the best methods to tackle this challenge. Maybe we can build on top of each other.

At present, I tried the following:

- Marginal Loss (ArcFace) (https://arxiv.org/abs/1801.07698) with efficientnet-b3 (300x300) encoder and autoaugment(imagenet). I managed to have performances around ~0.8. The optimizer is AdamW with OneCycleLR

- For this one, I've done some tunning, in particular with the loss parameters and epochs. What I have not varied much is the data augmentation.

- I also tried a variation where I do final preprocessing extracting the edges from images. My intuition is that models might have it easier if they only have to concentrate on the topology of the plastrons. However, when doing that in the arcface training the performance is worse (around 0.6 - 0.7).

- Another approach I tried was first training a supervised contrastive loss (https://arxiv.org/pdf/2004.11362.pdf) and then using it as a feature extractor for a classifier model. I have not managed to make this one work, evaluating both models have been difficult since for example, I don't know if simply best contrastive loss implies best classifier performance.

- For the contrastive approach, the max batch size I have been able to fit was 64 with a smaller efficientnet(b0 224x224). I understand that adding more samples to the batch might help increase the differentiation between the classes.

This is at a high level what I've been working on. I can share some of my nbs if that's ok with the rules but I don't know. If you have any ideas and would like to discuss them further please let me know. I'd love to learn how to do this right :)