I don't think an ensemble is a wise decision given the final metric is:

Final score will be calculated by: zindi_score = (PrivateLB_score + (1-(size/PARAM_SIZE))*PrivateLB_score )/2

So

• If an ensemble consists of multiple models, its overall size becomes the sum of the sizes of the individual models.
• This means the size term in the formula increases, which increases the penalty.
• If the ensemble consists of n models, and each model has a size 128mb for example, then the total ensemble size is approximately n * 128
• If size grows too large, the term 1−size/PARAM_SIZE size​ shrinks, reducing the final score..