In this case, I use cross-validation with 5 folds, generating the submission with an average of 5 folds. Will it count as 1 model or an ensemble of 5 models?

I think that it would count as 5 separate models, since they would be trained from different sets of data (even though they are not necessarily disjoint).

Well, technically, it is indeed considered as 5 separate models. However, I believe that this competition considers this technique as one "single model" since it is the result of a single training process

@amyflorida626 I think it's essential for the competition host to provide a concise explanation of what is considered a single model to ensure fairness in the competition.

I actually don't think they would be considered a single model instance in this case, since they are instantiated and trained from scratch in each fold. If that is the case, then the wording of the rules would have been something like "maximum 3 different model *architectures*", not "maximum of 3 models" (model *instances* are implied here).

Totally agree. We need a more precise explanation.

@amyflorida626 Could the Zindi team gives us a definitive direction on this soon? This impacts how we would do the experiments quite a lot.

You haven't answered all of the concerns that we've raised. Semantically, we can classify a weak learner such as a decision tree as a singular model instance even though the implementation for scikit-learn's sklearn.ensemble.RandomForestClassifier is an individual class that involves *many* trees (in a module called "ensemble" mind you).

Plus, I don't think that would solve the concern that you have, since you're still allowing a maximum of 15 model instances to contribute to submission.csv (3 models trained on 5 folds each). An alternative I think can be considered in terms of inference time of a fixed batch, size in kilobytes or some other metrics that favours low compute availability/usage.

We understand your goals and are trying to contribute to the competition as well. Hope you understand where we're coming from.

Some references in how other competitions have done this

• Both of these competitions specify an "efficiency" track that checks models that simply run the fastest on a given hardware/software configuration.

https://www.kaggle.com/c/tensorflow-speech-recognition-challenge

https://www.kaggle.com/competitions/tensorflow-great-barrier-reef/overview/code-requirements

Since this is still not cleared up: I think you want to avoid stacking/ensembling a lot of diverse models as done in many kaggle competitions. So, please clarify this and state that, e.g. a random forest (which is usually an ensemble of more than 3 decision trees) counts as a single model so that its predictions can be e.g. averaged with 2 other models.