Hello everyone!

Given that not everyone might be following the discussion from my previous post, and I'm concerned that the content might be overlooked without being noticed by the Zindi team, I've decided to create a dedicated post concerning the ensemble rule for this competition.

There's ambiguity in the term "ensemble". In many ML Competitions, "ensemble" typically refers to the act of combining predictions from various models to derive a final decision. However, certain algorithms, such as Random Forest and Gradient Boosting, are inherently "ensemble models" by their very design. Given this, the concerns raised by fellow competitors are quite valid.

Here's a proposed, clearer set of guidelines regarding ensemble models for this competition:

1. Definition: In the context of this competition, an "ensemble" is defined as the integration of predictions from several distinct models. This definition does not include the inherent ensemble mechanisms that are part of standard algorithms like Random Forest or Gradient Boosting Machines (LightGBM/XGBoost)

2. Restriction on Ensembles: Competitors can merge predictions from up to three unique models to create an ensemble. For instance, if one wishes to form an ensemble, predictions from models like Linear Regression, Random Forest, and a Neural Network can be combined. This combination will be recognized as a single ensemble.

3. Cross-Validation Clarification: Using cross-validation and averaging the predictions from multiple folds does NOT count as multiple models. It's considered a part of the training and evaluation process for a single model. For instance, if you train a Neural Network using 5-fold cross-validation and then average the predictions of those 5 folds to create a submission, it's still considered a single Neural Network model.

4. Inherent Ensemble Models: Algorithms like Random Forest, Gradient Boosting Machines, etc., which inherently use ensemble mechanisms, are treated as a single model regardless of the number of trees/estimators they use. For instance, a Random Forest with 100 trees is considered one model, not 100.

Of course, this is merely a suggestion. I'm eager to hear opinions from both the competitors and the Zindi team. Regardless of the eventual decision, I believe there's consensus that the text in the Data Section needs revision.