Hi all,
I’d like to clarify the score weighting, as this has come up in several chats.
The evaluation metric is a weighted multi-metric composed of Log Loss (75%) and AUC (25%). However, this 75%/25% split is not distributed evenly across the three prediction targets.
From a practical perspective, the 7-day target is more operationally useful than the 90- or 120-day targets. As a result, the 7-day target carries a higher weight in the overall score.
Happy coding!
Thanks for sharing this, noticed this too.
could you share the function then so that we can evaluate our models locally with that?
{Apparently, these are the weights that are being used :(
Here is a function to calculate the LB score:def calculate_weighted_score(target_07_auc,target_07_logloss,target_90_auc,target_90_logloss,target_120_auc,target_120_logloss):SCALING = 0.56926target_07_logloss_norm = 1 - (target_07_logloss / SCALING)target_90_logloss_norm = 1 - (target_90_logloss / SCALING)target_120_logloss_norm = 1 - (target_120_logloss / SCALING)weighted_score = (target_07_auc * 0.15 +target_07_logloss_norm * 0.65 +target_90_auc * 0.05 +target_90_logloss_norm * 0.05 +target_120_auc * 0.05 +target_120_logloss_norm * 0.05)return weighted_score