Adbot Ad Engagement Forecasting Challenge
'%3e%3cpath fill='%23001489' d='M0 0v6h9V0z'/%3e%3cpath fill='%23e03c31' d='M0 0v3h9V0z'/%3e%3cg stroke='%23fff' stroke-width='2'%3e%3cpath id='d' d='m0 0 4.5 3L0 6m4.5-3H9'/%3e%3cuse xlink:href='%23b' stroke='%23ffb81c' clip-path='url(%23c)'/%3e%3c/g%3e%3cuse xlink:href='%23d' fill='none' stroke='%23007749' stroke-width='1.2'/%3e%3c/g%3e%3c/svg%3e)
'/%3e%3c/defs%3e%3cpath fill='%23fff' d='M-120-80h240V80h-240z'/%3e%3cpath d='M-120-80h240v48h-240z'/%3e%3cpath fill='%23060' d='M-120 32h240v48h-240z'/%3e%3cg id='b'%3e%3cuse xlink:href='%23a' stroke='%23000'/%3e%3cuse xlink:href='%23a' fill='%23fff'/%3e%3c/g%3e%3cuse xlink:href='%23b' transform='scale(-1 1)'/%3e%3cpath fill='%23b00' d='M-120-24v48h101c3 8 13 24 19 24s16-16 19-24h101v-48H19C16-32 6-48 0-48s-16 16-19 24z'/%3e%3cpath id='c' d='M19 24c3-8 5-16 5-24s-2-16-5-24c-3 8-5 16-5 24s2 16 5 24'/%3e%3cuse xlink:href='%23c' transform='scale(-1 1)'/%3e%3cg fill='%23fff'%3e%3cellipse rx='4' ry='6'/%3e%3cpath id='d' d='M1 5.85s4 8 4 21-4 21-4 21z'/%3e%3cuse xlink:href='%23d' transform='scale(-1)'/%3e%3cuse xlink:href='%23d' transform='scale(-1 1)'/%3e%3cuse xlink:href='%23d' transform='scale(1 -1)'/%3e%3c/g%3e%3c/svg%3e){kind=small}
Hello Zindians,
Attached is a notebook to get you 12.27, potentially second on the lb. I would not have chosen this sub anyway as I had more sound approaches than this , so let us say this was just a lucky score. I would love to hear from more experienced guys like @yanteixeira why such an approach would get such a good score , because I also am confused and it has led to me thinking that this competition was only luck based. The luckiest solution won , and not the most sound solution. But this is likely not true for those who had public lb / private lb correlation.
The sound approach I am talking about here is where I train a model for each agent which honestly did not give me a very good result .
Also I would love the top teams to share their approaches too as I believe the time period of sending notebooks for evaluation has already ended ;)
Github link:
koleshjr/ADBOT: Can you predict the future success of a digital advert? (github.com)
If You find it helpful please star it. Makes me more motivated to open source more solutions. Thank you
And our good gist trick to post the code in the discussions page:) You guys should use this trick more often.
I'm curious why your solution also scored well. I'm comparing the results of yours and mine. Here's something interesting...
It's surprising that our models have such a large difference in predictions despite achieving similar performance on the private leaderboard.
Here's the most interesting finding: I looked at your submission and only changed the prediction for ID_629e729214035d4a8541e0b8_2024_02_28 from 444 (your original value) to 1107 (my model prediction). All other predictions remained the same. This resulted in:
So basically, nothing change! How? I also did the same experiment with my submission. I changed 1107 to your 444, and the output was the same score I was getting.
Maybe it was in the public leaderboard. That submission's public lb was >70 if I remember correctly
Oh, you are right. It seems I picked one example from the LB. I would have noticed if the public and private scores didn't have this bug.
Probably all the big differences between our models come from public LB IDs, and since I got a better LB score than you, we can assume that 1107 is closer to the true value than 444.
Well, we solved the mystery. RMSE punishes mispredictions very hard, and it seems your model got 1 or 2 IDs from the public LB wrong. BUT, if it weren't for these cases, your model is probably the best one here.
I don't think there is a way to know if this particular model would be good in the private LB because I don't think there is a single CV scheme that works for this problem. I wouldn't say 'The luckiest solution won' until we see @51pegasi solution because he is the only one in the top 10 with closer LB and private scores.
@yanteixeira true , we will wait for @51pegasi write up. also yours isn't far from each other. 7 and 12 , and they are both impressive. I will also wait for your write up to learn how you handled it.