Fault Impact Analysis: Towards Service-Oriented Network Operation & Maintenance by ITU
8 000 CHF
Completed (over 2 years ago)
Classification
273 joined
89 active
Start
Jul 26, 23
Close
Aug 18, 23
Reveal
Aug 18, 23
'%3e%3cg id='b'%3e%3cpath id='a' d='M0-8v8h4z' transform='rotate(18 0 -8)'/%3e%3cuse xlink:href='%23a' transform='scale(-1 1)'/%3e%3c/g%3e%3cuse xlink:href='%23b' transform='rotate(72)'/%3e%3cuse xlink:href='%23b' transform='rotate(144)'/%3e%3cuse xlink:href='%23b' transform='rotate(216)'/%3e%3cuse xlink:href='%23b' transform='rotate(288)'/%3e%3c/g%3e%3c/svg%3e){kind=small}
Updated CV score vs LB score
Platform · 14 Aug 2023, 10:13 · 19
CV:0.8015 & LB:0.7023
CV:0.8016 & LB:0.7100
what are yours!?
You had me for a moment with that LB 0.79 😱
🥶
CV 0.71 - LB 0.73
I have two models
first one is CV 0.72 - LB 0.72
second is CV 0.72 - LB 0.73
I definitely hit a wall. I don't know how many different things I tried in the last few days... just impossible to improve. Hope you all share your code after this competition 🤣
You're not the only one hitting a wall. I hope this competition ends. I can't wait to see the best approach.
0.71 - 0.71
Many report here CV = LB, for this comp that is also my experience fwiw. At least it is almost over. I also saw a ~10% gap (~70CV vs ~60LB) for some subs, but those were due to too few obs, so CV was overfit. You can fit e.g. a model to all obs or to obs only where a fault occurs, and this gap I saw when I did the latter.
Yes, when the model is fitted with the observations where a fault occurs, the deviation occurs. So I changed my approach and this discrepancy recurs. however, when I create many local tests (different seeds), my CV score is very close to my local test scores; it gives me more confidence. Also, LB is calculated on 20% of test data and depending on how the organizer has selected test samples, it could have a big shake up in the private ranking.
Can't wait to see the best solutions!
Oh what a competition, tried my best unfortunately could not hack >0.72 but I have learnt alot in this one. May the best model win and can't wait to learn from whoever will win. 0.75 guys kudos mahn, thats a really impressive score. Anyways let's enjoy the last moments of this interesting competition :)
Exciting competition!
You know, given the close correlation between CV and LB I personally see in this one and as reported here also, I doubt there will be a huge shake up.
But just a few hours more and we get the result ...
Well the correlation gets a little bit lost after >0.7
btw - how you get an image in your post?
Drag and drop
Since I couldn't reach the same high LB values as the top 5, I'm choosing submissions that I believe are the right approach to the problem.
Although the host did not answer my post, they will review the code of 20 submissions, so there is a reason to believe that a good private score is not the only metric considered.
Anyway, it was a very good competition. I also learned a lot, and I'm kinda sad I hit the wall in the last days. It was a good experience competing here at Zindi
hey @yanteixeira, since the evaluation of this competition as mentioned officially is only the f1 score. your approach does not matter. No weightage is given to any approach. The notebooks are asked to evaluate whether your code reproduces your exact score, if not ranks are adjusted accordingly.
Thank you for clarifying!
Well, bit of a shake up it seems.
Bit of a disaster for me, unable to get a score right at the end. My best two public LB subs (I wanted to submit more, but scoring was stuck) would have given me .737 on the private LB, but who knows if I would have chosen them ... I still wanted to sub a few and then make a final selection.
Congrats to @Krishna_Priya and @JuliusFx and others.
Wow @JuliusFx, seems heaven saw your hard work after all ... and what a shake-up @Koleshjr seems the >70 was a bit shuffled after all.
Hey @skaak, just woke up, I like what am seeing for sure. I liked the shuffling that pushed me upwards😂..some bit of luck plus putting in the hours worked. It's a blessing.