AI/ML for 5G-Energy Consumption Modelling by ITU AI/ML in 5G Challenge
'/%3e%3c/defs%3e%3cpath fill='%23fff' d='M-120-80h240V80h-240z'/%3e%3cpath d='M-120-80h240v48h-240z'/%3e%3cpath fill='%23060' d='M-120 32h240v48h-240z'/%3e%3cg id='b'%3e%3cuse xlink:href='%23a' stroke='%23000'/%3e%3cuse xlink:href='%23a' fill='%23fff'/%3e%3c/g%3e%3cuse xlink:href='%23b' transform='scale(-1 1)'/%3e%3cpath fill='%23b00' d='M-120-24v48h101c3 8 13 24 19 24s16-16 19-24h101v-48H19C16-32 6-48 0-48s-16 16-19 24z'/%3e%3cpath id='c' d='M19 24c3-8 5-16 5-24s-2-16-5-24c-3 8-5 16-5 24s2 16 5 24'/%3e%3cuse xlink:href='%23c' transform='scale(-1 1)'/%3e%3cg fill='%23fff'%3e%3cellipse rx='4' ry='6'/%3e%3cpath id='d' d='M1 5.85s4 8 4 21-4 21-4 21z'/%3e%3cuse xlink:href='%23d' transform='scale(-1)'/%3e%3cuse xlink:href='%23d' transform='scale(-1 1)'/%3e%3cuse xlink:href='%23d' transform='scale(1 -1)'/%3e%3c/g%3e%3c/svg%3e){kind=small}
Hello Zindians,
I would love to make a suggestion. As you all already know , using future values is prohibited and for example someone submits a solution that uses future values whether knowingly or unknowingly, we as participants won't know for sure since after the competition ends, the evaluation happens by the organization sponsoring the challenge.
With all the uncertainities and gray area of these "future" values , and whether someone has used future values or not, I am suggesting that the top 10 or 20 solutions that will be picked after the private leaderboard are posted here on the discussions , so that when a solution is disqualified or passes we as the community know for sure why it was disqualified or why it passed. This will enhance transparency in the evaluation process and will ensure public scrutiny.
I understand that this dataset would have been sampled properly to avoid this situation , but until we as the community or participants know why a certain solution passed and which never did , then we will get satisfied and know better ways of handling this kind of situations in future.
This is my own personal opinion and its just a humble suggestion. I don't know what are the opinions of other participants. Feel free to air out your thoughts
Very good thoughts @Koleshjr. Always on point.
The point is that the sponsor made it clear that the competition aims to have algorithms that solve real-world problems. That said, any kind of data leakage makes the solution not functional in reality. Not only future data leakage, but any kind of leakage that causes patterns to make that solution unviable in the real world, because those patterns were only available in the training or test data due to some mistake. Therefore, any mistake that could lead to better scores, but not bring a real-world solution should be considered data leakage. So, for transparency, it would be interesting to have clear rules or at least the notion that the real-world solution will be the one to win the competition.
Another point that stands out is that trusting internally that competition participants won't commit these errors or engage in data leakage is a naive way to approach the issue. We know that in reality this can be done either voluntarily or involuntarily. As highlighted, there's a large gray area in this matter, and perhaps only the sponsors truly know what constitutes data leakage or not. Therefore, it's important to consider this.