Smart Energy Supply Scheduling for Green Telecom Challenge by ITU
So now that the competition is over I'd like to exchange some experiences with it.
Part of the challenge was predicting load and solar energy. I first assumed we can basically not forecast this as there's always a small chance of a cloudy day and predicting weather in advance 7 days is challenging and unreliable even with much better data than we were given.
Nevertheless, some timeseries models made very confident and correct predictions for various splits of the data. After looking at the plots I noticed that of the 60 days, the second 30 days is basically the first 30 days shifted + some noise. So we are definitely dealing with synthetic data and not realistic weather data as it's rather unplausible that weather repeats almost exactly in a month.
So obviously I tried just repeating the data from the first 30 days again with some added slack (in case of load) or a subtraction of some slack (in case of solar energy). So using this method I got basically perfect predictions without even using any time series models, at least for various splits of the training data. However this surprisingly didn't result in valid solutions, even with rather larg slack values.
I can't rule out that I had a bug somewhere, but this seemed very odd. Of course it's possible that the weather / load doesn't repeat for a third time and some surprises were introduced, however that would be also rather unexpected. Why not have some odd days in the training data as well then?
Obviously some other participants successfully navigated this issue, congrats to you! What are your thoughts on this?
The current leaderboard is abit off. Kindly confirm @nicolapiovesan, @ZINDI, @meganomaly. Foristance my SampleSubmission which is a valid strategy (set to all TRUE for all sources of energy solar,diesel and grid) initially with a socre of about 12000 on public score now has a score of 0. Another argument is that all my other submissions did not pass through only this one which was a very risky one..These are grounds to conclude that there is something off with the leaderboard...
We would like to raise a concern regarding the recently revealed results and request a re-evaluation to ensure that all our submissions were properly considered.
It seems that some of our more recent submissions were not included in the final private leaderboard evaluation. Over the past several days, we submitted conservative solutions that we are confident would be feasible. However, the final leaderboard only reflects the solution we submitted over a month ago, which is notably worse than our recent submissions.
Could @ZINDI please re-check the submissions we made during the last four days of the competition (from 26th September to 30th September)? To ensure the fairness of the competition, we need confirmation that all our solutions were evaluated.
Additionally, we noticed that several teams, including ours, saw their rankings drop dramatically in the private leaderboard compared to the public leaderboard. We suspect that the private leaderboard evaluation may have been limited to the first 300 submissions, which includes infeasible submissions from earlier public leaderboard rounds. If this is the case, it may not align with the competition rules, as only feasible submissions should count towards the 300 limit. Given this, it’s essential that all submissions, especially those submitted more recently, be evaluated as they are critical to determining the final solution quality.
Thank you for your attention to this matter. @nicolapiovesan, @ZINDI, @meganomaly, maybe you can also opensource the private data to us and let us see the result.
Best regards
I feel either there is a problem with the submission evaluation, or one site has very uncommon data in the private period which requires extreme conservative strategies. The thing is all my submssions are scored 0 or 'The strategy is unfeasible'. I even don't know how my final score comes from. I have no any clues about the private tests.