AirQo Low-Cost Air Quality Monitor Calibration Challenge
'/%3e%3c/defs%3e%3cpath fill='%23fff' d='M-120-80h240V80h-240z'/%3e%3cpath d='M-120-80h240v48h-240z'/%3e%3cpath fill='%23060' d='M-120 32h240v48h-240z'/%3e%3cg id='b'%3e%3cuse xlink:href='%23a' stroke='%23000'/%3e%3cuse xlink:href='%23a' fill='%23fff'/%3e%3c/g%3e%3cuse xlink:href='%23b' transform='scale(-1 1)'/%3e%3cpath fill='%23b00' d='M-120-24v48h101c3 8 13 24 19 24s16-16 19-24h101v-48H19C16-32 6-48 0-48s-16 16-19 24z'/%3e%3cpath id='c' d='M19 24c3-8 5-16 5-24s-2-16-5-24c-3 8-5 16-5 24s2 16 5 24'/%3e%3cuse xlink:href='%23c' transform='scale(-1 1)'/%3e%3cg fill='%23fff'%3e%3cellipse rx='4' ry='6'/%3e%3cpath id='d' d='M1 5.85s4 8 4 21-4 21-4 21z'/%3e%3cuse xlink:href='%23d' transform='scale(-1)'/%3e%3cuse xlink:href='%23d' transform='scale(-1 1)'/%3e%3cuse xlink:href='%23d' transform='scale(1 -1)'/%3e%3c/g%3e%3c/svg%3e){kind=small}
Hi, I am new to competitions. I have two issues here.
1. My code is giving me poorer scores than it was sometimes back. Initially, I had not set a global random state. the code is the same, the only thing that I have added is the random state, I am now getting worse scores. The reason as to why I have set the random state is that I was getting random scores. Every time that I was running my code, I could get different results. Is there anything that I can do to get the scores which I was getting initially?
2. The scores that I am getting locally are different from the ones that I am getting on the leaderboard. As my local scores decrease, the public leaderboard scores do not decrease. I am not sure whether I should trust my local scores or the public leaderboard scores.
I would love to hear from anyone who knows how I can address these issues. Thank you.
Hi, it's unfortunate you can't reproduce the best score you had earlier gotten. It's always advised that you set the random seed before training just for the basis of reproducibility. Unfortunately, I'm not sure if there's a way to get back the score, unless by sheer luck, because it's hard to say which pseudo random number really got that score. There is a large set of random numbers and it's unproductive to search through this set to determine the right seed. All the best of luck.
Thank you @Gozie for your clarification. I will now work with the scores that I have got that are reproducible. It is unfortunate that I did not know the importance of reproducible results while I was starting this competition. I wasted very many submissions trying to figure this out.
Sorry about the reproducibility, I will advise generating a submission across 10 random seeds just to reduce the randomness of results by averaging different seeds.
Thank you @DamolaOr for sharing your approach. I will try your approach. It is also true that averaging different seeds can improve results too.