Hello Romaz, thank you for your reply! I was wondering about using that as a benchmark. Is that usually how benchmarks for a competition are determined? Because I don't think notebooks get released during competitions all the time!
I thought of using the scoreboard as a more objective benchmark, however it also looks like a few people might have used Tikhon's notebook as it is to get those scores, not too sure about that. As such, I was wondering if there was a benchmark or a range of scores which existed before Tikhon released his notebooks!
Do let me know if Tikhon's notebooks would truly be the benchmark! I am not sure how to evaluate what is a good benchmark, and what is not evidently haha
A good benchmark would normally be a model with no feature selection or hyperparameter tuning (e.g., lightGBM). If there are different ways to frame the problem (CV schemes, target creation) you could try your baseline model with different methods and see how the CV / leaderboard compare. Many competitions is much easier than this competition to frame the problem - for example in the X-ray challenge (https://zindi.africa/competitions/runmila-ai-institute-minohealth-ai-labs-tuberculosis-classification-via-x-rays-challenge) it's binary classification with a clear target, so your baseline can be just a simple NN.
Thank you for your detailed reply! I Guess I should read up more on establishing my own benchmark as well as target creation for this kind of dataset! Have a gr8 day ahead 👍
 scale(.21174)'%3e%3cpath d='m52.487 177.086 13.018 11.852 11.762-11.762 11.896 12.266 12.846-11.358 11.827 12.43 13.339-11.013 10.897 12.728 13.27-10.145 13.947 12.319'/%3e%3ccircle cx='89.425' cy='154.584' r='5.167'/%3e%3ccircle cx='111.237' cy='156.047' r='5.167'/%3e%3ccircle cx='30.05' cy='16.5' r='4'/%3e%3c/g%3e%3c/svg%3e)
'/%3e%3cuse xlink:href='%23a' transform='translate(288.889 -214.211)'/%3e%3cuse xlink:href='%23a' transform='translate(108 342.749)'/%3e%3cuse xlink:href='%23a' transform='translate(469.189 342.749)'/%3e%3c/svg%3e){kind=small}
You can find it in discussions, just try to research:)
There are a lot of notebooks by Tikhon there
Hello Romaz, thank you for your reply! I was wondering about using that as a benchmark. Is that usually how benchmarks for a competition are determined? Because I don't think notebooks get released during competitions all the time!
I thought of using the scoreboard as a more objective benchmark, however it also looks like a few people might have used Tikhon's notebook as it is to get those scores, not too sure about that. As such, I was wondering if there was a benchmark or a range of scores which existed before Tikhon released his notebooks!
Do let me know if Tikhon's notebooks would truly be the benchmark! I am not sure how to evaluate what is a good benchmark, and what is not evidently haha
A good benchmark would normally be a model with no feature selection or hyperparameter tuning (e.g., lightGBM). If there are different ways to frame the problem (CV schemes, target creation) you could try your baseline model with different methods and see how the CV / leaderboard compare. Many competitions is much easier than this competition to frame the problem - for example in the X-ray challenge (https://zindi.africa/competitions/runmila-ai-institute-minohealth-ai-labs-tuberculosis-classification-via-x-rays-challenge) it's binary classification with a clear target, so your baseline can be just a simple NN.
Thank you for your detailed reply! I Guess I should read up more on establishing my own benchmark as well as target creation for this kind of dataset! Have a gr8 day ahead 👍