Just curious, but was there a pre-existing benchmark to begin with? Just wanted to see if it was possible to have a decent gauge of my newbie skills
You can find it in discussions, just try to research:)
There are a lot of notebooks by Tikhon there
Hello Romaz, thank you for your reply! I was wondering about using that as a benchmark. Is that usually how benchmarks for a competition are determined? Because I don't think notebooks get released during competitions all the time!
I thought of using the scoreboard as a more objective benchmark, however it also looks like a few people might have used Tikhon's notebook as it is to get those scores, not too sure about that. As such, I was wondering if there was a benchmark or a range of scores which existed before Tikhon released his notebooks!
Do let me know if Tikhon's notebooks would truly be the benchmark! I am not sure how to evaluate what is a good benchmark, and what is not evidently haha
A good benchmark would normally be a model with no feature selection or hyperparameter tuning (e.g., lightGBM). If there are different ways to frame the problem (CV schemes, target creation) you could try your baseline model with different methods and see how the CV / leaderboard compare. Many competitions is much easier than this competition to frame the problem - for example in the X-ray challenge (https://zindi.africa/competitions/runmila-ai-institute-minohealth-ai-labs-tuberculosis-classification-via-x-rays-challenge) it's binary classification with a clear target, so your baseline can be just a simple NN.
Thank you for your detailed reply! I Guess I should read up more on establishing my own benchmark as well as target creation for this kind of dataset! Have a gr8 day ahead 👍