Hi team!
Currently it is stated that results should be returned in few seconds. What these "few seconds" mean. Sub 1 minute or 30s or even lower 10s-20s range.
How they are supposed to be used in field, even as an initial assessment tools, a 5-10s model that has much lower accuracy than a sub 1 minute has no point.
Please clarify why few seconds results return matter that much that you have not asked for any accuracy numbers on test set.
Hi @amadahmads
Thanks for the question - and you’re right to call out that “a few seconds” can mean very different things.
For this challenge, please interpret the requirement as: results should be returned ideally in under 1 minute per assessment on a typical mid-range Android device (offline).
In practice, most usable solutions may likely fall in the 15–60 second range, but we recognise that:
What matters most is that:
We are not benchmarking pure model latency. Instead, we are assessing whether the solution is viable as a real-world offline assessment tool.