Kenya Clinical Reasoning Challenge
'/%3e%3c/defs%3e%3cpath fill='%23fff' d='M-120-80h240V80h-240z'/%3e%3cpath d='M-120-80h240v48h-240z'/%3e%3cpath fill='%23060' d='M-120 32h240v48h-240z'/%3e%3cg id='b'%3e%3cuse xlink:href='%23a' stroke='%23000'/%3e%3cuse xlink:href='%23a' fill='%23fff'/%3e%3c/g%3e%3cuse xlink:href='%23b' transform='scale(-1 1)'/%3e%3cpath fill='%23b00' d='M-120-24v48h101c3 8 13 24 19 24s16-16 19-24h101v-48H19C16-32 6-48 0-48s-16 16-19 24z'/%3e%3cpath id='c' d='M19 24c3-8 5-16 5-24s-2-16-5-24c-3 8-5 16-5 24s2 16 5 24'/%3e%3cuse xlink:href='%23c' transform='scale(-1 1)'/%3e%3cg fill='%23fff'%3e%3cellipse rx='4' ry='6'/%3e%3cpath id='d' d='M1 5.85s4 8 4 21-4 21-4 21z'/%3e%3cuse xlink:href='%23d' transform='scale(-1)'/%3e%3cuse xlink:href='%23d' transform='scale(-1 1)'/%3e%3cuse xlink:href='%23d' transform='scale(1 -1)'/%3e%3c/g%3e%3c/svg%3e)
Helping Kenya
$10 000 USD
Completed (9 months ago)
Prediction
Natural Language Processing
SLM
1666 joined
440 active
Start
Apr 03, 25
Close
Jun 29, 25
Reveal
Jun 30, 25
0.35 Score Notebook with inference Below 100ms
Notebooks · 24 May 2025, 09:36 · 11
https://colab.research.google.com/drive/18ySAIJ-pfdlaXeLouaT19DSmMFfs3HrE?usp=sharing
Hi All above is a notebook I have been working on, testing to improve inference speeds to be below 100ms as per the constraints. You can run the whole notebook on colab and test the results there as well.
The first inference method (321.28 ms) gives a score of 0.38 which means for fast inference the accuracy is sacrifised a little bit. Please review and share your comments.
Also note Ram Usage is Below 2GB at 1.18GB
The inference speed constraint of 100ms is per vignette and not for the entire test set. I don't know if that helps
In the notebook it's per vignette. Only that it's an everage (Average time per vignette)
Alright
I will check the notebook
Thank you Great work!! that's a clear notebook
Great man.
very nice.
no need to share how you do it but am curious if you are able to hit that 100ms?
The inference time should be per vignette not the calculated average inference time per vignette after running the samples throught the model in batches (multiparallel). This is what your code does. As it stands, there is no model between 100M and 200M that meets the constraint of generating full response below 100ms per vignette. I have tested this out with all forms of tweaks (except int1-3).
That's what I noted as well. Something that needs checking and improvement.