in my case, there are several test images contains pbw, but when I put the count into the submission the MAE increases compare with when the count for those images is 0.
yes, since they said they annotated them automatically by tool.
Here is the full list:
'id_0660586f3ed26178d08c6636.jpg',
'id_0b2ce8235c041126ca62d9c1.jpg',
'id_0c2a4b8393c3c0dc26fe18bb.jpg',
'id_15fc1f078865759ff45638e6.jpg',
'id_169f24d5f7cb7b21fe60df1e.jpg',
'id_1c96db4131c3fab81949a9fd.jpg',
'id_22e2598aa43a8cd0a7fdb5df.jpg',
'id_2510cb8b891d54dd1570ecc5.jpg',
'id_253e58c20f3dfbcef9db86c5.jpg',
'id_259990c7e44466c1853ac2fe.jpg',
'id_284f01098fe5bd92d93cc883.jpg',
'id_291c3927b05f73a336bd791f.jpg',
'id_2e7277d3546e5959d4c4b5ca.jpg',
'id_33dc6bd2f9f3c54ad6c03e1f.jpg',
'id_35d66ed5826a36d6e75053c6.jpg',
'id_38aed360b54c1e388e713440.jpg',
'id_447bca3dd59e31efab6719dd.jpg',
'id_4e186cb9161481297072821f.jpg',
'id_553f205c03ad2267d1073295.jpg',
'id_57701dfba47796b86b81e667.jpg',
'id_57eefc079173a65293e00c49.jpg',
'id_6055a3f37d25ac3a72d48b70.jpg',
'id_616bac757f4121ec2c319ff1.jpg',
'id_6737603b610159ab05ec356a.jpg',
'id_69378b6d40a23b51e72ed293.jpg',
'id_70986ad58af6664bf6fd5c6f.jpg',
'id_722d421d27db409c84cd90f1.jpg',
'id_75ec767e2219455019c1479c.jpg',
'id_78c79994c0fc9a2b4ccde17a.jpg',
'id_7e51b6cd7c803c9be4fbd0ab.jpg',
'id_8018014190322c8b2aa2cc25.jpg',
'id_8568d9e1533d530d168c7a04.jpg',
'id_8d0c3546d3ec9335339fbe32.jpg',
'id_8ddb685e1e2e2541bef9e6b1.jpg',
'id_91c21353aad56c62e5663129.jpg',
'id_9782d37ca668b5e66d4c3cfd.jpg',
'id_9a86018c21369815ffca0b07.jpg',
'id_a79b37b318ba77014b645ae0.jpg',
'id_a9cdb42497f9fd81c8e14880.jpg',
'id_b11d7e28d21c8c0b4507d94b.jpg',
'id_b8e7b3fe9aecae663a3cb687.jpg',
'id_ba6406be65dd3658d5d15bb9.jpg',
'id_bfda5d27ab0701de4cd48a40.jpg',
'id_bff124ee45fc42692656fd48.jpg',
'id_cba3ee362a28041a9c37641f.jpg',
'id_cf25b62de3e859747098a445.jpg',
'id_d696413c7f76d077d69bb605.jpg',
'id_e062aea6d9c3388be5e93ff1.jpg',
'id_e12b4de3a39da61ecadc349c.jpg',
'id_e76a6ba0911184c4bd8dd6b4.jpg',
'id_e86db6164f04c87a1a9ba7ed.jpg',
'id_eecae5c7d33eb81d2f308f8f.jpg',
'id_c9b824b2e88a356a98c4d25f.jpg',
'id_f89153e9ea4ee941935daef0.jpg'
Some may not change your MAE because it's in private test, but most of them will increase the MAE. I have manually check them by hand, and with yolo I can detect them as well, not another worms
I think the boxes is correct. May be some error when they count the number of worms lead to different? I still suppose that your worms list above are not contains pbw
Maybe after the competition you guys share how you handled this problem? I am struggling breaking out from the 2.5-3 mae range. I would appreciate that. @mithilsalunkhe@tinyswish
It was just first simple regression model without using bboxes etc and it surprisingly give me 3rd place on public lb with first submission) So, this is fooled me)
We've used swin-large, trained for 40 epochs, simple MAE, imgsize 384
We've tried beit_large with resolution 512, but it didn't improve the CV, as for the split, we first log-binarized the per-worm target and then stratified on these bins
But the best sub was with yolov7, detector works very well on private lb
haha...lol, all the best!
Maybe not, I think the guys at top are doing something special.
yeah, they are so amazing
i doubt that the ground truths is correct.
in my case, there are several test images contains pbw, but when I put the count into the submission the MAE increases compare with when the count for those images is 0.
I have also checked it seems to be correct
'id_0660586f3ed26178d08c6636.jpg',
'id_0b2ce8235c041126ca62d9c1.jpg',
'id_0c2a4b8393c3c0dc26fe18bb.jpg',
'id_15fc1f078865759ff45638e6.jpg',
'id_169f24d5f7cb7b21fe60df1e.jpg',
'id_1c96db4131c3fab81949a9fd.jpg',
'id_22e2598aa43a8cd0a7fdb5df.jpg',
'id_2510cb8b891d54dd1570ecc5.jpg',
'id_253e58c20f3dfbcef9db86c5.jpg',
For example, can you check these?
It seems to be another worms? None of them when I put count for has improve my LB
i don't think it's another worms :v
oh really? So your opinion is ground truths are not correct?
yes, since they said they annotated them automatically by tool.
Here is the full list:
'id_0660586f3ed26178d08c6636.jpg',
'id_0b2ce8235c041126ca62d9c1.jpg',
'id_0c2a4b8393c3c0dc26fe18bb.jpg',
'id_15fc1f078865759ff45638e6.jpg',
'id_169f24d5f7cb7b21fe60df1e.jpg',
'id_1c96db4131c3fab81949a9fd.jpg',
'id_22e2598aa43a8cd0a7fdb5df.jpg',
'id_2510cb8b891d54dd1570ecc5.jpg',
'id_253e58c20f3dfbcef9db86c5.jpg',
'id_259990c7e44466c1853ac2fe.jpg',
'id_284f01098fe5bd92d93cc883.jpg',
'id_291c3927b05f73a336bd791f.jpg',
'id_2e7277d3546e5959d4c4b5ca.jpg',
'id_33dc6bd2f9f3c54ad6c03e1f.jpg',
'id_35d66ed5826a36d6e75053c6.jpg',
'id_38aed360b54c1e388e713440.jpg',
'id_447bca3dd59e31efab6719dd.jpg',
'id_4e186cb9161481297072821f.jpg',
'id_553f205c03ad2267d1073295.jpg',
'id_57701dfba47796b86b81e667.jpg',
'id_57eefc079173a65293e00c49.jpg',
'id_6055a3f37d25ac3a72d48b70.jpg',
'id_616bac757f4121ec2c319ff1.jpg',
'id_6737603b610159ab05ec356a.jpg',
'id_69378b6d40a23b51e72ed293.jpg',
'id_70986ad58af6664bf6fd5c6f.jpg',
'id_722d421d27db409c84cd90f1.jpg',
'id_75ec767e2219455019c1479c.jpg',
'id_78c79994c0fc9a2b4ccde17a.jpg',
'id_7e51b6cd7c803c9be4fbd0ab.jpg',
'id_8018014190322c8b2aa2cc25.jpg',
'id_8568d9e1533d530d168c7a04.jpg',
'id_8d0c3546d3ec9335339fbe32.jpg',
'id_8ddb685e1e2e2541bef9e6b1.jpg',
'id_91c21353aad56c62e5663129.jpg',
'id_9782d37ca668b5e66d4c3cfd.jpg',
'id_9a86018c21369815ffca0b07.jpg',
'id_a79b37b318ba77014b645ae0.jpg',
'id_a9cdb42497f9fd81c8e14880.jpg',
'id_b11d7e28d21c8c0b4507d94b.jpg',
'id_b8e7b3fe9aecae663a3cb687.jpg',
'id_ba6406be65dd3658d5d15bb9.jpg',
'id_bfda5d27ab0701de4cd48a40.jpg',
'id_bff124ee45fc42692656fd48.jpg',
'id_cba3ee362a28041a9c37641f.jpg',
'id_cf25b62de3e859747098a445.jpg',
'id_d696413c7f76d077d69bb605.jpg',
'id_e062aea6d9c3388be5e93ff1.jpg',
'id_e12b4de3a39da61ecadc349c.jpg',
'id_e76a6ba0911184c4bd8dd6b4.jpg',
'id_e86db6164f04c87a1a9ba7ed.jpg',
'id_eecae5c7d33eb81d2f308f8f.jpg',
'id_c9b824b2e88a356a98c4d25f.jpg',
'id_f89153e9ea4ee941935daef0.jpg'
Some may not change your MAE because it's in private test, but most of them will increase the MAE. I have manually check them by hand, and with yolo I can detect them as well, not another worms
If so, would you trust your model or the ground truth?
since they said in here: Worm count discrepancy - Zindi
could you trust the ground truth as there are errors in there?
I think the boxes is correct. May be some error when they count the number of worms lead to different? I still suppose that your worms list above are not contains pbw
Improves score for me
Waiting to see what the Magic Is
Maybe after the competition you guys share how you handled this problem? I am struggling breaking out from the 2.5-3 mae range. I would appreciate that. @mithilsalunkhe @tinyswish
I will surely share it once I get the time
@VictorDurnov @cloudyy be sure to share your approach later, it's very interesting :D
we've tried everything, we could think of, but couldn't manage to get CV lower than 1.45, we're waiting for your solution
edit: wild guess, - pseudo-labels, correct (some) labels with detector or something :D
Nothing special in my case, I just was fooled with public leaderboard) I'm also interested to hear about the winning solutions
I would be intrested in your solution too , how did you manage such a good public lb score
It was just first simple regression model without using bboxes etc and it surprisingly give me 3rd place on public lb with first submission) So, this is fooled me)
What is your solution?
Mine relied on yolov5
some kaggle GM magic :o
We've used swin-large, trained for 40 epochs, simple MAE, imgsize 384
We've tried beit_large with resolution 512, but it didn't improve the CV, as for the split, we first log-binarized the per-worm target and then stratified on these bins
But the best sub was with yolov7, detector works very well on private lb
im using dino by detrex, however, i am still doubting the ground truth, minor error can lead to different private score
I will share my solution later, but I've done overfit public LB :)