This is my link for the submission file https://pastebin.com/Di2h99e1
- As you can see, the error of bounding boxes for my submission, to the baseline solution is not miles apart; but the scores are soo low which indicates some bug for the LB calculation. @zindi
There is a huge flaw in the metric calculation:-
1. Spamming with bboxes in the submission file leads to a much, much higher score. This implies that boxes below certain thresholds are not penalized; Thus you can literally brute-force your way in the competition and still have a very high score.
2. This directly rewards imprecise models like Faster-RCNN who used less aggressive NMS and submit 7-9 boxes per image.
I think people are still sceptical of my claims - so allow me to demonstrate. This is a submission of 0.4 LB score:-You can easily see NON-CHERY PICKED examples here - and judge for yourself whether it is indeed a 0.4 submission (below baseline):-
https://postimg.cc/gallery/tFpFTCV
@zindi please check and confirm whether the metrics are working as intended. This is not a post to point out the firm's incompetency, but rather to understand why it works like that :)
@zindi I have updated the question with non-cherrypicked samples. kindly investigate and get back to us! :)