AI4D Yorùbá Machine Translation Challenge
Helping Nigeria
$2 000 USD
Completed (almost 5 years ago)
Machine Translation
683 joined
84 active
Start
Dec 04, 20
Close
May 30, 21
Reveal
May 30, 21
'%3e%3cpath fill='%23001489' d='M0 0v6h9V0z'/%3e%3cpath fill='%23e03c31' d='M0 0v3h9V0z'/%3e%3cg stroke='%23fff' stroke-width='2'%3e%3cpath id='d' d='m0 0 4.5 3L0 6m4.5-3H9'/%3e%3cuse xlink:href='%23b' stroke='%23ffb81c' clip-path='url(%23c)'/%3e%3c/g%3e%3cuse xlink:href='%23d' fill='none' stroke='%23007749' stroke-width='1.2'/%3e%3c/g%3e%3c/svg%3e){kind=small}
Error metric fixed and leaderboard rescored
Platform · 6 May 2021, 13:04 · 7
Dear competitors,
Thank you for your patience while we worked on the error metric. It proved harder than we initially thought due to the diacritics and different characters.
We have implemented the Rouge Score, reporting the F-measure. This error metric was implemented on 5 May 2021 and the leaderboard rescored.
The Recall-Oriented Understudy for Gisting Evaluation (ROUGE) scoring algorithm calculates the similarity between a candidate document and a collection of reference documents. Use the ROUGE score to evaluate the quality of document translation and summarization models [ref].
Once again, thank you for your patience and perseverance during this challenge.
Hi, Thanks for the update :)
But which ROUGE Score is used - ROUGE-L or ROUGE-1 ??
Hi, it is ROUGE-N (N-gram) scoring (Rouge1), reporting the F-measure.
Ok got it !
Is it possible to publish any starter code with this Rouge Score, used for evaluating and model training - for me as beginner is not clear how to use it?
Hello @Zindi, are punctuations important for the translation, are they taken into account for the ROUGE score?
Update - I see ROUGE Score ignores the punctuations when I try the python metric. Thanks
Yes, the diacritics and accents are taken into account.
Thank you for the clarification