South African COVID-19 Vulnerability Map by #ZindiWeekendz
Helping Africa
$300 USD
Challenge completed over 5 years ago
Prediction
319 joined
177 active
Start
Apr 03, 20
Close
Apr 05, 20
Reveal
Apr 05, 20
'%3e%3cpath fill='%23001489' d='M0 0v6h9V0z'/%3e%3cpath fill='%23e03c31' d='M0 0v3h9V0z'/%3e%3cg stroke='%23fff' stroke-width='2'%3e%3cpath id='d' d='m0 0 4.5 3L0 6m4.5-3H9'/%3e%3cuse xlink:href='%23b' stroke='%23ffb81c' clip-path='url(%23c)'/%3e%3c/g%3e%3cuse xlink:href='%23d' fill='none' stroke='%23007749' stroke-width='1.2'/%3e%3c/g%3e%3c/svg%3e){kind=small}
RMSE is the theoretically the wrong metric
Help · 5 Apr 2020, 19:36 · 6
RMSE makes the assumption that conditional distribution is symmetric and can range between -infinity and + infinity. Our per cent vulnerable values are bounded between 0 and 1. This should have been like the Deviance on the Beta Distribution or we would have been predicting the log(percent vulnerable). This is going to bias any insights we get from our models. Maybe this is for future reference.
@marcusinthesky i agree with you RMSE assigns a higher weight to larger errors meaning it is more useful when large errors are present
Did anyone train with MAE?
RMSE is the right metrics; if you log the target variable, low RMSE is obtained
logging values that are zeros and too close to zero doesn't correlate well with lb.I think it will e such better to use log when most values are not too close to zero
Noted! buddy...
I partially agree with @Engineer, RMSE(log target) would be a better assumption though 1. It is not the current metric and 2. it still suffers from scoring us against distribution which is not the same as our data. @DrFad I did not use MAE as MAE is not the metric used in the competition, MAE is also not differentiable so you cannot optimize directly on it.