Absa Customer Income Prediction Challenge
'%3e%3cpath fill='%23001489' d='M0 0v6h9V0z'/%3e%3cpath fill='%23e03c31' d='M0 0v3h9V0z'/%3e%3cg stroke='%23fff' stroke-width='2'%3e%3cpath id='d' d='m0 0 4.5 3L0 6m4.5-3H9'/%3e%3cuse xlink:href='%23b' stroke='%23ffb81c' clip-path='url(%23c)'/%3e%3c/g%3e%3cuse xlink:href='%23d' fill='none' stroke='%23007749' stroke-width='1.2'/%3e%3c/g%3e%3c/svg%3e)
Helping South Africa
$5 000 USD
Completed (~3 years ago)
Prediction
341 joined
54 active
Start
Nov 29, 22
Close
Feb 26, 23
Reveal
Feb 26, 23
Submission format and data
Data · 13 Dec 2022, 10:50 · 10
Unless I missed something, I think the competition description and sample submission are misleading as they both suggest a submission with raw NET income; I realised from the leaderboard errors that I should submit group codes instead (unless the entire public set just happens to be incomes less than R20.)
Considering people's scores: an RMSE of 5 isn't very good since it means that the predictions have a standard deviation of 5 income groups, so I don't think anyone is submitting the format expected.
?
Look at this thread https://zindi.africa/competitions/absa-customer-income-prediction-challenge/discussions/14364
I think you need to submit net income / 1000
Thanks, would be nice if this was stated somewhere.
Agree!
This *needs* to be stated explicitly.
@zindi @amyflorida626 this probably will continue to cause confusion, can you perhaps clarify somewhere visible that we have to predict
in the subs.
Hey hey, Absa provided us with a file called "Target". The file Target had 3 columns, as seen in the Train file [CUSTOMER_IDENTIFIER, RECORD_DATE, DECLARED_NET_INCOME]. We used the provided Target file and split it 70/30 on CUSTOMER_IDENTIFIER (if I am not mistaken) into the Train and Test files you see on Zindi. The exception being we kept the last column [DECLARED_NET_INCOME] for the Test file on our backend for scoring.
So to confirm. The test target/reference file was created as a direct subset from the same file Train was created so everything should follow similar formatting.
If you'd like you can recreate the Zindi data prep by working just with the Train file provided and see if you get similar scores to the leaderboard.
Thanks amy!
Hmmmm - it does make a difference. I just subbed two files. The same content, but different formatting. The one looks something like this
and the other follows Train formatting
and there is a *small* difference in score.
Aaah, this is frustrating! It means we (I) didn't check the type of the target which is meant to be float, not string.
We are working on a fix. Everyone's score should scale by 1000 now. I will make a discussion post when the update is done.
Thank you for being persistent with this!
No prob amy, of course, we all make mistakes ... that no matter, bounce back is what matters!
Thanks for discussion post, I think that will really be great to clarify to all.
Perhaps scoring system is also partly to blame - it should have rejected the funny format, how does it know to score correctly?
Anyhow, thanks amy ... look forward to the post
After reading these posts, including "Updated reference file: back to basics". I am still unsure of the format needed for the DECLARED_NET_INCOME in the submission.
If my ML predicts that a customer's DECLARED_NET_INCOME is R1 234,56 then which of these is a correct submission?:
a) 1234.56
b) 1.23456
c) other?
The answer is a. At least I hope.
A is the correct answer.