Primary competition visual

AutoInland Vehicle Insurance Claim Challenge

Helping Nigeria
$1 000 USD
Challenge completed over 4 years ago
Prediction
1603 joined
632 active
Starti
Mar 26, 21
Closei
Jun 27, 21
Reveali
Jun 27, 21
User avatar
Kenyatta university
zero_rank_score
Help · 9 Apr 2021, 22:41 · 15

How is it that I get good accuracy and f1_score say above 0.9 for both and still get a zero score. what am I missing.

Discussion 15 answers

0.9 is not exactly a good score . Follow tips in the starter notebook to improve your score.

10 Apr 2021, 07:40
Upvotes 0

you may be having data leakage. I also tried uploading the csv created by the starter notebook and getting zero score like you 😵😕

User avatar
Kenyatta university

I am trying everything I could think of and it like am running out of options

User avatar
CapitainData
UM6P

Hi Daniel!

If you are using the starter notebook, please pay attention to the constitution of the submission file!

instead of sub_file.predictions = predictions, use sub_file.target = predictions.

Regards,

CapitainData!

14 Apr 2021, 10:50
Upvotes 0
User avatar
CapitainData
UM6P

Always Buddy! Awesome Competition!

Hi CaptainData,

Thank you for the comment! I noticed it too in the starter notebook. But I put my predictions in the 'target' column instead of the 'predictions' column like you pointed out. I also cross checked with the sample submission. But I'm still getting a 0 score for some reason. Do you know what the problem might be?

Thanks in advance! Happy Learning :D

User avatar
CapitainData
UM6P

Hi fakejayduler,

Sorry for the lateness!

That is really strange! Because it worked by my side!

Here is the lines of code in my submission making cell:

# Make prediction on the test set

test_df = test_df[main_cols]

predictions = model.predict(test_df)

# Create a submission file

sub_file = ss.copy()

sub_file.target = predictions

# sub_file.target = sub_file.target.apply(lambda x: int(x))

# Check the distribution of your predictions

sns.countplot(sub_file.target);

And I got a score of : 0.3387....

Hey, thank you very much for the reply.

I am also doing the same thing. I reached out to team Zindi and they confirmed that my submission format is correct. Seems to be some problem with the host. Anyway I appreciate the reply.

Just one more question, the command where you convert the output (predictions) to int data type,

# sub_file.target = sub_file.target.apply(lambda x: int(x))

Is there a particular reason you used lamda function for the operation. Because,

sub_file.target = sub_file.target.astype(int)

This command also performs the same operation right.

Anyway, really appreciate the help. Hopefully they fix it soon.

Best of luck for the competition and happy learning! :D

Hey, were you able to come up with a solution? Because I'm having the same problem.

15 Apr 2021, 14:05
Upvotes 0
User avatar
University of lagos

Hi fakejaydulera!

If you are using the starter notebook, please pay attention to the constitution of the submission file!

instead of sub_file.predictions = predictions,

use sub_file.target = predictions.

Regards

User avatar
Kenyatta university

No not yet am still getting

User avatar
University of lagos

Have you ensured you're doing this

sub_file.target = predictions ### Ensure values in predictions are 0, 1, and not 0.2, 0.8

Also you mentioned getting a local f1 score of 0.9, that's really large, definitelly overfitting to a particular sample of the dataset.

Try to do cross validation maybe and also tune your model not to overfit.

Hi Dave, thank you for the reply. The format of my submission is correct (I reached out to team Zindi and they confirmed), and my predictions are binary, and not float point numbers like you correctly pointed out. But the score still shows 0. If the worst case scenario where my score was actually 0, I changed one value from 0 to 1 and submitted and I still got a score of 0 which should not be possible. I am running out of options now and don't know what to do.

Anyway, thank you for the reply, I appreciate it! Happy Learning :D

User avatar
CapitainData
UM6P

Here is a link towards the Python Starter Notebook working by my side: https://drive.google.com/file/d/1-iWt8J_u0jb7z4yfc8saxuGR2EbIpexY/view?usp=sharing

Regards