5 Nov 2019, 08:38

Meet the winners of the Wazihub Soil Moisture Prediction Challenge

Get insights from challenge winners! A special thank you to the winners for their generous feedback.

Zindi is excited to announce the winners of the Wazihub Soil Moisture Prediction Challenge. The objective of the competition was to create a machine learning model to predict the humidity for a particular plot in the next few days, using data from the past. A part of the challenge was to design algorithms that are resilient and can be trained with incomplete data (e.g. missing data points) and unclean data (e.g. lots of outliers).

This resulting model will enable farmers to anticipate water needs and prepare their irrigation schedules.

The challenge attracted 682 data scientists from across the continent and around the world, of whom 69 were on the leaderboard. We are happy to introduce the winner and two top-placed competitors who will share their strategies for solving this challenge - Olayinka Fadahunsi of Nigeria (1st place), Jasseur Abidi of Tunisia (4th place), and Sertac Ozker of South Africa (6th place)!

Name: Olayinka Fadahunsi (1st place)

Zindi handle: DrFad

Where are you from? Nigeria

Tell us a bit about yourself.

I am a Data Scientist at a financial institution in Nigeria who is passionate about solving real-world problems while expanding my skill sets.

Tell us about the approach you took

In solving this problem, I digested the problem statement and conducted some exploratory data analysis (EDA). The result of the EDA pointed me in the right direction to create features that gave a good representation of the model. Missing values were filled with averages of values around the missing cases.
The EDA showed interesting patterns and correlations between Crop Coefficient (Kc) & irrigation pattern with the target variable - soil humidity. The irrigation feature and the soil humidity level is analogous to filling a water tank and draining the tank. Once the irrigation is in the ON state, the soil humidity level rises, whilst in the OFF state, the soil humidity declines. Exceptions to this pattern as seen in the data are when features like Kc (Coefficient cultural), ETc (evapotranspiration rate) and ETo (Evapotranspiration reference) deviate from normal values. Cases of deviations could be attributed to rainfall or other environmental conditions. The next task was to model these patterns.
Feature engineering and feature extraction was the most difficult part of the challenge. While the patterns were obvious, it was difficult to create features that modeled the patterns. Leveraging data.table objects and dplyr package in R, I was able to create counter features, lag features and flow rate/drain rate per farm field.
It was observed that the longer the farm was left without irrigation, the faster the soil humidity declined. The period/length of irrigation was considered in creating a flow rate per farm field. I assumed a linear growth or decline in soil humidity. Future work could explore non-linear relationships to improve the model score.
Finally, I used an ensemble of 5 tree based models to achieve the winning solution.

What were the things that made the difference for you that you think others can learn from?

Visualization skills and the ability to program complex data manipulation ideas contributed significantly to my final score.
First, I spent a sizable portion of my time on Exploratory Data Analysis (EDA). This helped me understand the soil humidity trend for each of the farm fields. I could ascertain a strong correlation between Crop coefficient (Kc) & irrigation features with the target variable - soil humidity. After establishing the pattern, I embarked on a journey to create features to model the pattern.
Feature creation was the most difficult path. The patterns were obvious but difficult to represent by a feature. I had to learn and utilise data.table objects to achieve this. I created counter features, lag features and features to estimate the flow rate of soil humidity.

What are the biggest areas of opportunity you see for AI in Africa over the next few years?

I think that AI applied to farming will be big in the years to come. On Zindi, we have seen two challenges in 2019 dedicated to the Agriculture industry.
Another area that will experience rapid growth is the Financial industry. Banks and other financial institutions are turning to Data science and AI to solve problems such as financial inclusion, digital lending, smart credit scoring etc.

Watch DrFad's video here.

Name: Jasseur Abidi (4th place)

Zindi handle: jasseur

Where are you from? Tunisia

Tell us a bit about yourself.

I am a graduate from ENSTA. I have experience in Deep Learning and software development. I'm currently working as a Machine Learning Engineer at Vneuron.

Tell us about the approach you took.

My solution is inspired from controlled dynamic system models. I used LightGBM Regressor to approximate its dynamics.

What are the biggest areas of opportunity you see for AI in Africa over the next few years?

The biggest areas of opportunity are health, agriculture and logistics.

Name: Sertac Ozker (6th place)

Zindi handle: Sertac_Ozker

Where are you from? South Africa

Tell us a bit about yourself.

I am originally from Turkey but I'm currently living in Johannesburg, South Africa.

Tell us about the approach you took.

I'm a Senior Data Analyst with newly acquired machine learning skills. I have 10 years of IT experience in banking where I led DWH & BI teams.

What were the things that made the difference for you that you think others can learn from?

I focused on the change of the humidity for each data point. I've trained my model for each field twice since the results were changing a lot when the irrigation is on and off. I made a simple XGB Regressor with the data given.

What are the biggest areas of opportunity you see for AI in Africa over the next few years?

Everybody agrees that Africa will be a major powerhouse in the future since the continent has the youngest population (a huge one). The potential growth if achieved will transform the continent. And AI is going to be one of the core steps for this future. I believe AI will help people, especially in the mining and agricultural sectors and hopefully for the prevention of crime.

What are you looking forward to most about the Zindi community?

I want it to grow bigger. So every Zindian should bring people around him/her and also encourage to people to study Machine Learning and AI.

Watch Sertac_Ozker's video here.

This competition was hosted by Wazihub (www.wazihub.com) and sponsored by Microsoft (www.microsoft.com)

What are your thoughts on our winners' feedback? Engage via the Discussions page or leave a comment on social media.