Intro to ASR Workshop & Challenge:

In this 2-month challenge, we invite participants to create a speech corpus in an African language and train a speech recognition system on the data they collected. We will host a live workshop to cover key topics in NLP and ASR (please see details in Workshop section) and after the workshop, we'll provide a walkthrough video tutorial showing how to train a model using ELPIS (please see Data tab).

Participants are encouraged to use the ASR data collection app from LIG-Aikuma, which allows to add text prompts on your phone and record them yourself or ask others to record them. While the LIG-Aikuma app only runs on Android, it is not a requirement for this challenge to use it. Participants may use any other collection methods as long as they are free and available to the public or created by yourself.

Update: Some are having issues with the Lig-Aikuma app crashing. The Lig-Aikuma app was suggested because it was open source and designed with ASR collection in mind, but you're welcome to use any speech recording app you'd like (even if it's not open source, as long as it's free and you're not paying for any of the features).

Here is another open source app, Simple Voice Recorder: https://play.google.com/store/apps/details?id=com.simplemobiletools.voicerecorder

And here is a list of other speech recorder apps (you're not limited to this list, you can use anything that's free and publicly available). You may want to try a few out to see which you like best: https://www.rev.com/blog/best-voice-recorder-apps-for-android

Tip - When figuring out which app you'd like to use, some things you might want to consider are: Do I need the app to work offline and if so, can it work offline? Where does it store the data (do you want to store it to phone or SD card, and then transfer to computer or online storage)?

About Google (research.google)

Our goal in Google Research is to make Google's technologies, including automatic speech recognition (ASR) technologies, work great for everyone on the planet. Our team is a mix of linguists, program managers, engineers and researchers, addressing many exciting and unprecedented internationalization challenges. We have a huge commitment to the diversity of our users, and have made it a priority to deliver the best performance to every language on the planet. We currently have systems operating in more than 71 languages, and we continue to expand our reach to more users.

Speech technologies can enable interaction with services and applications that would be difficult to acquire and use via means other than voice. Automatic Speech Recognition (ASR) systems are revolutionizing the possibilities in this area but are not yet widely available in African languages. Although there have been many advances in ASR modeling and data collection methodologies in recent years, the greatest problem continues to be the requirement for large amounts of voice data.

The aim of this challenge is to collect written and spoken language data to create a speech corpus that can be used to train an ASR model in a beginner-friendly way. This challenge is a part of the Google NLP Hack Series, which starts with an ASR workshop (more info below).

In addition to increasing awareness of speech technologies, this hack series will raise awareness and appreciation for good quality, clean datasets in local languages, generate datasets in African languages that are open sourced, and lower the entry barrier for participants to run their own ASR models.

Workshop sessions

Date: February 5th, 2022 (13:00-17:00 WAT)

The workshop was live streamed and the recording is now available linked on the Data tab.

To kick off this NLP Hack Series, we will be hosting a live event which will include an introduction from host organizations, a workshop that will cover key topics in NLP and ASR, guidance on what makes a good dataset, and an overview of ASR model training. After the workshop, we will also provide a walkthrough video tutorial of model training using ELPIS on the Zindi YouTube channel.

You will be invited to the workshop via a calendar event using your registration email. For those who cannot attend the live event, the workshop will be recorded and later uploaded to Zindi's YouTube channel.

Challenge: This challenge calls on you to submit datasets you create and models for automatic speech recognition. Please focus on African languages. Small and indigenous African languages are encouraged.

How submissions will be evaluated: The datasets will be evaluated by an expert committee and will take into consideration the following criteria: diversity of speakers and prompts, size and uniqueness of the dataset, quality of documentation. If you’d like more guidance, please see the “What makes a good dataset” section in the workshop on February 5th (workshop video on Data tab). (Optional) ASR model bonus: The inclusion of an ASR model (that you train on the data you collected) is offered as a small “bonus point” opportunity. You are encouraged, but not required to submit an ASR model as part of your submission. To help you prioritize, please focus on the rest of the submission deliverables (dataset and related files). This bonus opportunity serves more as a tie breaking differentiator - if two dataset submissions are tied, then the submission that came with an ASR model would rank over the other submission. If you choose to include an ASR model, please know that judges will not look at the performance of your ASR model, they’ll only check Yes/No if a working model was included. (This is because there’s no way to compare model performance when your training data differs. Judges just want to see that you demonstrated the ability to train a model on your data.) Here’s a video on how to train an ASR model in a beginner-friendly way.

Dataset collection tips:

• To generate text prompts in your target language, you can use any existing open source text corpora, or use web scraping techniques to build your own text corpus. Remember it’s important they cover a range of different topics.
• Recordings should include speakers of various ages and genders, who must sign a consent form to participate.
• If the language varies from region to region, ideally there should be a representation of as many variants as possible.
• Since we can’t always expect the ASR systems to be used in quiet conditions, it would also be helpful to have some recordings made in noisy environments.

Submission Guidelines

For your submission to be eligible, the data and models must meet the following criteria:

• Submissions in appropriate form and appropriate file names (see Files/Formatting section below)
• Our intention is that the datasets are kept free and open for public use under a Creative Commons license 4.0 or similar. Data already licensed under more restrictive terms will not be eligible. You must be the collector/creator of the dataset.
• Each language you collect data for should be its own dataset / its own submission. Ex: if you collect data in Oromo and Igbo, they should be 2 separate submissions as two zip files. If you make multiple submissions, your submissions will be judged independently of each other.

Deliverables Checklist:

Each submission (1 language = 1 submission) should be a zipped folder containing 3 parts: Dataset, Metadata, and Documentation. And an optional 4th part if you choose to include an ASR model as bonus.

• 1) Dataset files: transcription + audio file pairs (the file pairs should be named the same, for example: file1.txt and file1.wav)
• 2) Metadata spreadsheet: this is a spreadsheet with metadata about your dataset, please use the example metadata template provided on the Data tab.
• 3) Documentation: each dataset submission must be accompanied by a write up that documents at least the A,B,C,D,E criteria below. You can find examples of documentation here, alongside a playbook to help you build your documentation. A) the language (region, dialect if any), B) the ISO code of the language, C) who collected the data, D) info about its motivation, composition, collection process (where/how), source of prompts, recommended uses/applications, and so on. E) written confirmation from you that you gathered informed consent for all recorded speakers. IMPORTANT NOTE: please do not submit the consent forms themselves as they contain personal information!
• 4) (Optional, bonus) Model: a model that has been trained on the dataset you collected. We will not measure the performance of the model - we are only looking to see if a trained model is included as a yes/no completion criteria. You are not required to include a model with your submission.

Files / Formatting

• Each submission to the competition must be uploaded as a single .zip file. In the .zip file please include all relevant submission files with your responses (.csv, .shp, .pkl, .zip files accepted).
• Please label your files: “username_language.zip”. For example, if someone collected datasets for Igbo and Twi, they would submit two submissions as two zip files named: “connie_igbo.zip” and “connie_twi.zip”

Confirmation: Note that there will be no scores on this leaderboard. If you’d like to double check that your submission was complete with all the necessary pieces, you can send a message to amyflorida626.

1st place: $600

2nd place: $500

3rd place: $400

$300 each for regional winner from: Nigeria, South Africa, Ghana, Benin and Ivory Coast.

Prizes will be awarded in this order.

This challenge closes on 17 April 2022.

Final submissions must be received by 11:59 PM GMT.

We reserve the right to update the contest timeline if necessary.

Teams and collaboration

You may participate in competitions as an individual or in a team of up to four people. When creating a team, the team must have a total submission count less than or equal to the maximum allowable submissions as of the formation date. A team will be allowed the maximum number of submissions for the competition, minus the total number of submissions among team members at team formation. Prizes are transferred only to the individual players or to the team leader.

Multiple accounts per user are not permitted, and neither is collaboration or membership across multiple teams. Individuals and their submissions originating from multiple accounts will be immediately disqualified from the platform.

Code must not be shared privately outside of a team. Any code that is shared, must be made available to all competition participants through the platform. (i.e. on the discussion boards).

The Zindi user who sets up a team is the default Team Leader. The Team Leader can invite other data scientists to their team. Invited data scientists can accept or reject invitations. Until a second data scientist accepts an invitation to join a team, the data scientist who initiated a team remains an individual on the leaderboard. No additional members may be added to teams within the final 5 days of the competition or the last hour of a hackathon, unless otherwise stated in the competition rules

A team can be disbanded if it has not yet made a submission. Once a submission is made individual members cannot leave the team.

All members in the team receive points associated with their ranking in the competition and there is no split or division of the points between team members.

Submissions and winning

You may make a maximum of 2 submissions per day.

You may make a maximum of 10 submissions for this competition.

There is no public/private leaderboard.

If your solution places 1st, 2nd, or 3rd or is a regional winner, you will be required to open source your work on your GitHub under a CC-BY SA 4.0 license. We encourage all participants to open source their work as a public good to the sector.

If two solutions earn identical score, the tiebreaker will be the date and time in which the submission was made (the earlier solution will win).

The winners will be paid via bank transfer, PayPal, or other international money transfer platform. International transfer fees will be deducted from the total prize amount, unless the prize money is under $500, in which case the international transfer fees will be covered by Zindi. In all cases, the winners are responsible for any other fees applied by their own bank or other institution for receiving the prize money. All taxes imposed on prizes are the sole responsibility of the winners. The top 3 winners or team leaders will be required to present Zindi with proof of identification, proof of residence and a letter from your bank confirming your banking details. Winners will be paid in USD or the currency of the competition. If your account cannot receive US Dollars or the currency of the competition then your bank will need to provide proof of this and Zindi will try to accommodate this.

Payment will be made after code review and an introductory call with the host.

You acknowledge and agree that Zindi may, without any obligation to do so, remove or disqualify an individual, team, or account if Zindi believes that such individual, team, or account is in violation of these rules. Entry into this competition constitutes your acceptance of these official competition rules.

Zindi is committed to providing solutions of value to our clients and partners. To this end, we reserve the right to disqualify your submission on the grounds of usability or value. This includes but is not limited to the use of data leaks or any other practices that we deem to compromise the inherent value of your solution.

Zindi also reserves the right to disqualify you and/or your submissions from any competition if we believe that you violated the rules or violated the spirit of the competition or the platform in any other way. The disqualifications are irrespective of your position on the leaderboard and completely at the discretion of Zindi.

Please refer to the FAQs and Terms of Use for additional rules that may apply to this competition. We reserve the right to update these rules at any time.

Consequences of breaking any rules of the competition or submission guidelines:

• First offence: No prizes for 6 months and 2000 points will be removed from your profile (probation period). If you are caught cheating, all individuals involved in cheating will be disqualified from the challenge(s) you were caught in and you will be disqualified from winning any competitions for the next six months and 2000 points will be removed from your profile. If you have less than 2000 points to your profile your points will be set to 0.
• Second offence: Banned from the platform. If you are caught for a second time your Zindi account will be disabled and you will be disqualified from winning any competitions or Zindi points using any other account.
• Teams with individuals who are caught cheating will not be eligible to win prizes or points in the competition in which the cheating occurred, regardless of the individuals’ knowledge of or participation in the offence.
• Teams with individuals who have previously committed an offence will not be eligible for any prizes for any competitions during the 6-month probation period.