How to Perform Data Augmentation in NLP Projects
Data skills · 3 Nov 2022, 11:18 · 6 mins read ·
6

A simple way to conduct Data Augmentation by using TextAttack Library

In machine learning, it is crucial to have a large amount of data in order to achieve strong model performance. Using a method known as data augmentation, you can create more data for your machine-learning project. Data augmentation is a collection of techniques that manage the process of automatically generating high-quality data on top of existing data.

In computer vision applications, augmenting approaches are extremely prevalent.If you are working on a computer vision project (e.g Image classification), for instance, you can apply dozens of techniques to each image: shift, modify color intensities, scale, rotate, crop, etc.

If you have a tiny dataset for your ML project or wish to reduce overfitting in your machine learning models, it is recommended that you may apply data augmentation approaches.

“We don’t have better algorithms. We just have more data.”- Peter Norvig

In the field of Natural Language Processing (NLP), the tremendous level of complexity that language possesses, makes it difficult to augment the text. The process of augmenting text data is more challenging and not as straightforward as some might expect.

In this article, you will learn how to use a library called TextAttack to improve data for natural language processing.

What is TextAttack?

TextAttack is a Python framework that was built by the QData team for the purpose of conducting adversarial attacks, adversarial training, and data augmentation in natural language processing. TextAttack has components that can be utilized independently for a variety of basic natural language processing tasks, including sentence encoding, grammar checking, and word substitution.

TextAttack excels in performing the following three functions:

1. Adversarial attacks (Python: textattack.Attack, Bash: textattack attack).

2. Data augmentation (Python: textattack.augmentation.Augmenter, Bash: textattack augment).

3. Model training (Python: textattack.Trainer, Bash: textattack train).

Note: For this article, we will focus on how to use the TextAttack library for Data augmentation.

How to Install TexAttack

To use this library make sure you have python 3.6 or above in your environment.

Run the following command to install textAttack.

pip install textattack

Note: Once you have installed TexAttack, you can run it via the python module or via command-line.

Data Augmentation Techniques for Text Data

TextAttack library has various augmentation techniques that you can use in your NLP project to add more text data. Here are some of the techniques that you can apply:

1. CharSwapAugmenter

It augments words by swapping characters out for other characters.

from textattack.augmentation import CharSwapAugmenter
text = "I have enjoyed watching that movie, it was amazing."
charswap_aug = CharSwapAugmenter()
charswap_aug.augment(text)
[‘I have enjoyed watching that omvie, it was amazing.’]

The Augmenter has swapped the word “movie” to “omvie”.

2. DeletionAugmenter

It augments the text by deleting some parts of the text to make new text.

from textattack.augmentation import DeletionAugmenter
text = "I have enjoyed watching that movie, it was amazing."
deletion_aug = DeletionAugmenter()
deletion_aug.augment(text)
[‘I have watching that, it was amazing.’]

This method has removed the word “enjoyed” to create a new augmented text.

3. EasyDataAugmenter

This augments the text with a combination of different methods, such as

  • Randomly swap the positions of the words in the sentence.
  • Randomly remove words from the sentence.
  • Randomly insert a random synonym of a random word at a random location.
  • Randomly replace words with their synonyms.
from textattack.augmentation import EasyDataAugmenter
text = "I was billed twice for the service and this is the second time it has happened"
eda_aug = EasyDataAugmenter()
eda_aug.augment(text)
[‘I was billed twice for the service and this is the second time it has happen’,
‘I was billed twice for the one service and this is the second time it has happened’,
‘I billed twice for the service and this is the second time it has happened’,
‘I was billed twice for the this and service is the second time it has happened’]

As you can see from the augmented texts, it shows different results based on the methods applied. For example in the first augmented text, the last word has been modified from “happened”to “happen”.

4. WordNetAugmenter

It can augment the text by replacing it with synonyms from the WordNet thesaurus.

from textattack.augmentation import WordNetAugmenter
text = "I was billed twice for the service and this is the second time it has happened"
wordnet_aug = WordNetAugmenter()
wordnet_aug.augment(text)
[‘I was billed twice for the service and this is the second time it has pass’]

This method has changed the word “happened” to “pass” in order to create a new augmented text.

5. Create your Own Augmenter

Importing transformations and constraints from textattack.transformations and textattack.constraintsallows you to build your own augmenter from the ground up. The following is an illustration of the use of the WordSwapRandomCharacterDeletionalgorithm to produce augmentations of a string:

from textattack.transformations import WordSwapRandomCharacterDeletion
from textattack.transformations import CompositeTransformation
from textattack.augmentation import Augmenter
my_transformation = CompositeTransformation([WordSwapRandomCharacterDeletion()])
augmenter = Augmenter(transformation=my_transformation, transformations_per_example=3)
text = 'Siri became confused when we reused to follow her directions.'
augmenter.augment(text)
[‘Siri became cnfused when we reused to follow her directions.’,
‘Siri became confused when e reused to follow her directions.’,
‘Siri became confused when we reused to follow hr directions.’]

The output shows different augmented texts after implementing theWordSwapRandomCharacterDeletionmethod. For example, in the first augmented text, the method randomly removes the character “o” in the word “confused”.

Conclusion

In this article, you have learned the significance of data augmentation for your Machine Learning project. In addition, you have learned how to execute data augmentation for textual data using the TextAttack library.

You can also try to use other available augmentation techniques from the TextAttack library such as:

  • EmbeddingAugmenter
  • CheckListAugmenter
  • CLAREAugmenter

Some questions for you Zindi reader

1. How do you think EasyDataAugmenter would do on African/Indigenous languages? EasyDataAugmenter randomly replaces words with their synonyms.

2. Do you know of a source similar to WordNet you can use for Augmentation?

3. If you create your own Augmenter share it below so others can learn and expand on it.

About the author

Davis David is a Zindi ambassador for Tanzania and a data scientist at Dlab. He is passionate about artificial intelligence, machine learning, deep learning and big data. He is a co-organizer and facilitator of the AI movement in Tanzania; conducting AI meetups, workshops and events with a passion to build a community of data scientists to solve local problems.

Back to top
If you enjoyed this content upvote this article to show your support
Discussion 1 answer

Amazing ! 🤩