Machine Learning Hackathon by AMLD Africa
'/%3e%3cpath d='m450 224.315-44.467 136.87 116.401-84.559h-143.87l116.403 84.559z' style='fill:none;stroke:%23006233;stroke-width:14.62993431;stroke-miterlimit:4;stroke-dasharray:none'/%3e%3c/svg%3e)
Helping Morocco
$1 000 USD
Challenge completed ~4 years ago
Natural Language Processing
Classification
171 joined
78 active
Start
Aug 27, 21
Close
Sep 05, 21
Reveal
Sep 06, 21
'%3e%3cpath fill='%23001489' d='M0 0v6h9V0z'/%3e%3cpath fill='%23e03c31' d='M0 0v3h9V0z'/%3e%3cg stroke='%23fff' stroke-width='2'%3e%3cpath id='d' d='m0 0 4.5 3L0 6m4.5-3H9'/%3e%3cuse xlink:href='%23b' stroke='%23ffb81c' clip-path='url(%23c)'/%3e%3c/g%3e%3cuse xlink:href='%23d' fill='none' stroke='%23007749' stroke-width='1.2'/%3e%3c/g%3e%3c/svg%3e){kind=small}
Text classification with Simple Transformers
23 Sep 2021, 10:17 · 2
Hi, here's a sample notebook for using the Simple Transformers library for text classification: https://github.com/FaatimahM/Tweet_classification_simpletransformers. This is by no means pro level, I'm relatively new to data science and programming in Python and I literally learnt about using this library when I enrolled in this hackathon. I hope that this will be useful to someone on their data science journey :)
hey @Mansoor.. thanks for sharing this
I'd like to know how did you handle all the unknowns in the dataset, and even violence phrases??
Also I noticed there was more than 1 language in the dataset, did you treat this as well or you fed them into the model?
Hey @ZzyZx using these models I found that applying no pre-processing produced the best results, i.e. feeding in the raw data into the model. The pre-processing which I tried for cleaning the text was removing 'RT', '&', '', '' and '#'. I chose not to remove stop words because I read somewhere that they may contain important info needed by the language models.
Just a general question, can anyone please advise what metrics/methods should be looked at when trying to interpret what a model is actually doing