Hate Speech Detection in Social Media

Abstract

The objective of our research is to detect hate speech on social media. Every day a huge amount of data is generated by the users of different social media. For this research we created a data set collecting data from twitter. This data set consists of tweets of different kind of people of different race and religion.

In this work we followed machine learning approach and as we know NB and SVM is most popular algorithm for sentiment analysis and classifying text, so we used Naïve Bayes and Support Vector Machine algorithm in this work. While using NB we find accuracy rate at 94.63% and in SVM the accuracy rate was 92.32%. As the action of particular event of social media not only bounded only in the internet it effects the real life events all well. Again anything spread faster on social media with compare of different other media. Many people post many hatred things of social media and it hurts others feeling and then difficulties arrives and people have to face further consequence. By detecting hate speech we can control this things and avoid this kind of situations. So our work have value to keep social media free from few bad things and conflict between people of different believes.

Introduction

If we look at the statics, we can clearly visualize that the number of people using social media increasing at great speed. Every day millions of people joining social media like Facebook, Twitter, and Instagram. This means that social media has become a very important communication medium today. By using social media technology people can send the message very quickly, so a particular topic can spared very quickly. Unfortunately, that also same for hate speech, it can also spread very fast. That can be matter of conflicts among different groups of our society. According to Cambridge Dictionary “Hate speech is a public speech that expresses hate or encourages violence towards a person or group based on something such as race, religion, sex, or sexual orientation “ [1]. By those criteria hate speech is defined is varied from one place to another place according to the globe. As we see in America there most of the inconvenient activity occurs on based issues like anti-American African, anti-immigrant, anti-White etc. On the contrary country like India most of the hate speech delivered on the basis of religion, cast, political views.

Right now it is very important having control over this huge scale of data on social media. There are several work on different methodology done to detect hate speech using data of social media like twitter, facebook or other sites. There two method popular among one is word bag method, where a data set is created consist of hate word. And another approach is machine learning method. Machine leaning is used in different field like Business data analysis, prediction system, recommendation system, speech or handwriting recognition, bioinformatics, sentiment analysis and more [2]. They uses of machine learning seems very efficient and successful throughout the days on several field including which just mentioned. Using of machine leaning and data mining technique is a great option for deal with enormous collection of data.  Where different machine learning algorithms is used to serve those purpose. There are different machine leaning algorithms like SVM, Naïve Bayes, Random Forest, Decision tree etc.

In our research we use machine learning techniques to detect hate speech and we applied SVM and Naïve Bayes algorithms and show the comparison of their performance. We collected data from twitter. For annotate a tweet we consider different factor like hate against other religion, race, ethnicity, political view, tradition, gender, sexual orientation. In this article we describe the process of our work in different section.

Literature Review

Several works have worked on hate speech detection. In this section we will discuss about few previous work and their outcome. In [3] Warner done a brief discussion on hate speech, give a clear concept about hate speech. They proposed a method of detecting hate speech. They collect data from the American Jewish Congress (AJC) and Yahoo. They pick those texts which marked as offensive by those sites readers and found as hate speech according their definition. After analyzing they divided the data by their stereotype, like anti-Hispanic, anti-African American, anti-immigrants, anti-white, anti-sematic language. They build classifier for each stereotype. For classification template-based strategy is used in this work. They used SVM classifier with linear carnal and perform 10 fold cross validation for each classifier. After classification performance and error report is represent in two different tables. The overall accuracy was 96%.

In [4] Waseem collected about 16 thousand tweets and annotates them. They analyses the features for finding the features which can output the best performance and improve hate speech detection.  They defines 13 conditions, where a tweet will be consider hate speech if it fulfil one or more pre-defined conditions. They test the impact of various feature on the performance using logistic regression with 10 fold cross validation. For model selection they uses grid search over all possible features. For each tweet n-gram is collected where value of n is 1, 2, 3, 4. Difference between one n-gram to another n-gram is found in their result.

Chen [5] analysis previous methods of text mining. They proposed an approach named Lexical Syntactical Feature (LSF) which may use for detecting offensive text as well as predict the potential of a user use of abusive content. According to their research combining context-specific features, structure features and style features with lexical features may improve existing machine learning methods and prediction of offensiveness.

A.H [6] used total 1525 massages from few group massaging services. Among them they annotated 68% of data as “OK” means not offensive and 32 percent as offensive. While they consider criteria like Slurs, Racism, Homophobia, Extremism, Crude language, Provocative language, Taboos, Unrefined language. Where 10 percent of data used for test and rest for training set. After processing data they gone three level classification. They uses Weke software for classification task. They first run few classifier and consider different criteria best classifier with high performance is chosen for classification. In three level they used Complement Naïve Bayes classifier, Multinomial Updatable Naïve Bayes classifier and DTNB (Decision Table/Naive Bayes hybrid classifier respectively. In those classifier level they percentage of correctly classified is varied from 84.26% to 96.72%.

Proposed Methodology

In this model first we collect data form twitter which consider as data source of this research. After that we have to gone through data preprocessing process. Then we label the data manually with the merit of the tweets. Then we divided data into two part train and test. Where test part stay for testing and we train the train set .Then we used different classifier for our selected algorithms. By which the result is generated .Those process are described in details in the next sections bellow.

A. Creating the Dataset

For the preparation of dataset we follow few steps. Each step we perform certain task like collecting and labeling tweets for dataset, preprocess them. Bellow those steps are discussed.

1. Data Collection

For collecting data for this work we chose twitter as data source. From twitter data can be collected form twitter API.  For this need to follow few steps. First of all need to create a twitter developer account, for this need to apply and tell them what exactly going to do with this account and what is the purpose of collecting data form twitter. Then when the application approved, access token secret is available and with this tweets from particular topic as requirement can be collected.

2. Labeling Data

Collected tweets are consisting both hate and non- hate speech. Those tweets are manually labeled into two categories. Where tweets which seems hateful labeled as “hate tweets” and other tweets consider as “non-hate tweets”. Where hate tweets given value of 1 and non-hate tweets given value of 0.

3. Preprocessing

Preprocessing task of collected data done in five steps,

a. Cleaning Data: Tweets often have many numeric values and special characters like ‘#’, ‘*’, ‘&’ etc. those does not had anything to do with the meaning of a sentence. So while preprocessing those are removed. User something share link in tweets, those links also remove while preprocessing.

b. Lowercasing: Every uppercase character of a tweet converted to lowercase. Like the sentence “OPEN LOCKS, Whoever knock” will be “open locks, whoever knock” after lowercasing.

c. Stemming and Lemmatization: Purpose of both stemming and lemmatization is to detract inflectional forms and sometimes derivationally related forms of a word to a common base form [7]. Where stemming just chop of certain part of a word on the other hand lemmatizing convert the word to its base form by dealing with vocabulary and morphological analysis of words. Lemmatization is more accurate, so we use lemmatization in the work. This task is done using NLTK provided lemmatizer, NLTK is a python based platform.

d. Correcting Spell: There is many words in tweets which in spelled incorrectly. It may cause dissimilarity to the written word to its actual word. For this a misspelled word converted to its closest possible form. Example: “Deprasion” > “Depression”.

e. Stop word removal: Generally there is many words in a sentence which does not make much sense to the actual sentiment of the sentence. Those are called stop word. In this step all stops are remove. As example, after removing stop word “This book is very nice” will be “book very nice” here “This” and “is” removed as they are stop word. We use NLTK corpus for this task.

B. Hate Speech Detection

1. Features Extraction

After gathering large tweet corpus, we have built and train classifier for tweet sentiment analysis. We examine mainly two classifiers: Naïve Bayes and Support Vector Machine. For each classifier we extract the same features from the tweets to classify on it .To build feature set, we process each tweet and extract meaningful feature and create feature matrix by union of unigram and trigram. For example, if hate tweet contains word “fool”, a feature for classification would be whether or not a tweet contains the word “fool”.

2. Training Module

The data is used as training dataset to train the model for sentiment analysis. On inspecting the model on test dataset, we receive the tweet sentiment labels as an output. We will use this dataset for detecting hate speech. Major problem in machine learning is that there exist a chance of duel meaning of same word which convey different meanings in different context that may confuse the statistical engines. This may sometimes provide wrong analysis in the sentiments of particular tweet.

3. Classification and Evaluation

We used machine learning techniques for detection of hate speech .In the work we compared the performance of NB, SVM algorithm on our dataset. Classification of text: Given a suitably annotated collection of data. The classification model relate the feature in to give data to any of the classification label. Such labels can be either 1 for hate speech or 0 for non-hate speech. Then given a suitably without label collection of document, the model predict a classification label for it. The supervised learning algorithm depends on the labeled training data. The training data contain a collection training examples. Each example in supervised learning consist of an input object and output value. At the very last the performance of classifier is measured using confusion matrix.

Result and Conclusion

In this research, we built a collection of tweets on English language for hate speech detection and conducted a comparative study among the performance of several features and machine learning algorithms. We manually labeled the tweets into two categories hate tweets and non-hate tweets. The resulting dataset had more than 560 tweets, consists of both hate tweets non-hate tweets. Based on the experimental results, The Naïve Bayes shows accuracy of 94.63% and in SVM the accuracy rate was 92.32%. The results also showed that using word combination of bigram or trigram it was better than only bigram or only trigram. We found out that NB performance was better than SVM. For the future work on hate speech detection in the Bengali language.

References

[1] Cambridge Dictionary, "hate speech," Cambridge Dictionary, [Online]. Available: https://dictionary.cambridge.org/dictionary/english/hate-speech. [Accessed 23 09 2019].

[2] Daffodil Software, "9 Applications of Machine Learning from Day-to-Day Life," Medium, 31 07 2017. [Online]. Available: https://medium.com/app-affairs/9-applications-of-machine-learning-from-day-to-day-life-112a47a429d0. [Accessed 23 09 2019].

[3] J. H. William Warner, "Detecting Hate on the World Wide Web," in Conference: Proceedings of the Second Workshop on Language in Social Media, 2012.

[4] D. H. Zeerak Waseem, "Hateful Symbols or Hateful People? Predictive Features for Hate Speech Detection on Twitter," in Conference: Proceedings of the NAACL Student Research Workshop, 2016.

[5] Y. Z. S. Z. H. X. H. X. Ying Chen, "Detecting Offensive Language in Social Media to Protect Adolescent Online Safety," in Privacy, Security, Risk and Trust (PASSAT), 2012 International Conference on and 2012 International Confernece on Social Computing (SocialCom), 2012.

[6] A. H. R. I. U. Matwin, "Offensive Language Detection Using Multi-level Classification," in Canadian Conference on Artificial Intelligence, 2010.

[7] P. R. a. H. S. Christopher D. Manning, Introduction to Information Retrieval, Cambridge University Press., 2008.

[8] Z. M. a. J.-H. Wang, "Vulnerable community identification using hate speech detection on social media," Information Processing and Management, vol. 57, no. 1, p. 16, 2019.