Sentiment Analysis using Deep Learning

Introduction

The growth of the internet due to social networks such as facebook, twitter, Linkedin,  instagram etc. has led to significant users interaction and has empowered users to express their opinions about products, services, events, their preferences among others. It has also provided opportunities to the users to share their wisdom and experiences with each other. The faster development of social networks is causing explosive growth of digital content. It has turned online opinions, blogs, tweets, and posts into a very valuable asset for the corporates to get insights from the data and plan their strategy. Business organizations need to process and study these sentiments to investigate data and to gain business insights(Yadav & Vishwakarma, 2020). Traditional approach to manually extract complex features, identify which feature is relevant, and derive the patterns from this huge information is very time consuming and require significant human efforts. However, Deep Learning can exhibit excellent performance via Natural Language Processing (NLP) techniques to perform sentiment analysis on this massive information. The core idea of Deep Learning techniques is to identify complex features extracted from this vast amount of data without much external intervention using deep neural networks. These algorithms automatically learn new complex features. Both automatic feature extraction and availability of resources are very important when comparing the traditional machine learning approach and deep learning techniques(Araque et al., 2017).  Here the goal is to classify the opinions and sentiments expressed by users. In this article, we will discuss about various sentiment analysis techniques and several ensemble models to aggregate the information from multiple features.

Abstract

Deep Learning uses powerful neural network algorithms to mimics the way human brain process data for translating languages, recognizing speech, detecting objects and making decisions. Deep Learning algorithms are able to identify and learn the patterns from both unstructured and unlabeled data without human intervention.  Deep Learning techniques learn through multiple layers of representation and generate state of the art predictive results. In the past years, Deep Learning techniques have been very successful in performing the sentiment analysis. It provides automatic feature extraction, rich representation capabilities and better performance than traditional feature based techniques. These long-established approaches can yield strong baselines, and their predictive capabilities can be used in conjunction with the arising deep learning methods(Preethi et al., 2017). Two techniques of neural networks are very common – Convolutional Neural Networks(CNN) for image processing and Recurrent Neural Networks (RNN) – for natural language processing (NLP) tasks(Goularas & Kamis, 2019). Deep Learning is used to optimize the recommendations depending on the sentiment analysis performed on the different reviews, which are taken from different social networking sites. The Experiments performed indicate that the RNN based Deep-learning Sentiment Analysis (RDSA) improvises the behavior by increasing the accuracy of the sentiment analysis, which in turn yields better recommendations to the user and thus helps to identify a particular position as per the requirement of the user need(Preethi et al., 2017). In this article, we will discuss popular deep learning models which are increasingly applied in the sentiment analysis including CNN, RNN, various ensemble techniques. This article provides insights on various techniques for sentiment analysis.

Sentiment analysis

It is a set of techniques / algorithms used to detect the sentiment (positive, negative, or neutral) of a given text. It is a very powerful application of natural language processing (NLP) and finds usage in a large number of industries. It refers to the use of NLP, text analysis, computational linguistics, and biometrics to systematically identify, extract, quantify, study different states and subjective information. The sentiment analysis sometimes goes beyond the categorization of texts to find opinions and categorizes them as positive or negative, desirable or undesirable. Below figure describes the architecture of sentiment classification on texts. In this, we modify the provided reviews by applying specific filters, and we use the prepared dataset by applying the parameters and implement our proposed model for evaluation(Ghorbani et al., 2020).

There are three approaches to perform sentiment analysis –

1. Lexicon based techniques – It can be classified in two types –
   1. Dictionary based – In this approach, classification is done by using dictionary of terms, which can be found in WordNet or SentiWordNet.
   2. Corpus based – In this approach, classification is done based on the statistical analysis of the content of group of documents using techniques such as hidden Markov models (HMM) , conditional random field (CRF), k-nearest neighbors (k-NN) among others.
2. Machine learning based techniques – It can be classified in two groups –
   1. Traditional Models – It refers to classical techniques of machine learning such as support vector machines , maximum entropy classifier, naive Bayes classifier. The inputs of these models includes sentiment lexicon based features, lexical features, parts of speech, adverbs and adjectives.
   2. Deep Learning Models – It provides more accurate results than traditional models. It includes models such as CNN, RNN, and DNN. These models address classification problems at document level, sentence level or aspect level.
3. Hybrid Approach – It combine machine learning and lexicon based approaches. Sentiment lexicons plays a significant role within most of these approaches. Below figure illustrates taxonomy of deep-learning-based methods for sentiment analysis techniques. Sentiment analysis, whether performed by means of deep learning or traditional machine learning, requires that text training data be cleaned before being used to induce the classification(Dang et al., 2020).

Deep Learning

Deep Learning leverages multilayer approach to the hidden layers of neural networks. Traditionally, in machine learning models, features are identified and extracted either manually or using feature selection methods. However, in the case of Deep Learning, features are learned, extracted automatically resulting in higher accuracy and performance. Below figure shows the differences in sentiment polarity classification between the two approaches: traditional machine learning (Support Vector Machine (SVM), Bayesian networks, or decision trees) and deep learning. Artificial neural networks and deep learning currently provide the best solutions to many problems in the fields of image and speech recognition, as well as in natural language processing(Ghorbani et al., 2020).  Below figure illustrates differences in sentiment polarity classification between the two approaches: traditional machine learning (Support Vector Machine (SVM), Bayesian networks, or decision trees) and deep learning techniques.

Key Deep Learning techniques, which can be used, are listed below –

Convolution Neural Networks (CNN) – It is a class of deep neural networks, most commonly used to analyze visual imagery. They are also known as space invariant or shift invariant artificial neural networks, due to shared-weights architecture and translation invariance characteristics. CNN consists of an input and an output layer, as well as multiple hidden layers. The hidden layers of a CNN typically consist of a series of convolutional layers that convolve with a multiplication or other dot product. The activation function is commonly a RELU layer, and is subsequently followed by additional convolutions such as pooling layers, fully connected layers and normalization layers, referred to as hidden layers because their inputs and outputs are masked by the activation function and final convolution. Below is the deep architecture using a 10-layer convolutional neural network. Starting from the inputs, this model consists of three conv-pool stages with a convolution and max-pooling each, one flatten layer, two fully-connected layers, and one softmax layer for outputs(Wang & Fey, 2018).

Deep Neural Networks (DNN) – It is an artificial neural network (ANN) with multiple layers between the input and output layers. It finds the correct mathematical manipulation to turn the input into the output, whether it be a linear relationship or a non-linear relationship.

Recurrent Neural Networks – A recurrent neural network (RNN) is a class of artificial neural networks where connections between nodes form a directed graph along a temporal sequence. This allows it to exhibit temporal dynamic behavior. The main function of RNN is the processing of sequential information on the basis of the internal memory captured by the directed cycles. Unlike traditional neural networks, RNN can remember the previous computation of information and can reuse it by applying it to the next element in the sequence of inputs. A special type of RNN is long short-term memory (LSTM), which is capable of using long memory as the input of activation functions in the hidden layer. Below figure illustrates the architecture of LSTM architecture. In this figure, input data is preprocessed to reshape the data for the embedding matrix, next layer is the LSTM and the final layer is fully connected layer for text classification(Dang et al., 2020).

Conclusion

In this article, we discussed the core of deep learning models and the techniques that can be applied to sentiment analysis for social network data. We discussed about various approaches for sentiment analysis including machine learning based, lexicon based and hybrid model. The architectures of CNN, DNN and LSTM are discussed. It is better to combine deep learning techniques with word embedding when performing a sentiment analysis. Also, the effectiveness of the algorithms is largely dependent on the characteristics of the datasets, hence the convenience of testing deep learning methods with more datasets is important in order to cover a greater diversity of characteristics.

Bibliography

Araque, O., Corcuera-Platas, I., Sánchez-Rada, J. F., & Iglesias, C. A. (2017). Enhancing deep learning sentiment analysis with ensemble techniques in social applications. Expert Systems with Applications, 77, 236–246. https://doi.org/https://doi.org/10.1016/j.eswa.2017.02.002

Dang, N. C., Moreno-García, M. N., & De la Prieta, F. (2020). Sentiment Analysis Based on Deep Learning: A Comparative Study. Electronics, 9(3), 483. https://doi.org/10.3390/electronics9030483

Ghorbani, M., Bahaghighat, M., Xin, Q., & Özen, F. (2020). ConvLSTMConv network: a deep learning approach for sentiment analysis in cloud computing. Journal of Cloud Computing, 9(1), 16. https://doi.org/10.1186/s13677-020-00162-1

Goularas, D., & Kamis, S. (2019). Evaluation of Deep Learning Techniques in Sentiment Analysis from Twitter Data. 2019 International Conference on Deep Learning and Machine Learning in Emerging Applications (Deep-ML), 12–17. https://doi.org/10.1109/Deep-ML.2019.00011

Preethi, G., Krishna, P. V, Obaidat, M. S., Saritha, V., & Yenduri, S. (2017). Application of Deep Learning to Sentiment Analysis for recommender system on cloud. 2017 International Conference on Computer, Information and Telecommunication Systems (CITS), 93–97. https://doi.org/10.1109/CITS.2017.8035341

Wang, Z., & Fey, A. M. (2018). Deep learning with convolutional neural network for objective skill evaluation in robot-assisted surgery. International Journal of Computer Assisted Radiology and Surgery, 13, 1959–1970.

Yadav, A., & Vishwakarma, D. K. (2020). Sentiment analysis using deep learning architectures: a review. Artificial Intelligence Review, 53(6), 4335–4385. https://doi.org/10.1007/s10462-019-09794-5