PDF  PubReader

Tedmori* and Awajan*: Sentiment Analysis Main Tasks and Applications: A Survey

Sara Tedmori* and Arafat Awajan*

Sentiment Analysis Main Tasks and Applications: A Survey

Abstract: The blooming of social media has simulated interest in sentiment analysis. Sentiment analysis aims to determine from a specific piece of content the overall attitude of its author in relation to a specific item, product, brand, or service. In sentiment analysis, the focus is on the subjective sentences. Hence, in order to discover and extract the subjective information from a given text, researchers have applied various methods in computational linguistics, natural language processing, and text analysis. The aim of this paper is to provide an in-depth up-to-date study of the sentiment analysis algorithms in order to familiarize with other works done in the subject. The paper focuses on the main tasks and applications of sentiment analysis. State-of-the-art algorithms, methodologies and techniques have been categorized and summarized to facilitate future research in this field.

Keywords: Feature Selection , Opinion Mining , Sentiment Analysis , Sentiment Analysis Applications , Sentiment Classification , Sentiment Visualization , Social Media Monitoring

1. Introduction

Sentiment analysis (SA) also known as opinion mining is a sub-division of data mining. SA refers to the practice of applying text analysis and natural language processing (NLP) for the purpose of identifying, extracting, and analyzing subjective information from textual sources. SA focuses on the task of classifying a given input text by the polarity of its sentiment as being positive, negative or neutral. More advanced SA techniques look at whether the textual sources have associations with emotional states such as fear, anger, happiness, and sadness. Alternatively, instead of classifying text as being either positive, negative, or neutral, the text could be associated with a number on a pre-defined scale (e.g., -5 to +5).

Subjectivity/objectivity identification is another important research topic in SA [1,2], which involves classifying a given textual source as being either objective or subjective. When compared with the task of polarity classification, the task of subjectivity/objectivity identification is more challenging [3]. The reason behind this challenge is that the subjectivity of words and phrases may well be dependent on the context and an objective document may possibly enclose subjective sentences and vice-versa. Furthermore, results are largely reliant on the definition of subjectivity chosen when annotating texts [4]. Research has shown that eliminating objective sentences from a document prior to polarity classification has a positive effect on the performance of the sentiment classification task [3].

A more challenging, but preferable SA task, is feature/aspect-based SA, which refers to the task of classifying the sentiments expressed on the different features or aspects of the entities. Feature/aspect-based SA involves several sub-tasks, including entity identification, entity features/aspects extraction, feature/aspect opinion identification, and feature/aspect opinion classification.

Every year, many researchers publish articles in the area of SA. Hence, there have been many attempts by researchers to summarize the recent research developments and directions of SA. A survey paper by Medhat et al. [5] focuses on the different SA techniques and provides a comprehensive overview of sentiment analysis. Fifty-four articles published between 2010 and 2013 covering a wide variety of SA fields are summarized and categorized according to the techniques used. Their survey paper also includes a discussion of related fields.

Tsytsarau and Palpanas [6] in their survey discuss in details the main topics of sentiment analysis, along with their definitions, problems, and development. They present a categorization of the articles using tables and graphs. Pang and Lee [3], Khan et al. [7], Liu [8], and Vinodhini and Chandrasekaran [9] also provide detailed surveys that focus on the applications and challenges of SA. Pang and Lee [3] divide all available approaches adopted before 2008 into two broad categories: namely, sentiment classification and extraction, and opinion summarization. Khan et al. [7] focus on the tasks and approaches related to subjectivity and polarity classification, opinion target identification, and opinion source identification. Cambria et al. [10] and Montoyo et al. [11] discuss new research avenues in SA. Giachanou and Crestani [12] specifically provide a survey of Twitter SA methods. Biltawi et al. [[13] survey sentiment classification techniques for the Arabic language.

This paper aims is to provide a comprehensive overview of SA. The details reported in this paper have noteworthy differences to those reported in the studies referenced above and which focus on only one or two of the main SA tasks. This paper links to new references and includes a detailed discussion of four main SA tasks. The paper also provides an overview of the diverse array of SA applications.

The paper is organized as follows. Section 2 provides an overview of SA and its categories. Section 3 provides details related to the main subtasks of SA. Section 4 provides a comprehensive overview of its common applications. Section 5 offers some discussion and challenges for future work. Finally, section 6 ends the paper.

2. Sentiment Analysis

SA refers to the process of detecting and extracting opinions, feelings, attitudes, views, and evaluations from a specific input data. There have been many efforts to resolve the general SA problem; some of which cast it as a text classification task in which the focus is to annotate the polarity of input text as positive or negative depending on the feelings the text conveys and that can be expressed in any form or language. Standard approaches to SA have been classified in many different ways. The most fundamental classification is into the three broad categories of word spotting, lexical affinity, and statistical methods [10]. In word spotting, text is classified into affect categories (e.g., happy, sad, angry, bored) based on the existence of unambiguous affect words. Lexical affinity, unlike word spotting, does not detect clear affect words, but assigns words a probability of association with a particular emotion. Statistical methods employ machine learning techniques such as latent semantic analysis (LSA), support vector machines (SVMs), and Bayesian inference, with a large set of affectively annotated text, known as a corpus, as training data. Statistical methods, similar to word spotting, are used to learn the emotional value of affect words, taking into consideration other features such as the emotional value of other arbitrary keywords, punctuation, and word co-occurrence [10]. Cambria et al. [10] point out that although keyword spotting is a very popular and simple SA approach; however, there are two weaknesses associated with it. First, in the case of negation, the recognition of affect is poor. Second, the method is dependent on the presence of obvious affect words. Lexical affinity, on the other hand, outperforms keyword spotting, but similarly has its own weaknesses. Lexical affinity is word level based and hence will not operate properly in the presence of negation. In addition, the performance of lexical affinity is dependent on the employed linguistic corpora, making the development of a reusable, domain-independent model difficult.

More recent approaches to SA, lead to the introduction of a fourth category: namely, the concept level approach [14] which relies on ontologies and semantic networks that take into account the implicit affective information expressed when utilizing natural language. Concept-level based approaches are capable of detecting subtly expressed sentiments. Hybrid approaches to sentiment analyses that combine two or more of the afore-mentioned approaches have also been proposed [15].

In 2010, Qiu et al. [16] tackled the problem of contextual advertising by proposing a strategy named DASA (dissatisfaction-oriented advertising based on sentiment analysis) that helps optimize advertisement relevance and user experience. The authors use a rule-based approach along with syntactic parsing and a sentiment dictionary to extract the topic words from negative subjective sentences.

Maks and Vossen [17], Cao et al. [18], Zhou et al. [19], Zhang et al. [20], and Pai et al. [2] exploit the semantic characteristics of the text to help perform the SA task at hand. Maks and Vossen [17] employ a syntactic semantic approach to build a SA lexicon model whose aim is to portray the subjectivity associations found between the actors in a sentence conveying the distinct attitudes of each actor. Cao et al. [18] explore the semantic characteristics of reviews and show that they are more significant than other characteristics in affecting the volume of helpfulness votes reviews garner. Similarly, Pai et al. [2] put forward a method for analyzing the content of product/service reviews for the purpose of assisting consumers in their decision making process. Zhou et al. [19] present an unsupervised technique that utilizes the semantic sequential representations to recognize the rhetorical structure theory based discourse relations for the purpose of diminishing polarity ambiguities inside the sentence. Zhang et al. [20] present a system that uncovers products’ weakness from Chinese reviews. The authors use statistical methods to identify implicit feature words and semantic methods to group together explicit feature words.

Min and Park [21] apply NLP techniques and use linguistic hints in another effort to rank the helpfulness of product reviews and introduce a new measure referred to as ‘mentions about experiences’ that discovers the expressions of time associated with the use of products and product entities over various purchasing periods. Hu et al. [22] employ a statistical method to detect the manipulation of online reviews, assess how consumers interact with the products that these manipulated reviews target, and examine whether these reviews have been manipulated through ratings and/or sentiments. Moreo et al. [23] propose a system that uses a news analysis taxonomy model to assess user opinions on topics expressed in news items. NLP techniques were used to build the taxonomy model. Other researchers employ supervised learning models in order to conduct the SA task at hand [24-26].

In addition to the SA works discussed above, Table 1 summarizes other SA works; it is organized in seven columns. The third column specifies whether the approaches detailed in the articles are domain

Table 1.

Summary of related SA works
Ref. Year DO Algorithms used Polarity Data scope Data set/sources
[35] 2008 N SVM, Naïve Bayes (NB), GA Movie reviews IMDB
Alternating decision tree (ADTree)
[36] 2008 Y SVM, Similarity measure (SIM), GA Game software review N/A
Maximum entropy (ME), documents
Support vector regression (SVR)
[37] 2009 N SVM GA Online forums/td> N/A
[38] 2009 N NB, Dictionary BC Blogs, movie reviews IMDB
[39] 2009 N Rule-based, Dictionary, Statistic, SVM GA Movie reviews, IMDB
Product reviews
[40] 2010 N Multinomial Naïve Bayes (MNB), BC Micro-blogs Twitter
Stochastic Gradient Descent (SGD),
Hoe ding Tree (HT)
[41] 2010 N MNB GA Micro-blogs Twitter
[16] 2010 Y Rule-based GA Web forums automotvieforums.com
[18] 2011 N Semantic GA Software programs users’ CNETD
[42] 2011 N CRF GA Mobile customer reviews amazon.com,
blogs, and emails
[24] 2011 N Multi-class SVM GA Digital cameras, N/A
MP3 reviews
[25] 2011 Y SVM, chi-square GA Buyers’ posts web pages ebay.com,
[19] 2011 N Semantic GA Chinese training data NTCIR7, MOAT
[22] 2012 N Statistical GA Book reviews amazon.com
[23] 2012 N Lexicon-based NLP GA News N/A
[20] 2012 N Pointwise mutual information (PMI), GA Product reviews N/A
[21] 2012 N NLP GA Product reviews amazon.com
[43] 2012 N Sentiment Hyperspace Analogue to GA News pages Sogou CS corpus
Language (S-HAL), Semantic Orientation
inference from PMI (SO-PMI)
[17] 2012 N Semantic BC Lexicons Dutch WordNet
[26] 2012 N SVM, K-nearest neighbor, BC Media Media-analysis company
Radial Basis Function (RBF),
Bayesian Network (BN),
Decision Trees (DT),
a propositional rule learner (JRip)
[44] 2013 Y Context-based method NLP GA Restaurant reviews N/A
[45] 2013 Y Formal Concept Analysis (FCA) GA Smartphones, Tweets Twitter
[2] 2013 N Semantic BC Fast food reviews N/A
[46] 2013 Y n-gram, Dynamic ANN GA Tweets Twitter
[47] 2013 Y ML, NLP Tools BC Advertising strategies Twitter
[48] 2014 N Supervised term weighting scheme GA Product reviews Cornell and Stanford movie
Amazon product review
[49] 2014 Y Lexicon-based, ML BC SentBuk on Facebook Facebook
[50] 2014 Y Lexicon-based, ML BC Tweets Twitter
[51] 2014 Y NB, SVM, Random Forest, GA Tweets Twitter
Logistic Regression
[52] 2015 N SVM GA Tweets Twitter
[53] 2015 N Linguistic feature scoring GA customer reviews Multi-domain SA dataset
[54] 2015 N Latent Dirichlet allocation BC Mobile phone producer, Movie reviews dataset
[55] 2016 Y n-gram, Dynamic ANN GA Tweets Twitter

oriented (DO) or not (Y=domain oriented, N=not domain oriented). The fourth column lists the algorithms used. The fifth column indicates whether each of SA techniques detailed in the articles performs general text analysis (GA) or tackles the task as binary classification (BC) (positive/negative). The sixth column highlights the scope of the data used for evaluation. The seventh column lists the benchmark dataset or data source used, if available.

Fig. 1.

Generic SA architecture.

SA involves the main subtasks of opinion identification (identifying the opinionative words or phrases), feature extraction (identifying the features required for the analysis), sentiment classification (SC) (classifying the polarity as positive, negative or neutral), and summarization of results [27]. These subtasks are depicted in Fig. 1. Web users use various blogs, forums, social networking sites, and review sites to express their personal feelings, opinions, emotions, and feedback in relation to a particular product, service, individual, or event. As presented in Fig. 1, the method begins with data collection, which involves collecting data from opinion sources. To facilitate SA, from these opinion rich sources, only data related to the search topic under study is collected. The input document is then preprocessed. Opinion identification is then performed which involves determining whether the textual source being analyzed contains opinions. Only textual sources containing opinions undergo further processing. Additional refinements can be made at this stage (e.g., negation removal, word disambiguation). Feature extraction then follows. The feature extraction subtask, which follows opinion identification, is a subtask of SC task that aims to identify the important words in the inputted text document. Examples of text features include part-of-speech (POS), term presence and frequency, negations, and opinion words and phrases [5. Feature extraction subtask will reduce the amount of data that needs to be considered in the subsequent SC subtask. As the extracted features will be fed to the classifier, it is vital to extract the correct features which will contribute to the success of the machine learning technique that will be used in the SC subtask that will follow. The aim of the SC task is to classify the sentiment polarity. Finally, the summarization of results subtask involves presenting the user with a summary of the results of the analysis. This summary can range from a simple list of positive and negative evaluations to more advanced graph based summarization and summary models. The majority of the works in SA usually focus on only one or two of these four main tasks. A detailed discussion of the SA subtasks highlighted above is provided in the subsections that follow.

3. Sentiment Analysis Subtasks

3.1 Opinion Identification

A crucial step towards SA is the identification of words and phrases that express opinions. Bethard et al. [28 tackle this task by extracting propositional opinions and their holders. A propositional opinion is an opinion that is located in the propositional argument of specific verbs, such as “believe”. The authors make the assumption that opinion is a sentence, or a fragment of a sentence that answers the question “How does X feel about Y?”. Although, sentences may contain multiple sentiment expressions, the authors identify within each sentence only one verb with a propositional argument. Breck et al. [29 propose an opinion identification approach that uses conditional random fields (CRF). Other opinion identification works include the works of Wiebe et al. [30, Munson et al. [31, and Wilson et al. [32. Kim and Hovy [33 present an approach that helps identify opinions, opinion holders, and topics from sentences. The approach has a few steps, the first of which is the identification of opinion-bearing words which are in essence opinion-bearing verbs and adjectives classified into positive, negative, and neutral classes. Htay and Lynn [34 propose an aspect/feature based approach that focuses of extracting for each aspect, from customer reviews, opinion words or phrases. The authors make use of a POS tagger to identify opinion phrases (adjectives, adverbs, verb, and nouns).

3.2 Feature Extraction

Feature extraction methods can be classified as lexicon-based or statistical based. Lexicon-based approaches use a small set of ‘seed’ words and then grow this set using synonyms or online resources. Such approaches can be time intensive. In addition, some terms may not occur frequently enough for proper classification. Statistical methods are fully automated methods, which work by extracting linguistic rules from domain-oriented corpus to detect candidate sentiment terms and structures. Feature extraction techniques deal with documents either as a bag of words or as a sequence of words. The bag of words approach is more popular due to its simplicity [5.

Feature extraction involves a preprocessing step that includes the application of NLP techniques such as POS, stemming, stop words removal, and noun identification to help in the identification of features. Examples of features include opinion words and phrases, terms presence and frequency, POS, n-grams, negations, etc. Many studies exist that have addressed feature selection. A study of Pang et al. [56 addresses the problem of classifying documents according to the overall sentiment they score. A few machine learning algorithms were analyzed with different feature sets (e.g., unigrams, bigrams, POS, position, feature frequency, feature presence) on a movie review dataset.

Statistical techniques are used as feature extraction methods in order to discriminate between features that need to be considered for classification and those that shouldn’t [57. In general, these feature selection techniques fit into one of two categories, filter approach or wrapper approach. In filter approach, the feature subset selection is performed where for each given feature a measure (often statistical) is calculated. Only features with a measure greater than a threshold are classified. The most frequently used filters are mutual information, chi-square, frequency-based, and latent semantic indexing. Other statistical measures used in feature selection include hidden Markov model (HMM), latent Dirichlet allocation (LDA), weight by correlation, information gain, Gini index, and many more. In wrapper approach, the given features are evaluated by training a classifier.

There have been various attempts by researchers to address the feature selection problem using different statistical measures [58-61]. The PMI which is a measure of association/dependence between any two objects was exploited by Yu et al. [58], who developed a contextual entropy model that employs sentiment annotation to add to a list of seed words created from a small corpus of stock market news. The model calculates the similarity between two words by using an entropy system to compare their relative contextual distributions, leading to the discovery of new words related to the seed words. Both set of words are then used in the classification process. Results show that the proposed technique is capable of discovering useful emotion words, hence improving the classification results. The authors of [59] propose a machine learning approach that selects semantically relevant features minimizing the risk of over-fitting. In their work, the authors examined whether it is possible to improve stock price predication from financial news as earlier approaches have resulted prediction accuracies near guessing likelihood. The authors used expressive features to represent text and market in order to enhance the existing text mining methods. Robust feature selection allows a significant improvement in classification accuracies when used in combination with complex feature types. Duric and Song [60] used a model that combines both content and syntax and automatically learns from a review a set of features. The model distinguishes between the entities being reviewed and the subjective expressions that describe those entities in terms of polarities. By concentrating only on the subjective expressions, it was possible to choose more salient features for document-level sentiment analysis. Reyes and Rosso [61] tackled the problem of irony detection by analyzing a set of ironic customer reviews collected from social and mass media. The purpose of their research is to collect a set of discriminating elements that represent irony. The authors built a data set with ironic reviews collected from Amazon and provided valuable insights into language issues that can encounter sentiment analysis, opinion mining and decision-making tasks. Table 2 contains details of the approaches proposed in the field of feature extraction.

Table 2.

Summary of related feature selection works
Ref Year DO Algorithms used Polarity Data scope Data set/sources
[60] 2012 N Statistical (HMM - LDA) ME BC Movie reviews N/A
[61] 2012 N Semantic, NB, SVM, DT GA News, Satiric articles, amazon.com
Customer reviews
[59] 2013 Y Chi-square, Bi-normal separation GA Stock market DGAP, EuroAhoc
[58] 2013 Y PMI-based GA Stock news N/A
[62] 2016 Y Gini index GA Movie reviews N/A
3.3 Sentiment Classification

SC, as mentioned previously, is a fundamental task in the SA field which refers to the to the task of automatically categorizing a piece of text into classes such as, “positive”, “negative” or “neutral”. Most of the approaches to SC can be classified as being machine learning approach, lexical based approach, or hybrid approach. The machine learning approach combines linguistic features and machine learning algorithms. SC methods that use machine learning can be roughly divided into supervised, unsupervised, and semi-supervised methods. Supervised methods depend on the availability of large annotated training data. When an annotated training data is not available, an unsupervised method can be an attractive alternative. Semi-supervised method can be used when only a small amount of labeled data is available. The lexical-based approach uses a sentiment lexicon. A sentiment lexicon is a compiled set of affect words and phrases. A positive or negative score is associated with each of these affect words or phrases to reflect its sentiment polarity and strength. Lexical-based approaches can be further divided into two categories: dictionary-based and corpus based. The dictionary-based approach often starts with a small set of seed words and then grows this seed word list by iteratively searching the dictionary of their synonyms and antonyms. The corpus based approach employs statistical or syntactic methods to mine opinion words in a large corpus. The hybrid approach, a combination of machine learning and lexical based approaches, is the most popular. There have been many studies on sentiment polarity classification. These studies are summarized in Table 3.

Table 3.

Summary of related sentiment classification works
Ref. Year DO Algorithms used Polarity Data scope Data set/sources
[63] 2009 Y Semantic BC Blogs TREC
[64] 2009 N Statistics BC News, Reviews IMDB
[65] 2009 Y Dictionary GA Blogs N/A
[66] 2009 N NB, ME, SVM BC Micro-blogs Twitter
[67] 2010 N Graph-based approach, BC Camera reviews Chinese Opinion Analysis
NB, SVM Domain (COAE)
[68] 2011 Y Graph-based approach BC Movie, Product reviews N/A
[69] 2011 Y Weakly and semi supervised BC Movie reviews multi-domain IMDB, Amazon.com
classification sentiment data set
[70] 2011 N CRFs BC Car, Hotel, Computer reviews N/A
[71] 2011 N Markov Blanket, SVM, NB, ME BC Movie reviews, News articles IMDB
[72] 2011 N Lexicon-based semantic BC Movie reviews IMDB
[73] 2011 N Markov logic network (MLN), BC Product reviews amazon.com
[74] 2012 N NB, SVM BC Restaurant reviews, N/A
[75] 2012 N Semi supervised, BN GA Artificial data sets N/A
[76] 2012 Y NB, SVM rule-based GA 2 sided debates convinceme.net
[77] 2013 N Unsupervised, LDA GA Social reviews 2000-SINA blog data set,
300-SINA, Hownet
[78] 2013 N FCA, Fuzzy FCA (FFCA) BC Movie, eBook reviews Reuters 21578
[79] 2013 N SVM, BC Movie, GPS, Camera, amazon.com
Artificial neural network (ANN) Books reviews
[80] 2013 N SVM, NB, C4.5 BC Film reviews MC, MCE corpus
[81] 2013 N NB, SVM BC Movie reviews, Tweets Twitter
[82] 2013 N SVM GA Tweets Twitter
[83] 2013 Y Taxonomy-based, corpus-based BC Headphones, Car, Hotel reviews epinions.com
[84] 2013 N Semantic BC Blog posts TREC 2006, TREC 2007,
and TREC 2008
[85] 2015 N lSVM GA Micro-blogs Micro-blogs
3.4 Sentiment Summarization

The widespread use of social media platforms such as Facebook, Twitter, Tumbler, Instagram, and YouTube have provided individuals with first-hand access to the sentiments and reactions of millions of people. Analyzing such data, entails high computational power, before the results can be summarized and efficiently presented to the indented users in a concise understandable form. It is essential for the users when they look at the results to get a general understanding of peoples’ sentiments towards an item/product or a specific feature. In the case where the volume of data is large, presenting this data in a graphical manner can be more efficient than presenting it in basic tabular or numerical formats [86].

Some researchers present opinions in the form of an aspect based summary [87-93]. When presenting the summary of opinion, Hu and Liu [87] employ two numbers for each aspect; one shows the number of people who have positive feelings towards this aspect and the other shows the number of people who have negative feelings towards the same aspect. Users can click links to access the actual sentences that express these sentiments. Ku et al. [88] and Mei et al. [90], on the other hand, make use of a summary timeline to highlight opinion changes over time. Lu et al. [89], Popescu and Etzioni [91], and Titov and McDonald [92] in their summarization technique show the most occurring phrase in each aspect. Zhuang et al. [93] provide a statistical summary showing the sentiment distribution of each aspect along with the corresponding sentences for each aspect and sentiment.

Fukuhara et al. [94] propose a system for temporal SA that produces two kinds of graphs represented by a line diagram. The first graph is a topic graph that shows over a period of time the temporal change of topics related to a particular sentiment (e.g., happy, sad, anger) and the second graph is a sentiment graph that shows over a period of time the temporal change of sentiments related to a particular topic. Visualizations by Fukuhara et al. [94] are limited to showing a range of sentiments associated with a single event or a range of events associated with a single sentiment. Havre et al. [95] propose ThemeRiver, a system that uses a river metaphor to visualize thematic changes in documents over time. The river in the graph is pictured flowing from left to right, altering in width to mirror changes in thematic strength. Colored “currents” flowing within the river narrow or widen to designate decreases or increases in the strength of a single topic or set of topics in the related documents.

Marcus et al. [96] present TwitInfo, a system that visualizes and summarizes events on Twitter. TwitInfo extracts the tweets that match a keyword and arranges them on a timeline that highlights peaks of high tweet activity. TwinInfo utilizes information retrieval techniques to assign peaks with meaningful labels using text from the tweets. TwitInfo uses only two categories of sentiment (positive and negative). Vox Civitas [97] is another visualization tool for Twitter. Similar to TwinInfo, Vox Civitas uses temporal visualization. Vox Civitas is used by journalists to support the exploration of tweets about specific events. Vox Civitas makes the use of a sentiment bar. The color of the bar for each time interval represent the polarity (positivity, negativity, or controversial) of the aggregate of all messages in that interval. Towards the bottom of the Vox Civitas interface, a set of keywords appears along the sentiment time-line.

The VISA [98] and the TriVis [99] systems also employ visualization techniques to summarize SA results and interactively present them to the user. VISA has three different visualization views. The first view is the sentiment trend view which shows chronologically, for the different categories/topics, the sentiment dynamics and sentiment comparisons. The second view employs chart visualization to highlight the associated structured facets. Finally, the third view shows the snippet/document panel which provides details of the documents and the context of sentiment [98].

TriVis [99] employs the three sentiment classes (positive, neutral and negative) to visualize multivariate social sentiment data. TriVis combines sentiment and volume in one graph. Trivas uses color to distinguish between multiple topics. In addition, TriVis uses a modified scatter plot with three axes to allow the user to easily interpret and understand multivariate data. Other systems employ simpler visualization techniques. The Semantize [100] system uses font styles and background colors to mark the sentiments found in documents. Pino et al. [101] provide a tool that visualizes sentiments related to major events in geographically confined populations on interactive maps.

4. Applications of Sentiment Analysis

SA has found its way in many applications across multiple domains. This section discusses some of the common applications. The examples presented in this section are not complete but exist to provide a snap shot of the possibilities.

People are sensitive to sentiment; hence, it is only natural from them to always want to find out what others’ opinions are on, amongst many, a specific product, application, service, or political figure. Classically, when someone needed to know others’ opinions, he/she would ask his/her friends, family, and/or others for their opinion. Moreover, when an organization needed to know public or customer opinions, it conducted surveys and/or focus groups.

Recent years have witnessed the global tremendous growth of e-commerce and social media. This growth, combined with their sentiment rich nature, has led to an increasing use of the content available in these sites for purposes of decision-making. Unfortunately, the large number of diverse sites available coupled with the large volume of information contained in them, makes the task of finding and monitoring opinions very difficult. Hence, the need for automated SA systems has dramatically increased.

Documents containing opinions do not only exist on the web. Many companies possess internal sentiment rich sources. Examples of such sources include customer feedback collected via email, surveys, and/or over the telephone.

The applications of SA are numerous. In order to best understand their varying applications, they are grouped into three categories: applications for business analytic, applications for political/government/public analytic, and other types of applications.

4.1 Applications for Business Analytic

Mining sentiments from public sources can be particularly effective in the area of business analytics. The most common SA applications in this area center around reviews of consumer products and services. Hence, many works have been published in literature detailing the design of such applications. Liu et al. [102] in their paper propose, ARSA, a sentiment-aware model that uses blogs to predict sales performance. McGlohon et al. [103] exploit the sentiments expressed in reviews to rank products and merchants.

Some works exploit sentiment rich sources for the purpose of predicting movies box-office revenues [104-106], whilst others [107] test the effectiveness of their approach in improving the predication quality of box-office revenues.

Another widespread set of applications is in stock market analysis. Bollen et al. [108] exploited the sentiments in tweets for purposes of performing stock market analysis. Bar-Haim et al. [109] and Feldman et al. [110] propose a framework for identifying expert investors, and then use this information in models that exploit micro blogging messages for the purpose of predicting stock rise and fall.

Twitter and Facebook are a central source of many sentiment analysis applications in this area, of which brand reputation monitoring is the most common. One application that allows companies to explore consumers’ opinions about their brands or products is Trackur [111]. Many other brand monitoring applications exist, including BrandsEye [112], TweetFeel [113], Twitrratr [114], and Trendrr [115].

4.2 Applications for Public Opinion Analytic

Other SA research efforts and applications utilize political information sources, such as political blogs, tweets with political content, candidates' web sites/Facebook pages, and/or online news.

Hong and Skiena [116] study the relationships between the National Football League betting line and public opinions in blogs and Twitter and show that a significant correlation exits between the public sentiments (expressed in blogs and tweets) and the betting line. Tumasjan et al. [117] were the first researchers who published their efforts detailing the application of SA on twitter data for purposes of predicting the national German Federal election results, reporting a strong correlation between the data embedded in 140 character tweets and the election results. Subsequent efforts include the works of DiGrazia et al. [118], Franch [119], Ceron et al. [120], Caldarelli at al. [121], and Burnap et al. [122]. Chen et al. [123] use an opinion scoring model to explore political standpoints. Applications under this category also include those that try to look at the public image and reputation of a specific population (e.g., gender, race, social class, and religious minority) [124-126]. Kwon and Lee [127] exploit personal and social opinions for the purpose of bias detection.

Other applications are those that target the sentiments expressed in news articles. Some of these applications are interested in the polarity of these articles towards the subject of interest [126], whilst others are interested in the attitude that these articles impart [128]. Applications targeting sentiment in political campaign discourse [129] and campaign publications and news releases [130-132] also fall under this category.

SA has also been explored in the area of eRulemaking, for the purpose of analyzing the opinions of people towards a specific policy or government regulation [133-135]. Other efforts reported exploit “blawgs”, weblogs devoted to legal tissues [136].

4.3 Other Types of Applications

To better depict the diversity of SA applications, this subsection includes some additional examples of such applications. Groh and Hauffa [137] employ NLP based SA techniques on a collection of email messages to characterize social relations, whilst Mohammad and Yang [138], apply SA on emails to uncover the emotional variations between genders.

Sakunkoo and Sakunkoo [139], in an attempt to answer the question of whether and how previous opinions on a certain item affect the opinions that come after, study the social influences in online book reviews.

Visual SA is another area that has recently attracted some attention. Wang et al. [140] propose USEA, an unsupervised SA framework that predicts the sentiments of social media images. Tedmori and Al-Lahaseh [141] propose an addition to social networks that automatically analyses selfies for their underlying sentiments. The system then automatically generates sentiment bearing hashtags.

5. Discussions and Challenges

The general SA task and the SC subtask are and will continue to be the most attractive opinion mining research areas. In regards to SC, machine learning algorithms are the most popular due to their simplicity, and the fact that these algorithms learn from training data which make them particularly useful for domain adaptability. The lexicon-based algorithms, on the other hand are very popular in addressing the general SA problems. Lexicon-based algorithms are not only scalable, but also simple and computationally efficient [5]. In relation to feature extraction, the use of unigram is simple and helps improve the performance of the SC. However, looking beyond unigrams, into bigrams and trigrams can boost SC accuracy considerably. In addition, POS tagging has also been shown to also improve the SC performance. Moreover, the correct ensemble of feature selection, suitable to the task at hand, should improve further the accuracy. As stated by many, SVM performs better than other classifiers including Naïve Bayes for sentiment analysis. It is worth however, to continue experimenting with other classifiers or with an ensemble of classifiers in order to significantly improve SC.

Although a variety of research can be found employing lexicon-based approaches to tackle SA as a general analysis of text using, it still remains largely unsolved. General sentiment is an immensely more difficult task than binary polarity classification (positive/negative classification). Moreover, it’s only natural that more accurate results can be obtained when using domain-specific data instead of domain-general data. According to [5], and as can be seen in Tables 1–3, more researchers have worked and are continuing to work on domain-general data, making domain-specific SA an ongoing field of interest. Moreover, the majority of works detailed in this research target the English language. Other languages remain under-explored. Another main problem in SA for non-English is a significant lack of resources. Hence, some researchers can focus their efforts on building resources such as lexica, corpora and dictionaries, and other necessary resources.

6. Conclusions

SA has found its way in many applications across multiple domains such as the political, government, public and business analytic. SA refers to the process of identifying and extracting opinions, emotions, and evaluations from a specific input. Standard approaches to SA have been classified in many different ways. The most fundamental classification is into the three broad categories of: keyword spotting, lexical affinity, and statistical methods. SC, a fundamental task in the SA field, refers to the to the task of automatically categorizing a piece of text into classes such as, “positive”, “negative” or “neutral”. SC approaches can be categorized as being either machine learning based, lexical based approach, or hybrid. Moreover, the methods of feature extraction, another vital SA task, can be classified into two categories: namely, lexicon-based methods that need human annotation, and statistical methods which are fully automatic methods.

The aim of this paper is to provide an in-depth up-to-date study of the SA techniques in order to familiarize with other works done in the subject. The paper focuses on the main tasks and applications of sentiment analysis. The majority of the works detailed in this survey target the English language. Hence, various English language lexica, corpora and dictionaries are freely available for research purposes. However, SA for non-English languages including Arabic, Spanish, Italian, German, Dutch, Chinese, Japanese, and Taiwanese has only recently attracted researchers’ attention paving way to new challenges including building language specific resources. Not only micro-blogs, blogs and forums are popular data sources for sentiment analysis, social networks can also be used to identify authors’ sentiments on brands, products, or services. The majority of researchers use different datasets for their different testing objectives. This calls for the need for a single dataset that can be used to accurately test and compare the performance of these algorithms.


Sara Tedmori

In 2001, she received her B.Sc. degree in Computer Science from the American University of Beirut, Lebanon. In 2003, she obtained her M.Sc. degree in Multimedia and Internet Computing from Loughborough University. In 2008, she received her Engineering Doctorate in Computer Science from Loughborough University, UK. Currently she is an associate Professor in the Computer Science Department at Princess Sumaya University of Technology, Jordan. Her research interests include natural language processing, sentiment analysis, data mining, image processing, video coding, cryptography, social networks, web technologies, and classification.


Arafat Awajan

He is a full professor at Princess Sumaya University for Technology (PSUT). He received his Ph.D. degree in computer science from the University of Franche-Comte, France in 1987. He held different academic positions at the Royal Scientific Society and Princess Sumaya University for Technology. He was appointed as the chair of the Computer Science Department (2000–2003) and the chair of the Computer Graphics and Animation Department (2005–2006) at PSUT. He had been the dean of the King Hussein School for Information Technology from 2004 to 2007, the Dean of Student Affairs from 2011–2014 and the director of the Information Technology Center in the Royal Scientific Society from 2008–2010. He is currently the dean of the King Hussein School for computing Sciences and leads a research group on natural language processing and arabic text mining.


  • 1 M. Thelwall, K. Buckley, G. Paltoglou, D. Cai, A. Kappas, "Sentiment strength detection in short informal text," Journal of the American Society for Information Science and T echnologyno 12, vol. 61, pp. 2544-2558, 2010.doi:[[[10.1002/asi.21416]]]
  • 2 M. Y. Pai, H. C. Chu, S. C. Wang, Y. M. Chen, "Electronic word of mouth analysis for service experience," Expert Systems with Applications, vol. 40, no. 6, pp. 1993-2006, 2013.doi:[[[10.1016/j.eswa.2012.10.024]]]
  • 3 B. Pang, L. Lee, Foundations and Trends in Information Retrieval, vol, 2, no. 1–pp.1-135, pp. no.1–1-135, 2008.custom:[[[-]]]
  • 4 F. Su, K. Markert, "From words to senses: a case study of subjectivity recognition," in Proceedings of the 22nd International Conference on Computational Linguistics, Manchester, UK, 2008;pp. 825-832. custom:[[[-]]]
  • 5 W. Medhat, A. Hassan, H. Korashy, "Sentiment analysis algorithms and applications: a survey," Ain Shams Engineering Journal, vol. 5, no. 4, pp. 1093-1113, 2014.doi:[[[10.1016/j.asej.2014.04.011]]]
  • 6 M. Tsytsarau, T. Palpanas, "Survey on mining subjective data on the web," Data Mining and Knowledge Discovery478-514, vol. 24, no. 3, 2012.doi:[[[10.1007/s10618-011-0238-6]]]
  • 7 K. Khan, B. Baharudin, A. Khan, A. Ullah, "Mining opinion components from unstructured reviews: a review," Journal of King Saud University-Computer and Information Sciences, vol. 26, no. 3, pp. 258-275 2014. doi:[[[10.1016/j.jksuci.2014.03.009]]]
  • 8 B. Liu, Synthesis Lectures on Human Language T echnologies, vol, 5, no. 1, pp. 1-167, 2012.custom:[[[-]]]
  • 9 G. Vinodhini, R. M. Chandrasekaran, "Sentiment analysis and opinion mining: a survey," International Journal, vol. 2, no. 6, pp. 282-292, 2012.doi:[[[10.5120/ijca2016911545]]]
  • 10 E. Cambria, B. Schuller, Y. Xia, C. Havasi, "New avenues in opinion mining and sentiment analysis," IEEE Intelligent Systems, vol. 28, no. 2, pp. 15-21, 2013.doi:[[[10.1109/MIS.2013.30]]]
  • 11 A. Montoyo, P. MartiNez-Barco, A. Balahur, "Subjectivity and sentiment analysis: an overview of the current state of the area and envisaged developments," Decision Support Systems, vol. 53, no. 4, pp. 675-679, 2012.doi:[[[10.1016/j.dss.2012.05.022]]]
  • 12 A. Giachanou, F. Crestani, "Like it or not: a survey of twitter sentiment analysis methods," ACM Computing Surveys (CSUR), vol. 49, no. 2, 2016.doi:[[[10.1145/2938640]]]
  • 13 M. Biltawi, W. Etaiwi, S. Tedmori, A. Hudaib, A. Awajan, "Sentiment classification techniques for Arabic language: a survey," in Proceedings of 2016 7th International Conference on Information and Communication Systems (ICICS), Irbid, Jordan, 2016;pp. 339-346. custom:[[[-]]]
  • 14 E. Cambria, S. Poria, F. Bisio, R. Bajpai, I. Chaturvedi, in Computational Linguistics and Intelligent Text Processing, Cham: Springer, pp. 3-22, 2015.custom:[[[-]]]
  • 15 S. Poria, E. Cambria, G. Winterstein, G. B. Huang, "Sentic patterns: dependency-based rules for concept-level sentiment analysis," Knowledge-Based Systems, vol. 69, pp. 45-63, 2014.doi:[[[10.1016/j.knosys.2014.05.005]]]
  • 16 G. Qiu, X. He, F. Zhang, Y. Shi, J. Bu, C. Chen, "DASA: dissatisfaction-oriented advertising based on sentiment analysis," Expert Systems with Applications, vol. 37, no. 9, pp. 6182-6191, 2010.doi:[[[10.1016/j.eswa.2010.02.109]]]
  • 17 I. Maks, P. V ossen, "A lexicon model for deep sentiment analysis and opinion mining applications," Decision Support Systems, vol. 53, no. 4, pp. 680-688, 2012.doi:[[[10.1016/j.dss.2012.05.025]]]
  • 18 Q. Cao, W. Duan, Q. Gan, "Exploring determinants of voting for the "helpfulness" of online user reviews: a text mining approach," Decision Support Systems, vol. 50, no. 2, pp. 511-521, 2011.doi:[[[10.1016/j.dss.2010.11.009]]]
  • 19 L. Zhou, B. Li, W. Gao, Z. Wei, K. F. Wong, "Unsupervised discovery of discourse relations for eliminating intra-sentence polarity ambiguities," in Proceedings of the Conference on Empirical Methods Natural Language Processing, Edinburgh, UK, 2011;pp. 162-171. custom:[[[-]]]
  • 20 W. Zhang, H. Xu, W. W an, "W eakness Finder: find product weakness from Chinese reviews by using aspects based sentiment analysis," Expert Systems with Applications, vol. 39, no. 11, pp. 10283-10291, 2012.custom:[[[-]]]
  • 21 H. J. Min, J. C. Park, "Identifying helpful reviews based on customer’s mentions about experiences," Expert Systems with Applications, vol. 39, no. 15, pp. 11830-11838, 2012.doi:[[[10.1016/j.eswa.2012.01.116]]]
  • 22 N. Hu, I. Bose, N. S. Koh, L. Liu, "Manipulation of online reviews: an analysis of ratings, readability, and sentiments," Decision Support Systems, vol. 52, no. 3, pp. 674-684, 2012.doi:[[[10.1016/j.dss.2011.11.002]]]
  • 23 A. Moreo, M. Romero, J. L. Castro, J. M. Zurita, "Lexicon-based comments-oriented news sentiment analyzer system," Expert Systems with Applications, vol. 39, no. 10, pp. 9166-9180, 2012.doi:[[[10.1016/j.eswa.2012.02.057]]]
  • 24 C. C. Chen, Y. D. Tseng, "Quality evaluation of product reviews using an information quality framework," Decision Support Systems, vol. 50, no. 4, pp. 755-768, 2011.doi:[[[10.1016/j.dss.2010.08.023]]]
  • 25 T. K. Fan, C. H. Chang, "Blogger-centric contextual advertising," in Expert Systems with Applications, 2011;vol. 38, no. 3, pp. 1777-1788. custom:[[[-]]]
  • 26 P. C. Lane, D. Clarke, P. Hender, "On developing robust models for favourability analysis: model choice, feature sets and imbalanced data," Decision Support Systems, vol. 53, no. 4, pp. 712-718, 2012.doi:[[[10.1016/j.dss.2012.05.028]]]
  • 27 H. Siqueira, F. Barros, "A feature extraction process for sentiment analysis of opinions on services," in Proceedings of International W orkshop on W eb and T ext Intelligence, Sao Bernardo, Brazil, 2010;pp. 404-413. custom:[[[-]]]
  • 28 S. Bethard, H., Yu, A. Thornton, V. Hatzivassiloglou, D. Jurafsky, "Automatic extraction of opinion propositions and their holders," in Proceedings of 2004 AAAI Spring Symposium on Exploring Attitude and Affect T ext, Palo Alto, CA, 2004;pp. 1-8. custom:[[[-]]]
  • 29 E. Breck, Y. Choi, C. Cardie, "Identifying expressions of opinion in context," in Proceedings of the 20th International Joint Conference on Artificial Intelligence, Hyderabad, India, 2007;pp. 2683-2688. custom:[[[-]]]
  • 30 J. Wiebe, E. Breck, C. Buckley, C. Cardie, P. Davis, B. Fraser, et al., "Recognizing and organizing opinions expressed in the world press," in Proceedings of 2003 AAAI Spring Symposium on New Directions Question Answering, Menlo Park, CA, 2003;pp. 12-19. custom:[[[-]]]
  • 31 A. Munson, C. Cardie, R. Caruana, "Optimizing to arbitrary NLP metrics using ensemble selection," in Proceedings of the Conference on Human Language Technology and Empirical Methods Natural Language Processing, V ancouver, Canada, 2005;pp. 539-546. custom:[[[-]]]
  • 32 T. Wilson, P. Hoffmann, S. Somasundaran, J. Kessler, J. Wiebe, Y. Choi, C. Cardie, E. Riloff, S. Patwardhan, "OpinionFinder: a system for subjectivity analysis," in Proceedings of HLT/EMNLP 2005 Interactive Demonstrations, V ancouver, Canada, 2005;pp. 34-35. custom:[[[-]]]
  • 33 S. M. Kim, E. Hovy, "Extracting opinions, opinion holders, and topics expressed in online news media text," in Proceedings of the W orkshop on Sentiment and Subjectivity T ext, Sydney, Australia, 2006;pp. 1-8. custom:[[[-]]]
  • 34 S. S. Htay, K. T. Lynn, "Extracting product features and opinion words using pattern knowledge in customer reviews," The Scientific W orld Journalarticle ID 394758, vol. 2013, 2013.doi:[[[10.1155/2013/394758]]]
  • 35 M. Annett, G. Kondrak, "A comparison of sentiment analysis techniques: polarizing movie blogs," in Advances in Artificial Intelligence. Heidelberg: Springer, pp. 25-35, 2008.doi:[[[10.1007/978-3-540-68825-9_3]]]
  • 36 K. Shimada, T. Endo, in Advances in Knowledge Discovery and Data Mining, Heidelberg: Springer, pp. 1006-1014, 2008.custom:[[[-]]]
  • 37 F. Bodendorf, C. Kaiser, "Detecting opinion leaders and trends in online social networks," in Proceedings of the 2nd ACM W orkshop on Social W eb Search and Mining, Hong Kong, China, 2009;pp. 65-68. custom:[[[-]]]
  • 38 P. Melville, W. Gryc, R. D. Lawrence, "Sentiment analysis of blogs by combining lexical knowledge with text classification," in Proceedings of the 15th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Paris, France, 2009;pp. 1275-1284. custom:[[[-]]]
  • 39 R. Prabowo, M. Thelwall, "Sentiment analysis: a combined approach," Journal of Informetrics, vol. 3, no. 2, pp. 143-157, 2009.doi:[[[10.1016/j.joi.2009.01.003]]]
  • 40 A. Bifet, E. Frank, in Discovery Science, Heidelberg: Springer, pp. 1-15, 2010.custom:[[[-]]]
  • 41 A. Pak, P. Paroubek, "Twitter as a corpus for sentiment analysis and opinion mining," in Proceedings of the 7th Conference on International Language Resources and Evaluation (LREC’10)V alletta, Malta, pp. 1320-1326, 2010.doi:[[[10.17148/ijarcce.2016.51274]]]
  • 42 K. Xu, S. S. Liao, J. Li, Y. Song, "Mining comparative opinions from customer reviews for competitive intelligence," Decision Support Systems, vol. 50, no. 4, pp. 743-754, 2011.doi:[[[10.1016/j.dss.2010.08.021]]]
  • 43 T. Xu, Q. Peng, Y. Cheng, "Identifying the semantic orientation of terms using S-HAL for sentiment analysis," Knowledge-Based Systems, vol. 35, pp. 279-289, 2012.doi:[[[10.1016/j.knosys.2012.04.011]]]
  • 44 L. Di Caro, M. Grella, "Sentiment analysis via dependency parsing," Computer Standards Interfaces, vol. 35, no. 5, pp. 442-453, 2013.doi:[[[10.1016/j.csi.2012.10.005]]]
  • 45 E. Kontopoulos, C. Berberidis, T. Dergiades, N. Bassiliades, "Ontology-based sentiment analysis of twitter posts," Expert Systems with Applications, vol. 40, no. 10, pp. 4065-4074, 2013.doi:[[[10.1016/j.eswa.2013.01.001]]]
  • 46 M. Ghiassi, J. Skinner, D. Zimbra, "Twitter brand sentiment analysis: a hybrid system using n-gram analysis and dynamic artificial neural network," Expert Systems with Applications, vol. 40, no. 16, pp. 6266-6282, 2013.doi:[[[10.1016/j.eswa.2013.05.057]]]
  • 47 S. Dacres, H. Haddadi, and M. Purver, 2013;, https://arxiv.org/abs/1312.6635
  • 48 Z. H. Deng, K. H. Luo, H. L. Y u, "A study of supervised term weighting scheme for sentiment analysis," Expert Systems with Applications, vol. 41, no. 7, pp. 3506-3513, 2014.doi:[[[10.1016/j.eswa.2013.10.056]]]
  • 49 A. Ortigosa, J. M. Martin, R. M. Carro, "Sentiment analysis in Facebook and its application to e-learning," Computers in Human Behavior, vol. 31, pp. 527-541, 2014.doi:[[[10.1016/j.chb.2013.05.024]]]
  • 50 F. H. Khan, S. Bashir, U. Qamar, "TOM: Twitter opinion mining framework using hybrid classification scheme," Decision Support Systems, vol. 57, pp. 245-257, 2014.doi:[[[10.1016/j.dss.2013.09.004]]]
  • 51 N. F. Da Silva, E. R. Hruschka, E. R. Hruschka Jr, "Tweet sentiment analysis with classifier ensembles," Decision Support Systems, vol. 66, pp. 170-179, 2014.doi:[[[10.1016/j.dss.2014.07.003]]]
  • 52 J. Kranjc, J. Smailovic, V. Podpecan, M. Grcar, M. Znidarsic, N. Lavrac, "Active learning for sentiment analysis on data streams: methodology and workflow implementation in the ClowdFlows platform," Information Processing Management, vol. 51, no. 2, pp. 187-203, 2015.doi:[[[10.1016/j.ipm.2014.04.001]]]
  • 53 S. Krishnamoorthy, "Linguistic features for review helpfulness prediction," Expert Systems with Applications, vol. 42, no. 7, pp. 3751-3759, 2015.doi:[[[10.1016/j.eswa.2014.12.044]]]
  • 54 F. Colace, L. Casaburi, M. De Santo, L. Greco, "Sentiment detection in social networks and in collaborative learning environments," Computers in Human Behavior, vol. 51, pp. 1061-1067, 2015.doi:[[[10.1016/j.chb.2014.11.090]]]
  • 55 D. Zimbra, M. Ghiassi, S. Lee, "Brand-related Twitter sentiment analysis using feature engineering and the dynamic architecture for artificial neural networks," in Proceedings of 2016 49th Hawaii International Conference on System Sciences (HICSS), Koloa, HI, 2016. pp.1930-, 1938;pp. 2016. 1930-1938. custom:[[[-]]]
  • 56 B. Pang, L. Lee, S. V aithyanathan, "Thumbs up?: sentiment classification using machine learning techniques," in Proceedings of the ACL-02 Conference on Empirical Methods Natural Language Processing (EMNLP), Pennsylvania, P A, 2002;pp. 79-86. custom:[[[-]]]
  • 57 Y. W. Seo, A. Ankolekar, K. Sycara, Carnegie Mellon University, Pittsburgh, P A, Report No, CMU-RI-TR-04-18, 2004.custom:[[[-]]]
  • 58 L. C. Y u, J. L. Wu, P. C. Chang, H. S. Chu, "Using a contextual entropy model to expand emotion words and their intensity for the sentiment classification of stock market news," Knowledge-Based Systems, vol. 41, pp. 89-97, 2013.doi:[[[10.1016/j.knosys.2013.01.001]]]
  • 59 M. Hagenau, M. Liebmann, D. Neumann, "Automated news reading: stock price prediction based on financial news using context-capturing features," Decision Support Systems, vol. 55, no. 3, pp. 685-697, 2013.doi:[[[10.1016/j.dss.2013.02.006]]]
  • 60 A. Duric, F. Song, "Feature selection for sentiment analysis based on content and syntax models," Decision Support Systems, vol. 53, no. 4, pp. 704-711, 2012.doi:[[[10.1016/j.dss.2012.05.023]]]
  • 61 A. Reyes, P. Rosso, "Making objective decisions from subjective data: detecting irony in customer reviews," Decision Support Systems, vol. 53, no. 4, pp. 754-760, 2012.doi:[[[10.1016/j.dss.2012.05.027]]]
  • 62 A. S. Manek, P. D. Shenoy, M. C. Mohan, K. R. V enugopal, "Aspect term extraction for sentiment analysis in large movie reviews using Gini Index feature selection method and SVM classifier," World Wide Web, vol. 20, no. 2, pp. 135-154, 2017.doi:[[[10.1007/s11280-015-0381-x]]]
  • 63 M. M. S. Missen, M. Boughanem, in Advances in Information Retrieval, Heidelberg: Springer, pp. 729-733, 2009.custom:[[[-]]]
  • 64 J. Read, J. Carroll, "Weakly supervised techniques for domain-independent sentiment classification," in Proceedings of the 1st International CIKM Workshop on T opic-Sentiment Analysis for Mass Opinion, Hong Kong, China, 2009;pp. 45-52. custom:[[[-]]]
  • 65 S. Feng, D. Wang, G. Yu, C. Yang, N. Yang, in Advances in Data and W eb Management, Heidelberg: Springer, pp. 332-344, 2009.custom:[[[-]]]
  • 66 A. Go, R. Bhayani, L. Huang, "T witter sentiment classification using distant supervision," Stanford University Stanford, CA, CS224N Project Report, 2009.custom:[[[-]]]
  • 67 Y. Y. Zhao, B. Qin, T. Liu, "Integrating intra-and inter-document evidences for improving sentence sentiment classification," Acta Automatica Sinica, vol. 36, no. 10, pp. 1417-1425, 2010.custom:[[[-]]]
  • 68 Y. Hu, W. Li, "Document sentiment classification by exploring description model of topical terms," Computer Speech Language, vol. 25, no. 2, pp. 386-403, 2011.doi:[[[10.1016/j.csl.2010.07.004]]]
  • 69 Y. He, D. Zhou, "Self-training from labeled features for sentiment analysis," Information Processing Management, vol. 47, no. 4, pp. 606-616, 2011.doi:[[[10.1016/j.ipm.2010.11.003]]]
  • 70 J. Jiao, Y. Zhou, "Sentiment polarity analysis based multi-dictionary," Physics Procedia, vol. 22, pp. 590-596, 2011.doi:[[[10.1016/j.phpro.2011.11.091]]]
  • 71 X. Bai, "Predicting consumer sentiments from online text," Decision Support Systems, vol. 50, no. 4, pp. 732-742, 2011.doi:[[[10.1016/j.dss.2010.08.024]]]
  • 72 B. Heerschop, F. Goossen, A. Hogenboom, F. Frasincar, U. Kaymak, F. de Jong, "Polarity analysis of texts using discourse structure," in Proceedings of the 20th ACM International Conference on Information and Knowledge Management, Glasgow, Scotland, 2011;pp. 1061-1070. custom:[[[-]]]
  • 73 C. Zirn, M. Niepert, H. Stuckenschmidt, M. Strube, "Fine-grained sentiment analysis with structural features," in Proceedings of 5th International Joint Conference on Natural Language Processing, Chiang Mai, Thailand, 2011;pp. 336-344. custom:[[[-]]]
  • 74 H. Kang, S. J. Y oo, D. Han, "Senti-lexicon and improved Naïve Bayes algorithms for sentiment analysis of restaurant reviews," Expert Systems with Applications, vol. 39, no. 5, pp. 6000-6010, 2012.doi:[[[10.1016/j.eswa.2011.11.107]]]
  • 75 J. Ortigosa-Hernandez, J. D. Rodriguez, L. Alzate, M. Lucania, I. Inza, J. A. Lozano, "Approaching sentiment analysis by using semi-supervised learning of multi-dimensional classifiers," Neurocomputing, vol. 92, pp. 98-115, 2012.doi:[[[10.1016/j.neucom.2012.01.030]]]
  • 76 M. A. W alker, P. Anand, R. Abbott, J. E. F. T ree, C. Martell, J. King, "That is your evidence?: Classifying stance in online political debate," Decision Support Systems, vol. 53, no. 4, pp. 719-729, 2012.doi:[[[10.1016/j.dss.2012.05.032]]]
  • 77 X. Fu, G. Liu, Y. Guo, Z. W ang, "Multi-aspect sentiment analysis for Chinese online social reviews based on topic modeling and HowNet lexicon," Knowledge-Based Systems, vol. 37, pp. 186-195, 2003.doi:[[[10.1016/j.knosys.2012.08.003]]]
  • 78 S. T. Li, F. C. Tsai, "A fuzzy conceptualization model for text mining with application in opinion polarity classification," Knowledge-Based Systems, vol. 39, pp. 23-33, 2013.doi:[[[10.1016/j.knosys.2012.10.005]]]
  • 79 R. Moraes, J. F. Valiati, W. P. G. Neto, "Document-level sentiment classification: an empirical comparison between SVM and ANN," Expert Systems with Applications, vol. 40, no. 2, pp. 621-633, 2013.doi:[[[10.1016/j.eswa.2012.07.059]]]
  • 80 M. T. MartiN-Valdivia, E. MartiNez-CaMara, J. M. Perea-Ortega, L. A. UrenA-LoPez, "Sentiment polarity detection in Spanish reviews combining supervised and unsupervised approaches," Expert Systems with Applications, vol. 40, no. 10, pp. 3934-3942, 2013.doi:[[[10.1016/j.eswa.2012.12.084]]]
  • 81 H. Rui, Y. Liu, A. Whinston, "Whose and what chatter matters? The effect of tweets on movie sales," Decision Support Systems, vol. 55, no. 4, pp. 863-870, 2013.doi:[[[10.1016/j.dss.2012.12.022]]]
  • 82 Y. M. Li, T. Y. Li, "Deriving market intelligence from microblogs," Decision Support Systems, vol. 55, no. 1, pp. 206-217, 2013.doi:[[[10.1016/j.dss.2013.01.023]]]
  • 83 F. L. Cruz, J. A. Troyano, F. Enriquez, F. J. Ortega, C. G. Vallejo, "‘Long autonomy or long delay?’ The importance of domain in opinion mining," Expert Systems with Applications, vol. 40, no. 8, pp. 3174-3184, 2013.doi:[[[10.1016/j.eswa.2012.12.031]]]
  • 84 J. M. Chenlo, A. Hogenboom, D. E. Losada, in Natural Language Processing and Information Systems, Heidelberg: Springer, pp. 13-24, 2013.custom:[[[-]]]
  • 85 S. M. Liu, J. H. Chen, "A multi-label classification based approach for sentiment classification," Expert Systems with Applications, vol. 42, no. 3, pp. 1083-1093, 2015.doi:[[[10.1016/j.eswa.2014.08.036]]]
  • 86 E. R. T ufte, The Visual Display of Quantitative Information, CT: Graphics Press, Cheshire, 1983.custom:[[[-]]]
  • 87 M. Hu, B. Liu, "Mining and summarizing customer reviews," in Proceedings of the 10th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Seattle, W A, 2004;pp. 168-177. custom:[[[-]]]
  • 88 L. W. Ku, Y. T. Liang, H. H. Chen, "Opinion extraction, summarization and tracking in news and blog corpora," in Proceedings of AAAI-2006 Spring Symposium on Computational Approaches to Analyzing Weblogs, Palo Alto, CA, 2006;custom:[[[-]]]
  • 89 Y. Lu, C. Zhai, N. Sundaresan, "Rated aspect summarization of short comments," in Proceedings of the 18th International Conference on W orld Wide W eb, Madrid, Spain, 2009;pp. 131-140. custom:[[[-]]]
  • 90 Q. Mei, X. Ling, M. Wondra, H. Su, C. Zhai, "Topic sentiment mixture: modeling facets and opinions in weblogs," in Proceedings of the 16th International Conference on World Wide Web, Banff, Canada, 2007;pp. 171-180. custom:[[[-]]]
  • 91 A. M. Popescu, O. Etzioni, "Extracting product features and opinions from reviews," in Natural Language Processing and T ext Mining. London: Springer, 2007;pp. 9-28. custom:[[[-]]]
  • 92 I. Titov, R. McDonald, "Modeling online reviews with multi-grain topic models," in Proceedings of the 17th International Conference on W orld Wide W eb, Beijing, China, 2008;pp. 111-120. custom:[[[-]]]
  • 93 L. Zhuang, F. Jing, X. Y. Zhu, "Movie review mining and summarization," in Proceedings of the 15th ACM International Conference on Information and Knowledge Management, Arlington, V A, 2006;pp. 43-50. custom:[[[-]]]
  • 94 T. Fukuhara, H. Nakagawa, T. Nishida, "Understanding sentiment of people from news articles: temporal sentiment analysis of social events," in Proceedings of the 1st International Conference on W eblogs and Social Media (ICWSM), Boulder, CO, 2007;custom:[[[-]]]
  • 95 S. Havre, E. Hetzler, P. Whitney, L. Nowell, "ThemeRiver: visualizing thematic changes in large document collections," IEEE Transactions on Visualization and Computer Graphics, vol. 8, no. 1, pp. 9-20, 2002.doi:[[[10.1109/2945.981848]]]
  • 96 A. Marcus, M. S. Bernstein, O. Badar, D. R. Karger, S. Madden, R. C. Miller, "TwitInfo: aggregating and visualizing microblogs for event exploration," in Proceedings of the SIGCHI Conference on Human Factors Computing Systems, V ancouver, Canada, 2011;pp. 227-236. custom:[[[-]]]
  • 97 N. Diakopoulos, M. Naaman, F. Kivran-Swaine, "Diamonds in the rough: social media visual analytics for journalistic inquiry," in Proceedings of 2010 IEEE Symposium on Visual Analytics Science and T echnology, Salt Lake City, UT, 2010;pp. 115-122. custom:[[[-]]]
  • 98 D. Duan, W. Qian, S. Pan, L. Shi, C. Lin, "VISA: a visual sentiment analysis system," in Proceedings of the 5th International Symposium on Visual Information Communication and Interaction, Hangzhou, China, 2012;pp. 22-28. custom:[[[-]]]
  • 99 M. Doyle, A. F. Smeaton, A. Bermingham, "T riVis: visualising multivariate data from sentiment analysis," in Proceedings of the 8th Annual Irish Human Computer Interaction (iHCI) Conference, Dublin, Ireland, 2014;custom:[[[-]]]
  • 100 A. J. W ecker, J. Lanir, O. Mokryn, E. Minkov, T. Kuflik, "Semantize: visualizing the sentiment of individual document," in Proceedings of the 2014 International W orking Conference on Advanced Visual Interfaces, Como, Italy, 2014;pp. 385-386. custom:[[[-]]]
  • 101 C. Pino, I. Kavasidis, C. Spampinato, "Assessment and visualization of geographically distributed event-related sentiments by mining social networks and news," in Proceedings of 2016 13th IEEE Annual Consumer Communications Networking Conference (CCNC), Las V egas, NV, 2016;pp. 354-358. custom:[[[-]]]
  • 102 Y. Liu, X. Huang, A. An, X. Y u, "ARSA: a sentiment-aware model for predicting sales performance using blogs," in Proceedings of the 30th Annual International ACM SIGIR Conference on Research and Development Information Retrieval, Amsterdam, The Netherlands, 2007;pp. 607-614. custom:[[[-]]]
  • 103 M. McGlohon, N. Glance, Z. Reiter, "Star quality: aggregating reviews to rank products and merchants," in Proceedings of the 4th International AAAI Conference on W eblogs and Social Media, W ashington, DC, 2010;custom:[[[-]]]
  • 104 S. Asur, B. A. Huberman, "Predicting the future with social media," in Proceedings of the 2010 IEEE/WIC/ACM International Conference on Web Intelligence and Intelligent Agent Technology, Toronto, Canada, 2010;pp. 492-499. custom:[[[-]]]
  • 105 M. Joshi, D. Das, K. Gimpel, N. A. Smith, "Movie reviews and revenues: an experiment in text regression," in Human Language T echnologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics, Los Angeles, CA, 2010;pp. 293-296. custom:[[[-]]]
  • 106 E. Sadikov, A. Parameswaran, P. Venetis, "Blogs as predictors of movie success," in Proceedings of the 3rd International AAAI Conference on W eblogs and Social Media, San Jose, CA, 2009;custom:[[[-]]]
  • 107 H. Ihm, "Mining consumer attitude and behavior: an exploratory study on movie audience attitude extracted from Twitter," Journal of Convergence, vol. 4, no. 1, pp. 29-35, 2013.custom:[[[-]]]
  • 108 J. Bollen, H. Mao, X. Zeng, "T witter mood predicts the stock market," Journal of Computational Science, vol. 2, no. 1, pp. 1-8. 2011. custom:[[[-]]]
  • 109 R. Bar-Haim, E. Dinur, R. Feldman, M. Fresko, G. Goldstein, "Identifying and following expert investors in stock microblogs," in Proceedings of the Conference on Empirical Methods Natural Language Processing, Edinburgh, UK, 2011;pp. 1310-1319. custom:[[[-]]]
  • 110 R. Feldman, B. Rosenfeld, R. Bar-Haim, M. Fresko, "The stock sonar: sentiment analysis of stocks based on a hybrid approach," in Proceedings of the 23rd Annual Conference on Innovation Applications of Artificial Intelligence, San Francisco, CA, 2011;custom:[[[-]]]
  • 111 T rackur (Online). Available from:, https://www.crunchbase.com/organization/trackur
  • 112 BrandsEye (Online). Available:, http://www.brandseye.com
  • 113 TweetFeel (Online). Available:, https://www.crunchbase.com/organization/tweetfeel
  • 114 Twitrratr (Online). Available:, https://www.crunchbase.com/organization/twittrratr
  • 115 T rendrr (Online). Available:, https://www.crunchbase.com/organization/trendrr
  • 116 Y. Hong, S. Skiena, "The wisdom of bookies? Sentiment analysis vs. the NFL point spread," in Proceedings of the 4th International AAAI Conference on W eblogs and Social Media, W ashington, DC, 2010;pp. 251-254. custom:[[[-]]]
  • 117 A. Tumasjan, T. O. Sprenger, P. G. Sandner, I. M. Welpe, "Predicting elections with twitter: what 140 characters reveal about political sentiment," in Proceedings of the 4th International AAAI Conference on W eblogs and Social Media, W ashington, DC, 2010;pp. 178-185. custom:[[[-]]]
  • 118 J. DiGrazia, K. McKelvey, J. Bollen, F. Rojas, "More tweets, more votes: social media as a quantitative indicator of political behavior," PloS One, vol. 8 no.11, no. vol.8 11, 2013.doi:[[[10.2139/ssrn.2235423]]]
  • 119 F. Franch, "(Wisdom of the Crowds)2: 2010 UK election prediction with social media," Journal of Information T echnology Politicsno, 1, vol. 10, pp. 57-71, 2013.doi:[[[10.1080/19331681.2012.705080]]]
  • 120 A. Ceron, L. Curini, S. M. Iacus, G. Porro, "Every tweet counts? How sentiment analysis of social media can improve our knowledge of citizens’ political preferences with an application to Italy and France," New Media Society, vol. 16, no. 2, pp. 340-358, 2014.doi:[[[10.1177/1461444813480466]]]
  • 121 G. Caldarelli, A. Chessa, F. Pammolli, G. Pompa, M. Puliga, M. Riccaboni, G. Riotta, "A multi-level geographical study of Italian political elections from Twitter data," PloS One, vol. 9, no. 5, 2014.doi:[[[10.1371/journal.pone.0095809]]]
  • 122 P. Burnap, O. Rana, M. Williams, W. Housley, A. Edwards, J. Morgan, L. Sloan, J. Conejero, "COSMOS: towards an integrated and scalable service for analysing social media on demand," International Journal of ParallelEmergent and Distributed Systems, vol. 30, no. 2, pp. 80-100, 2015.doi:[[[10.1080/17445760.2014.902057]]]
  • 123 B. Chen, L. Zhu, D. Kifer, D. Lee, "What is an opinion about? Exploring political standpoints using opinion scoring model," in Proceedings of the 24th AAAI Conference on Artificial Intelligence, Atlanta, GA, 2010;pp. 1007-1012. custom:[[[-]]]
  • 124 P. A. Kerr, P. Moy, "Newspaper coverage of fundamentalist Christians, 1980–2000," Journalism Mass Communication Quarterly, vol. 79, no. 1, pp. 54-72, 2002.doi:[[[10.1177/107769900207900105]]]
  • 125 M. E. Len-Rios, S. Rodgers, E. Thorson, D. Yoon, "Representation of women in news and photos: comparing content to perceptions," Journal of Communication, vol. 55, no. 1, pp. 152-168, 2005.doi:[[[10.1093/joc/55.1.152]]]
  • 126 D. P. Haider-Markel, M. D. Allen, M. Johansen, "Understanding variations in media coverage of US Supreme Court decisions: comparing media outlets in their coverage of Lawrence v. Texas," Harvard International Journal of Press/Politics, vol. 11, no. 2, pp. 64-85, 2006.custom:[[[-]]]
  • 127 A. Kwon, K. S. Lee, "Opinion bias detection based on social opinions for twitter," Journal of Information Processing Systems, vol. 9, no. 4, pp. 538-547, 2013.doi:[[[10.3745/JIPS.2013.9.4.538]]]
  • 128 H. M. Kepplinger, "Mediatization of politics: theory and data," Journal of Communication, vol. 52, no. 4, pp. 972-986, 2002.doi:[[[10.1093/joc/52.4.972]]]
  • 129 W. L. Benoit, A. Harthcock, "Functions of the great debates: acclaims, attacks, and defenses in the 1960 presidential debates," vol.66, no. 4, pp. 341-357, 1999.doi:[[[10.1080/03637759909376484]]]
  • 130 C. H. De Vreese, H. A. Semetko, "Cynical and engaged: strategic campaign coverage, public opinion, and mobilization in a referendum," Communication Research, vol. 29, no. 6, pp. 615-641, 2002.custom:[[[-]]]
  • 131 S. J. Farnsworth, S. R. Lichter, "Source Material: New presidents and network news: covering the first year in office of Ronald Reagan, Bill Clinton, and George W . Bush," Presidential Studies Quarterly, vol. 34, no. 3, pp. 674-690, 2004.custom:[[[-]]]
  • 132] S. J. Farnsworth and S. R. Lichter, "Reporting on two presidencies: news coverage of George W . Bush's first year in office, " Congress the Presidency: A Journal of Capital Studies, vol. 32, no. 2, pp. 91-108, 2005. [133] C. Cardie, C. R. Farina, and T . R. Bruce, "Using natural language processing to improve eRulemaking [Project Highlight Reporting on two presidencies: news coverage of George W . Bush's first year in office, " Congress the Presidency: A Journal of Capital Studies, vol. 32, no. 2, pp. 91-108, 2005. [133] C. Cardie, C. R. Farina, and T . R. Bruce, "Using natural language processing to improve eRulemaking [Project Highlight-sciedit-2-03" Farnsworth , S. J. , & Lichter , S. R. ( 2005 ). Reporting on two presidencies: news coverage of George W . Bush's first year in office . Congress the Presidency: A Journal of Capital Studies. [133 , 32 ( 2 ), 91 - 108 , custom:[[[ - ]]].
  • 134 N. Kwon, S. W. Shulman, E. Hovy, "Multidimensional text analysis for eRulemaking," in ", "ACM International Conference Proceeding Series, 2006;vol. 151, pp. 157-166. custom:[[[-]]]
  • 135 S. Shulman, E. Hovy, J. Callan, S. Zavestoski, "Language processing technologies for electronic rulemaking: a project highlight," in Proceedings of the 2005 National Conference on Digital Government Research, 2005;pp. 87-88. custom:[[[-]]]
  • 136 J. G. Conrad, F. Schilder, "Opinion mining in legal blogs," in Proceedings of the 11th International Conference on Artificial Intelligence and Law, Stanford, CA, 2007;pp. 231-236. custom:[[[-]]]
  • 137 G. Groh, J. Hauffa, "social relations via NLP-based sentiment analysis," in Proceedings of the 5th International AAAI Conference on W eblogs and Social Media, Barcelona, Spain, 2011;pp. 502-505. custom:[[[-]]]
  • 138 S. M. Mohammad, T. W. Yang, "Tracking sentiment in mail: how genders differ on emotional axes," in Proceedings of the 2nd W orkshop on Computational Approaches to Subjectivity and Sentiment Analysis, Portland, OR, 2011;pp. 70-79. custom:[[[-]]]
  • 139 P. Sakunkoo, N. Sakunkoo, "Analysis of social influence in online book reviews," in Proceedings of the 3rd International AAAI Conference on W eblogs and Social Media, San Jose, CA, 2009;pp. 308-310. custom:[[[-]]]
  • 140 Y. Wang, S. Wang, J. Tang, H. Liu, B. Li, "Unsupervised sentiment analysis for social media images," in Proceedings of the 24th International Joint Conference on Artificial Intelligence, Buenos Aires, Argentina, 2015;pp. 2378-2379. custom:[[[-]]]
  • 141 S. Tedmori, R. Al-Lahaseh, "Towards a selfie social network with automatically generated sentiment-bearing hashtags," in Proceedings of 2016 7th International Conference on Computer Science and Information Technology (CSIT), Amman, Jordan, 2016;pp. 1-6. custom:[[[-]]]