Towards Metadata Analysis on Opinionated Content in Tweets

Anderson Almeida Firmino

, Cláudio de Souza Baptista

, André Luiz Firmino Alves

1,2

Davi Oliveira Serrano de Andrade

, Hugo Feitosa de Figueirêdo

Geraldo Braz Junior

and Anselmo Cardoso de Paiva

Information Systems Laboratory, University of Campina Grande, Campina Grande, Brazil

Information Technology Coordination, State University of Paraiba, Campina Grande, Brazil

Federal Institute of Technology of Paraiba, Esperança, Brazil

Applied Computing Group, Federal University of Maranhão, São Luiz, Brazil

Keywords: Opinion Mining, Sentiment Analysis, Tweets.

Abstract: Recently, much research has been done in the area of sentiment analysis of microtexts, specially using tweets.

In most studies, the sentiment polarity detection methods are solely based on textual information. The

detection of opinionated content in texts is not a simple task, and even less simple in the context of social

media. Furthermore, processing microtexts using just natural language techniques may lead to unsatisfactory

results. There is a lack of works which link other properties of the tweets (metadata), such as retweets and

likes, and the their opinion (i.e., the presence of sentiments). Using tweets collected during the 2013 FIFA

Confederations Cup, which occurred in Brazil, this work proposes an analysis of metadata properties on

tweets, in order to verify which of these properties have more impact on their opinionatedness. The results

indicate that the properties “presence of links” and “retweets” are the most significant with respect to the

opinionatedness of a tweet.

1 INTRODUCTION

Understanding what people think, i.e., knowing their

opinions, is a fundamental part of the decision-

making process, especially in the context in which

they express their feelings voluntarily in order to

cooperate with one another. The growth of social

media propitiated by the WEB 2.0 has led to the

generation of a large volume of non-structured textual

data. Microblogging is a very popular means of

communication among Internet users (Pak and

Paroubek, 2010). The messages shared by the users

concern not only their private lives, but also current

affairs, products, services and general events.

Websites that provide microblogging services, such

as Twitter, have been subject of study in the field of

sentiment analysis, with the purpose of generating

content recommendation tools, security tools, and

many other applications (Alves et al., 2014; Pak and

Paroubek, 2010).

According to Liu (2012), the main objective of

sentiment analysis is to obtain and formalize the

opinion and the subjective knowledge contained in

non-structured documents (texts), for a posterior

analysis in a specific domain. The sentiment analysis

process can be defined by three major tasks:

identification, classification and summarization (Liu,

2012; Tsytsarau and Palpanas, 2012). The

identification task may include, besides the

recognition of entities and their aspects, the

recognition of subjective/opinionated sentences. In

the classification process, which is the main task in

sentiment analysis applications, the goal is to obtain

the polarity of the sentiment. The summarization, in

turn, is intended to obtain metrics and summaries that

represent the general sentiment of a group of people

about either a certain entity or the aspects of that

entity. In most studies in the field of sentiment

analysis, just the textual information in each tweet is

analyzed. The main proposed methodologies employ

Natural Language Processing or Machine Learning in

order to classify the polarity of the sentiments

expressed in tweets (Sharma and Dey, 2012).

According to Suh et al. (2010), a tweet contains,

besides the textual information, content and context

properties, apart from the textual information, content

314

Firmino A., Baptista C., Alves A., Andrade D., FigueirÃłdo H., Filho G. and de Paiva A.

Towards Metadata Analysis on Opinionated Content in Tweets.

DOI: 10.5220/0005890803140320

In Proceedings of the 18th International Conference on Enterprise Information Systems (ICEIS 2016), pages 314-320

ISBN: 978-989-758-187-8

and context properties describe metadata. The content

properties, that can be found in the tweet, include

URLs, hashtags and mentions (references to other

users). The context properties, on the other hand,

include the number of followers of a user, the number

of likes in a tweet, number of retweets and many

others. According to Harris et al. (2015), the act of

liking a tweet shows that the user agrees with its

content or with the opinion it expresses. Hence, if

there is a tweet with positive sentiment polarity and

ten likes, this means that, besides the author, other ten

people agree with that opinion (Meier et al., 2014). It

is possible to make a more thorough sentiment

analysis, taking into account the impact that a tweet

has over its followers.

Detecting opinionated content in texts is not a

simple task, especially in microtexts, since they may

contain abbreviations, repetition of letters and typing

errors. In general, the use of text processing

techniques alone may lead to unsatisfactory results.

In this scenario, Alves (2014) suggest the exploration

of other properties (metadata) of the tweets besides

the textual message in order to provide improvements

in the accuracy of the polarity detection. The

exploration of additional attributes on Twitter allows

the discovery of other attributes contained in their

metadata. These attributes may help to identify

opinionated content, which is very important in the

sentiment analysis process.

This study explores the identification of

opinionated content in the context of the sentiment

analysis. The main goal is to verify which attributes

of a tweet contribute to the identification of

opinionated sentences, in order to improve the

polarity classification task. The metadata attributes of

interest in this work are: likes, mentions, retweets,

links, and replies. The main contribution of this work

is the investigation based on statistical analysis, in

order to verify whether there are metadata attributes

that are significantly important to the identification of

opinionated content in tweets.

The rest of this paper is organized as following.

In Section 2, we analyze the related works. In Section

3, we address the methodology adopted in this study.

In Section 4, we highlight the results. Finally, in

Section 5, we present the conclusions and discuss

further work to be undertaken.

2 RELATED WORK

Many studies in the area of sentiment analysis obtain

the sentiment polarity of a tweet just based on its

textual information (Alves et al., 2014; Pak and

Paroubek, 2010). Pak and Paroubek (2010) used

Naive-Bayes text classifiers and techniques for

grammatical classification of words (POS-Tagging)

to identify sentiment in tweets written in English. So,

no human effort was needed to classify the texts.

Alves et al. (2014) use a similar approach to that

of Pak and Paroubek, with the help of a Naive-Bayes

text classifier. However, they collected tweets written

in Portuguese. In Portuguese, the use of grammatical

classification in order to obtain the sentiment of a text

is not a simple task since, besides the problems

concerning the texts of the tweets themselves

(abbreviations, repetition of letters, among others),

there is also the grammatical complexity of the

language. Their work proposes a text classifier that

uses Natural Language Processing and Supervised

Learning techniques to detect the polarity of

sentiment in tweets. By doing so, they avoided the use

of grammatical classification with the texts (POS

Tagging).

Tsai et al. (2013) propose the building of a

dictionary at concept level with sentiment values

based on common knowledge. The authors suggest

not just the concepts dictionary, but also the way it is

built. They use a two-step method combining iterative

regression and random walk with in-link

normalization. The dictionary is built based on

common concepts and relationships between the so-

called “seed words” to propagate the value of the

sentiment among the concepts.

Poria et al. (2013) present a methodology to

automatically assign emotional labels to the concepts

present in SenticNet (Cambria et al., 2010), in order

to improve the results of the sentiment analysis. They

used SVM as a classifier. The training of the machine

was conducted with a subset of concepts of SenticNet

(Cambria et al., 2010). They used characteristics of

the authors of the messages in the analysis (age,

gender, parent's occupation, etc).

Weichselbraun et al. (2013) used a lexical

dictionary, considering the context of both the word

and the message, in order to execute the sentiment

analysis of text messages. The ambiguities were

removed by means of context analysis, with the use

of frequency graphs and even Bayesian networks for

detection of the context of the term. The combination

of these dictionaries is used to perform the sentiment

analysis.

Xia et al. (2013) execute the sentiment analysis

considering POS-tags and separation of domains in

order to enhance the sentiment associated with each

word. They execute the sentiment analysis using

sentiment associated with the words, POS-tags, and

Bayesian Networks.

Towards Metadata Analysis on Opinionated Content in Tweets

315

Cambria et al. (2013a) make an introduction to

sentiment analysis techniques that employ knowledge

bases. Their work represents an important study in

this field and summarizes some contemporary work.

They also divide the opinion mining problem into two

areas: Natural Language Processing (NLP) and

Language Interpretation (Cambria et al, 2013b).

However, they neither indicate solutions nor point out

the main characteristics used to execute the sentiment

analysis.

Hogenboom et al. (2015) perform the sentiment

analysis of documents by combining several

sentences in order to identify the general sentiment

through Rhetorical Structure Theory (RST). They

created an RST-based tree, by which they perform the

combination of sentiments. However, they do not

identify which characteristics are more relevant to the

sentiment analysis.

Liu et al. (2015) propose a multi-label approach

for classification of sentiment in microblogs.

Additionally, they present a comparative study

between different multi-label methods for

classification of text in microblogs. They also

presente a comparative study on the effects of

different sentiment dictionaries over the multi-label

classifiers.

Cambria et al. (2014) present an approach that

uses an open-domain knowledge base (i.e., not

concerned with a specific domain of content) to

execute the opinion mining and sentiment analysis.

Furthermore, they use a "Bag of Concepts" together

with the multidimensional knowledge base built.

Rosas et al. (2013) present a complete approach

for sentiment analysis of videos. They use the

linguistic (texts transcribed from the video), visual

and audio data to identify the sentiment associated

with the video. They execute the sentiment analysis

in these data separately and then combine the results

into a single sentiment.

Wollmer et al. (2013) present a similar approach

to that of Rosas et al. (2013), in which they use

linguistic, visual and audio data of YouTube videos

to perform sentiment analysis. They join the

characteristics in order to find the sentiment

associated with the video but do not make clear which

of these characteristics are more relevant to the

analysis.

Other works analyze the context of the properties

of a tweet, such as the number of retweets, for

example. Meier et al. (2014) conducted a study in

order to understand the behavior of the “like”

functionality on Twitter. They found that the act of

“retweeting” indicates that the user considers the

information to be interesting enough to be forwarded

to their followers. The act of “liking”, on the other

hand, indicates that the user simply agrees with the

content of the tweet.

Some studies have attempted to establish a

relationship between some of the context properties

of the content of a tweet and its opinionatedness.

Stieglitz and Dang-Xuan (2012) and Pfitzner et al.

(2012) established a relationship between the opinion

present in a tweet and its likelihood to be retweeted.

According to Stieglitz and Dang-Xuan (2012), tweets

that contain more words with either positive or

negative sentiment tend to be more retweeted.

Pfitzner et al. (2012) on the other hand, conclude that

emotionally diversified tweets, i.e., tweets containing

words with both positive and negative sentiments,

have fivefold chances of being retweeted.

The literature presents solutions to sentiment

analysis, but, to the best of our knowledge, none of

the works is intended to analyze which of the

content/context properties of a tweet are more closely

related to its opinionatedness. So, the main

contribution of this article is the discovery of which

metadata characteristics are more relevant to the

sentiment analysis in the context of Twitter.

Furthermore, we present a logistic regression model

used to identify those characteristics.

3 METHODOLOGY

In this section, we describe the methodology used in

the development of our experiment. It is presented in

two subsections: experiment configuration, which

describes the dataset used and the hypothesis raised

about each aspect under analysis; and experiment

execution, which describes the creation of a logistic

regression model based on that data.

3.1 Experiment Configuration

In the work by Alves (2014), he collected about

120.000 tweets concerning the 2013 FIFA

Confederations Cup, with the objective of developing

a sentiment polarity classifier. To this end, he

separated a set containing 3,500 tweets (labelled as a

gold standard dataset) which were used for training

and testing of the classifiers. After implementing the

sentiment polarity classifier, the author classified all

the collected tweets with a mean accuracy of 80%.

Since one of the goals of this work is to verify

which of the properties of a tweet can be used for the

detection of opinionated tweets (i.e., tweets in which

users express their opinions), we use the tweets

labelled in the work by Alves (2014) (gold standard

dataset and the set labelled by the classifiers). In order

ICEIS 2016 - 18th International Conference on Enterprise Information Systems

316

to generate the logistic regression model, we opted to

use the gold standard dataset instead of the set of the

tweets labelled by the classifier. In doing so, we

intended to minimize the introduction of errors in the

model. Hence, the set of all the collected tweets

(about 120,000) was only used to perform a

comparison between the layout of their metadata and

those of the rest of the tweets. The tweets which had

the sentiment polarity classified either as positive or

as negative were considered opinionated tweets while

those classified as neutral were considered

informative tweets.

It is important to highlight that the methodologies

implemented in other studies on sentiment analysis

only use the textual information of the tweets

(maximum of 140 characters) (Alves et al., 2014; Pak

and Paroubek, 2010). However, a tweet contains,

besides the text written by the author, other pieces of

information added implicitly by Twitter. These

metadata may inform, for example, the time and the

geographic location of the user at the moment the

message was sent. Besides the text of the tweets, we

explored the following metadata:

1. Replies – indicates if a tweet was replied by some

user;

2. Likes (favourites) – indicates if a tweet was

marked as favourite (liked) by some user;

3. Retweets – indicates if the tweet was the cause of

another tweet sent by another user;

4. Mentions – quantifies the mentions to other users

of the network;

5. Links (URLs) in the text – indicates if the tweet

contains links to external websites.

In short, the experiment is intended to help in the

task of identification of opinionated tweets through an

analysis of the correlation between the metadata

listed in the previous section and the opinionatedness

of a tweet. This way, in order to check which

metadata are connected to the opinionatedness of a

tweet, some hypotheses were created based on the

following hypothesis model:

"The existence of Mi in a tweet is not significant

to determine the opinionatedness of tweet", where Mi

is one of the metadata explored by this work (e.g.

replies, likes, retweets, mentions and links).

The identification of the hypotheses follows the

same pattern of the identification of metadata.

This way, let H be the set of hypotheses and Hi-0

the hypothesis related to the characteristic Mi. The

hypotheses are:

1. H1-0: The existence of replies in a tweet is not