PolyakovPYuKalininaMVPleshkoVV (Аннотации), страница 2

Forexample, (6) У Билайна надежная связь is a positive statement, while adding theadverb наименее changes its sentiment to opposite: (7) У Билайна наименее надежная связь. By the means of forbidding nodes we can distinguish between these twosentences, stating that the adjective should not be modified by the adverb наименее.Usage of forbidding nodes significantly increases the precision of sentiment analysis.Fig. 1 demonstrates a semantic template used to detect sentiment expressedby a verb or adverb in sentences like: (8) Билайн ловит хорошо; Интернет летает.Fig.

1. Example of a semantic templateAutomatic Object-oriented Sentiment AnalysisNodes contain restrictions on parts of speech (SpeechPart == “Verb”; SpeechPart== “Adverb”), lexical items (Name == “ПЕРЕСТАТЬ” or Name == “ПРЕКРАТИТЬ”),semantic categories (SemanticType == “Organization:Name” or SemanticType ==“Attribute:Mobile”). Restrictions on semantic and syntactic relations between wordsinclude: relation name (RelationName == “аргумент»; RelationName == «квантор»), semantic role (RelationRole == “субъект»), case (RelationCase == “И”).

Forbidding nodes state that the verb expressing sentiment should not be controlled by theverbs перестать or прекратить or modified by the predicative должен. Thus, thistemplate will match the sentence (8) Билайн хорошо ловит (which is positive), butnot (9) Билайн перестал хорошо ловить (which is negative) or (10) Билайн долженхорошо ловить (which we consider neutral).Restrictions of the semantic templates were enriched by the use of special dictionaries (so-called filters), containing vocabulary for positive and negative appraisals. This vocabulary includes nouns, adjectives, verbs, adverbs, and collocations.A word from a filter must be syntactically related to the target of evaluation.

Selectionof terms for the filters was manual, performed by a linguistic expert. Examples of positive terms: супербыстрый, шустро, красота, крутяк, блистать, радовать, обеспечивать уверенный прием. Examples of negative terms: завышенный, препротивнейший, позорище, тормознутость, обдирать, терять соединение, фигово.For example, a set of particular words from the semantic filters are applied to thetemplate in Fig.1 as restrictions: verbs or verbal nouns parameterize the node with therestriction SpeechPart == “Verb” or SemanticType == “Event”; adverbs parameterize the node with the restriction SpeechPart == “Adverb”, both these nodes have thesemantic role ‘Appraisal’.Ultimate targets of evaluation were main Russian mobile phone providers (Beeline, Megafon, MTS, Rostelecom, Tele2), but also users’ appraisals of providers’ attributes were taken into account (communication quality, mobile Internet, customerservice, etc.).Analyzing users’ comments and opinions on social networking sites and forumsexperts defined a set of attributes which were most frequently mentioned by mobilephone users.

Thus, a list of most important things for users was made. Given attributes were divided into three classes: 1) Mobile Attributes—terms strictly connectedto the mobile telephony: SMS, MMS, 3G, LTE, SIM-card, roaming, etc.; 2) InternetAttributes—terms strictly connected to the Internet: Internet, ping, etc.; 3) GeneralAttributes—terms often used related to the mobile telephony but which can also refer to other domains: call center, signal, network, customer support, balance, etc.

Eachlist was extended by synonyms and spelling variants (интернет=инет=и-нет;lte=лте =lteшечка =лте-шечка; баланс счета=состояние счета=средствана счету=деньги на счету, etc.). When a sentiment related to a certain attribute wasdetected, given sentiment was also ascribed to the corresponding mobile provider.In Fig.1 the node with the restriction SemanticType == “Organization:Name” orSemanticType == “Attribute:Mobile” or SemanticType == “Attribute:Internet”is parameterized by names of mobile operators, mobile attributes or Internet attributes; the semantic role of the node is ‘Target Of Evaluation’.This method provides a very high precision, though not so high recall.Polyakov P. Yu., Kalinina M.

V., Pleshko V. V. 3.2.‘Free’ SentimentAlthough usage of semantic templates provides very good accuracy, this methodhas its disadvantage—a word expressing sentiment must be in the same sentence as thetarget of evaluation and must be syntactically related to it. As it is not always so in natural texts, some cases of clearly expressed sentiment will be omitted by this method,and the recall will suffer. This problem becomes extremely significant when we analyzeinformal texts—forums, social networking websites, blogs, etc. Writing an informaltext message, users often disregard punctuation and spelling rules, mistype, becauseof which the syntactic parser may fail to correctly analyze the structure of a sentenceand build a semantic network.

Users often express their sentiment through interjections,which are not a part of the syntactic tree; hence the semantic templates are of no usein this case. We call words that express sentiment but have no syntactic relation to thetarget of evaluation (or such relation has not been built by the parser) ‘free sentiment’.To solve this problem another method has been applied. We used an algorithm whichis looking for free sentiment in the text using dictionaries (or profiles) of positive and negative lexicon, and if such sentiment has been found tries to relate it to the target object.These two methods complement each other, with the semantic template methodworking first.

In this regard, the classifier ‘ignores’ terms already found and relatedto the target object by templates, because we assume that the accuracy provided by thesemantic templates is close to 100%.As profiles for positive and negative classes we used corresponding filters, having removed context-dependent sentiment words and leaving only explicit emotionalor evaluative vocabulary.

For example, we removed verbs УМЕРЕТЬ, ПРОИГРЫВАТЬ,because although they are obviously negative in the context like: (11) интернетумер; (12) оператор Х проигрывает оператору Y; but in another context, not related to the mobile telephony, they may be neutral and just state a fact. At the sametime we enriched our profiles with interjections and other emotional expressionswhich cannot be syntactically related to the object of evaluation, for example: (13)не надо так! что за нах; ни фига себе; ну как так можно, etc.Having found a sentiment, our algorithm was looking for an object of evaluation—a name of a mobile company—in the given text and ascribed this sentimentto the target.

If several mobile operators were mentioned in the text, the appraisal wasascribed to the nearest operator. If both positive and negative sentiment was detectedrelated to the same mobile provider mentioned, we gave preference to the negativesentiment, regarding positive expressions as sarcasm.No machine learning had been used. The methods applied were based on linguistic analysis only.4. DatasetThe training and test collection granted by organizers consisted of 5,000 labeledand 5,000 not labeled tweets containing sentiment-oriented opinions or positive andnegative facts about telecom companies.Automatic Object-oriented Sentiment AnalysisAs the main goal of social networks sentiment analysis is to find sentiment-oriented opinions, we labeled texts containing reprints of news and additionally measured sentiment detection quality for the training collection with news reprints excluded.

We excluded news texts from the final dataset because we believe that thedifference in structure and vocabulary between formal (news) and informal (posts,blogs, tweets) texts is crucial. As a rule, in news texts authors don’t express their attitude openly; news is more likely to contain coverage of events and facts, which canbe interpreted as positive or negative for the newsmaker, rather than explicit sentiment; and therefore analyzing news demand a different approach. Furthermore, vocabulary of informal texts is quite different from vocabulary of formal texts.That is why we additionally estimated the method performance on the collectionwith news reprints and companies’ press releases excluded from the dataset. Since ourmethod is based on linguistic analysis only, we did not use training collection.5.

ResultsInitially, for the purpose of estimation of coincidence between assessorswe asked our expert to evaluate the test collection manually and marked each reference to mobile phone companies as being positive, negative or neutral. Results of ourexpert’s evaluation are presented in Table 1.