PolyakovPYuKalininaMVPleshkoVV (Аннотации), страница 3
Описание файла
Файл "PolyakovPYuKalininaMVPleshkoVV" внутри архива находится в следующих папках: Аннотации, 3. PDF-файл из архива "Аннотации", который расположен в категории "". Всё это находится в предмете "английский язык" из 10 семестр (2 семестр магистратуры), которые можно найти в файловом архиве МГУ им. Ломоносова. Не смотря на прямую связь этого архива с МГУ им. Ломоносова, его также можно найти и в других разделах. .
Просмотр PDF-файла онлайн
Текст 3 страницы из PDF
F1-measure macro- and micro-averagedwas used as a primary evaluation metric [1]. Additionally, for convenience, recall andprecision are also present in the tables. As shown in Table 1, the estimation of tweetsby our expert differed from one granted by the organizers. We consider the score givenby our expert as the highest possible for an automatic sentiment detection system forthe given collection. The agreement between our expert and organizers’ labeling washigher when we excluded news from the dataset, which confirms our assumption thata different approach should be used for sentiment analysis of news.Table 1.
The estimation of coincidencebetween expert and assessorsWith newsWithout newsMacro-averageMicro-averageRecallRecall0.7220.785Precision F10.686 0.7030.694 0.7370.7710.831Precision F10.728 0.7490.735 0.780The results of all participants are shown in Fig. 1, our results are highlightedby bold lines and are labeled as “RCO”. It is interesting that several methods probablybased on different approaches demonstrate very similar high scores of F1 (about 0.5),nevertheless, these scores are sufficiently less than theoretical maximum that corresponds to coincidence between assessors (see bars “Expert” on Fig.
1). It could provethat automatic sentiment detection task is still a challenging problem.Polyakov P. Yu., Kalinina M. V., Pleshko V. V. 0,8F10,6macro0,4micro0,20Fig. 2. Macro- and micro-averaged F1 measure calculated on testcollection for all participants. The scores for our method are labeledas “RCO”.
The scores of expert’s evaluation are labeled as “expert”The detailed results of our method are presented in Table 2. We calculated recall,precision and F1 for original collection (labeled as “With news”) and for collectionwith exclusion of messages contained news and press releases (labeled as “Withoutnews”). For comparison, the best scores among the methods of all participants arepresented.Table 2. The performance of our method and bestF1 measure among the methods of all participantsWith newsWithout newsBest resultMacro-averageMicro-averageRecallRecall0.4360.465Precision F10.566 0.4800.562 0.4920.4920.4510.475Precision F10.585 0.5090.583 0.5240.5366. ConclusionOur combined linguistic method showed a very high quality, which roughly coincides with the best results of machine learning methods and hybrid approaches (combining machine learning with elements of syntactic analysis).
In the future we areplanning to add machine learning to our linguistic approach.Automatic Object-oriented Sentiment AnalysisReferences1.Blinov P. D., Kotelnikov E. V. (2014), Using distributed representations for aspectbased sentiment analysis, Dialog ’14, Bekasovo.2. Chetviorkin I., Braslavski P. I., Loukachevitch N. (2012), Sentiment analysis trackat ROMIP 2011, Bekasovo.3. Ermakov A. E., Pleshko V. V. (2009), Abstract Semantic Interpretation in ComputerText Analysis Systems [Semanticheskaya interpretatsiya v sistemakh kompyuternogo analiza teksta], Information Technologies [Informacionnye tehnologii],Vol. 6, pp.
2–7.4. Ermakov A. E. (2009), Knowledge Extraction from Text and its Processing: Current State and Prospects [Izvlecheniye znaniy iz teksta i ikh obrabotka: sostoyaniye i perspektivy], Information Technologies [Informacionnye tehnologii],Vol. 7, pp. 50–55.5. Hu M., Liu B. (2004), Mining and summarizing customer reviews, InternationalConference on Knowledge Discovery and Data Mining (ICDM).6.
Jakob N., Gurevych I. (2010), Extracting Opinion Targets in a Single-and CrossDomain Setting with Conditional Random Fields, Proceedings of Conferenceon Empirical Methods in Natural Language Processing (EMNLP-2010).7. Kan D. (2012), Rule-based approach to sentiment analysis at ROMIP ’11 , Bekasovo.8. Loukachevitch N., Blinov P., Kotelnikov E., Rubtsova Yu., Ivanov V., Tutubalina E.(2015), SentiRuEval Testing Object-Oriented Sentiment Analysis Systems in Russian.9. Popescu A., Etzioni O. (2005), Extracting product features and opinions from reviews, Proceedings of Conference on Empirical Methods in Natural LanguageProcessing (EMNLP) .10. Polyakov P. Yu., Kalinina M. V., Pleshko V. V. (2012), Research of applicabilityof thematic classification to the problem of book review classification.
Dialog ’12.Naro-Fominsk.11. Polyakov P. Yu., Frolov A. V., Pleshko V. V. (2013), Using semantic categories in application to book reviews sentiment analysis, Dialog’13, Bekasovo..
