аннотация9 (Аннотации)

Mikhail Korobov:Morphological Analyzer and Generator for Russian and UkrainianLanguages.The article presents Pymorphy2 (a morphological analyzer and generator forRussian and Ukrainian languages). It uses large efficiently encoded lexicons builtfrom OpenCorpora and LanguageTool data. A set of linguistically motivated rulesis developed to enable morphological analysis and generation of out-of-vocabularywords observed in real-world documents.Pymorphy2 uses deterministic acyclic finite state automaton (DAFSA) forcorpora representation. It allows more efficient morphological information retrievalthen XML queries.

The words can be also factorized in stem plus paradigm form,where paradigm is an inflection pattern of a lexeme. To analyzeout-of-vocabulary words instrument uses such techniques as: common prefixesremoval, searching words ending with other dictionary words, common endingsmatching, analysis of hyphenated words. Pymorphy2 provides morphologicaldisambiguation based on probability estimation of “analysis” under condition“word”.The author evaluates the comparison with MyStem analyzer on disambiguatedsentences from OpenCorpora and Russian National Corpus. Both instrumentsshowed similar high results. Previous version of Pymorphy2 participated inmorphological analyzers competition on Dialogue conference and finished 1st onfull morphology analysis, 3rd on lemmatization, 3rd on POS tagging and 5th on therare words track.

For now, Pymorphy2 provides state-of-the-arts morphologicalanalysis quality..