An introduction to information retrieval. Manning_ Raghavan (2009) (811397), страница 46
Текст из файла (страница 46)
l); cf. Figure 6.15 (page 128). Pseudo relevance feedback consisted of adding 20terms to each query.how fast does a user find relevant documents with relevance feedback vs.another strategy (such as query reformulation), or alternatively, how manyrelevant documents does a user find in a certain amount of time. Such notions of user utility are fairest and closest to real system usage.9.1.6PSEUDO RELEVANCEFEEDBACKBLIND RELEVANCEFEEDBACK9.1.7IMPLICIT RELEVANCEFEEDBACKPseudo relevance feedbackPseudo relevance feedback, also known as blind relevance feedback, provides amethod for automatic local analysis. It automates the manual part of relevance feedback, so that the user gets improved retrieval performance without an extended interaction.
The method is to do normal retrieval to find aninitial set of most relevant documents, to then assume that the top k rankeddocuments are relevant, and finally to do relevance feedback as before underthis assumption.This automatic technique mostly works. Evidence suggests that it tendsto work better than global analysis (Section 9.2). It has been found to improve performance in the TREC ad hoc task. See for example the results inFigure 9.5. But it is not without the dangers of an automatic process.
Forexample, if the query is about copper mines and the top several documentsare all about mines in Chile, then there may be query drift in the direction ofdocuments on Chile.Indirect relevance feedbackWe can also use indirect sources of evidence rather than explicit feedback onrelevance as the basis for relevance feedback. This is often called implicit (relevance) feedback. Implicit feedback is less reliable than explicit feedback, but ismore useful than pseudo relevance feedback, which contains no evidence ofuser judgments.
Moreover, while users are often reluctant to provide explicitfeedback, it is easy to collect implicit feedback in large quantities for a highvolume system, such as a web search engine.Online edition (c) 2009 Cambridge UP1889 Relevance feedback and query expansionCLICKSTREAM MINING9.1.8On the web, DirectHit introduced the idea of ranking more highly documents that users chose to look at more often. In other words, clicks on linkswere assumed to indicate that the page was likely relevant to the query.
Thisapproach makes various assumptions, such as that the document summariesdisplayed in results lists (on whose basis users choose which documents toclick on) are indicative of the relevance of these documents. In the originalDirectHit search engine, the data about the click rates on pages was gatheredglobally, rather than being user or query specific. This is one form of the general area of clickstream mining. Today, a closely related approach is used inranking the advertisements that match a web search query (Chapter 19).SummaryRelevance feedback has been shown to be very effective at improving relevance of results.
Its successful use requires queries for which the set of relevant documents is medium to large. Full relevance feedback is often onerousfor the user, and its implementation is not very efficient in most IR systems.In many cases, other types of interactive retrieval may improve relevance byabout as much with less work.Beyond the core ad hoc retrieval scenario, other uses of relevance feedbackinclude:• Following a changing information need (e.g., names of car models of interest change over time)• Maintaining an information filter (e.g., for a news feed).
Such filters arediscussed further in Chapter 13.• Active learning (deciding which examples it is most useful to know theclass of to reduce annotation costs).?Exercise 9.3Under what conditions would the modified query q m in Equation 9.3 be the same asthe original query q0 ? In all other cases, is q m closer than q0 to the centroid of therelevant documents?Exercise 9.4Why is positive feedback likely to be more useful than negative feedback to an IRsystem? Why might only using one nonrelevant document be more effective thanusing several?Exercise 9.5Suppose that a user’s initial query is cheap CDs cheap DVDs extremely cheap CDs.
Theuser examines two documents, d1 and d2 . She judges d1 , with the content CDs cheapsoftware cheap CDs relevant and d2 with content cheap thrills DVDs nonrelevant. Assume that we are using direct term frequency (with no scaling and no documentOnline edition (c) 2009 Cambridge UP9.2 Global methods for query reformulation189frequency). There is no need to length-normalize vectors.
Using Rocchio relevancefeedback as in Equation (9.3) what would the revised query vector be after relevancefeedback? Assume α = 1, β = 0.75, γ = 0.25.Exercise 9.6[ ⋆]Omar has implemented a relevance feedback web search system, where he is goingto do relevance feedback based only on words in the title text returned for a page (forefficiency). The user is going to rank 3 results. The first user, Jinxing, queries for:banana slugand the top three titles returned are:banana slug Ariolimax columbianusSanta Cruz mountains banana slugSanta Cruz Campus MascotJinxing judges the first two documents relevant, and the third nonrelevant. Assumethat Omar’s search engine uses term frequency but no length normalization nor IDF.Assume that he is using the Rocchio relevance feedback mechanism, with α = β =γ = 1.
Show the final revised query that would be run. (Please list the vector elementsin alphabetical order.)9.2Global methods for query reformulationIn this section we more briefly discuss three global methods for expanding aquery: by simply aiding the user in doing so, by using a manual thesaurus,and through building a thesaurus automatically.9.2.1Vocabulary tools for query reformulationVarious user supports in the search process can help the user see how theirsearches are or are not working.
This includes information about words thatwere omitted from the query because they were on stop lists, what wordswere stemmed to, the number of hits on each term or phrase, and whetherwords were dynamically turned into phrases. The IR system might also suggest search terms by means of a thesaurus or a controlled vocabulary. A usercan also be allowed to browse lists of the terms that are in the inverted index,and thus find good terms that appear in the collection.9.2.2QUERY EXPANSIONQuery expansionIn relevance feedback, users give additional input on documents (by marking documents in the results set as relevant or not), and this input is usedto reweight the terms in the query for documents. In query expansion on theother hand, users give additional input on query words or phrases, possiblysuggesting additional query terms.
Some search engines (especially on theOnline edition (c) 2009 Cambridge UP1909 Relevance feedback and query expansionYeWbImagesVideoLocalShoppingmorOSpalearcptionlsotry:lptrhoo!MyYahoo!MailWelcome,Guest[SignIn]Helpshm1Aaeeo,Mormtrees,laep..msprings,lapmªc1elmbATtrfaobout5,3lp4,00,0000forpalm(Aboutthispage)ª0.1s1ec.ma.SPo0na&PONSORRESULTSHandhSPnneOlNdSsORaRtEDSelULTSlTaSatt.com/wirelesslGomobileeffortlesslywiththePALMTreofrota&TPl(CminHgualnadr)hCoectedwithmaHTAyndheldPCs&PDsA..eldShSiotpeatDell™Official.sawPaPlhm.otcoomsl&VOirdegaonizer,Planner,WiFi,MusicBluetooth,Gamesww.Dell.com,.BuyPlmCentroaCPlm,Incases.alUMakerofhandhelacdPDdAevicesthatallowmobileuserstomanagchedules,contts,andotherpersonalandbusinessinformationaww.palm.comlCacheolm,Inc.bTreoamaertesbsausnieldneaectcsicsonedseosvifocreiess.dwPis.fwtecsndCentrosmartphones,handheldsww.Cases.com,aFandaccessorierGPalm,Inc.,innovatorofeasyltolusemobileproductsincludinalm®Treo_andCentro_smartphones,Palmhandhelds,services0ndaccessoriesoww.palm.comPlFAWrPameTehPoneral.eomTPardayrticeoipate..Ewt0,Taeeg7Pes/uslCacheavluationNation.com/dtreo◮ Figure 9.6 An example of query expansion in the interface of the Yahoo! websearch engine in 2006.
The expanded query suggestions appear just below the “SearchResults” bar.web) suggest related queries in response to a query; the users then opt to useone of these alternative query suggestions. Figure 9.6 shows an example ofquery suggestion options being presented in the Yahoo! web search engine.The central question in this form of query expansion is how to generate alternative or expanded queries for the user. The most common form of queryexpansion is global analysis, using some form of thesaurus.
For each termt in a query, the query can be automatically expanded with synonyms andrelated words of t from the thesaurus. Use of a thesaurus can be combinedwith ideas of term weighting: for instance, one might weight added termsless than original query terms.Methods for building a thesaurus for query expansion include:• Use of a controlled vocabulary that is maintained by human editors.
Here,there is a canonical term for each concept. The subject headings of traditional library subject indexes, such as the Library of Congress SubjectHeadings, or the Dewey Decimal system are examples of a controlledvocabulary. Use of a controlled vocabulary is quite common for wellresourced domains. A well-known example is the Unified Medical Language System (UMLS) used with MedLine for querying the biomedicalresearch literature.
![](https://s.studizba.com/z.php?f=/templates/si/i/noavatar.png&w=100&h=100&t=1"){kind=avatar}
