An introduction to information retrieval. Manning_ Raghavan (2009) (811397), страница 4
Текст из файла (страница 4)
Thesetechniques are critical for achieving subsecond response times to user queriesin large search engines. The indexes and queries considered in Chapters 1–5only deal with Boolean retrieval, in which a document either matches a query,or does not. A desire to measure the extent to which a document matches aquery, or the score of a document for a query, motivates the development ofterm weighting and the computation of scores in Chapters 6 and 7, leadingto the idea of a list of documents that are rank-ordered for a query.
Chapter 8focuses on the evaluation of an information retrieval system based on theOnline edition (c) 2009 Cambridge UPPrefacexxxiiirelevance of the documents it retrieves, allowing us to compare the relativeperformances of different systems on benchmark document collections andqueries.Chapters 9–21 build on the foundation of the first eight chapters to covera variety of more advanced topics.
Chapter 9 discusses methods by whichretrieval can be enhanced through the use of techniques like relevance feedback and query expansion, which aim at increasing the likelihood of retrieving relevant documents. Chapter 10 considers information retrieval fromdocuments that are structured with markup languages like XML and HTML.We treat structured retrieval by reducing it to the vector space scoring methods developed in Chapter 6. Chapters 11 and 12 invoke probability theory tocompute scores for documents on queries. Chapter 11 develops traditionalprobabilistic information retrieval, which provides a framework for computing the probability of relevance of a document, given a set of query terms.This probability may then be used as a score in ranking.
Chapter 12 illustrates an alternative, wherein for each document in a collection, we build alanguage model from which one can estimate a probability that the languagemodel generates a given query. This probability is another quantity withwhich we can rank-order documents.Chapters 13–17 give a treatment of various forms of machine learning andnumerical methods in information retrieval. Chapters 13–15 treat the problem of classifying documents into a set of known categories, given a set ofdocuments along with the classes they belong to. Chapter 13 motivates statistical classification as one of the key technologies needed for a successfulsearch engine, introduces Naive Bayes, a conceptually simple and efficienttext classification method, and outlines the standard methodology for evaluating text classifiers.
Chapter 14 employs the vector space model from Chapter 6 and introduces two classification methods, Rocchio and kNN, that operate on document vectors. It also presents the bias-variance tradeoff as animportant characterization of learning problems that provides criteria for selecting an appropriate method for a text classification problem.
Chapter 15introduces support vector machines, which many researchers currently viewas the most effective text classification method. We also develop connectionsin this chapter between the problem of classification and seemingly disparatetopics such as the induction of scoring functions from a set of training examples.Chapters 16–18 consider the problem of inducing clusters of related documents from a collection. In Chapter 16, we first give an overview of anumber of important applications of clustering in information retrieval. Wethen describe two flat clustering algorithms: the K-means algorithm, an efficient and widely used document clustering method; and the ExpectationMaximization algorithm, which is computationally more expensive, but alsomore flexible.
Chapter 17 motivates the need for hierarchically structuredOnline edition (c) 2009 Cambridge UPxxxivPrefaceclusterings (instead of flat clusterings) in many applications in informationretrieval and introduces a number of clustering algorithms that produce ahierarchy of clusters. The chapter also addresses the difficult problem ofautomatically computing labels for clusters. Chapter 18 develops methodsfrom linear algebra that constitute an extension of clustering, and also offerintriguing prospects for algebraic methods in information retrieval, whichhave been pursued in the approach of latent semantic indexing.Chapters 19–21 treat the problem of web search.
We give in Chapter 19 asummary of the basic challenges in web search, together with a set of techniques that are pervasive in web information retrieval. Next, Chapter 20describes the architecture and requirements of a basic web crawler. Finally,Chapter 21 considers the power of link analysis in web search, using in theprocess several methods from linear algebra and advanced probability theory.This book is not comprehensive in covering all topics related to information retrieval.
We have put aside a number of topics, which we deemedoutside the scope of what we wished to cover in an introduction to information retrieval class. Nevertheless, for people interested in these topics, weprovide a few pointers to mainly textbook coverage here.Cross-language IR (Grossman and Frieder 2004, ch. 4) and (Oard and Dorr1996).Image and Multimedia IR (Grossman and Frieder 2004, ch. 4), (Baeza-Yatesand Ribeiro-Neto 1999, ch.
6), (Baeza-Yates and Ribeiro-Neto 1999, ch. 11),(Baeza-Yates and Ribeiro-Neto 1999, ch. 12), (del Bimbo 1999), (Lew 2001),and (Smeulders et al. 2000).Speech retrieval (Coden et al. 2002).Music Retrieval (Downie 2006) and http://www.ismir.net/.User interfaces for IR (Baeza-Yates and Ribeiro-Neto 1999, ch. 10).Parallel and Peer-to-Peer IR (Grossman and Frieder 2004, ch. 7), (Baeza-Yatesand Ribeiro-Neto 1999, ch. 9), and (Aberer 2001).Digital libraries (Baeza-Yates and Ribeiro-Neto 1999, ch. 15) and (Lesk 2004).Information science perspective (Korfhage 1997), (Meadow et al. 1999), and(Ingwersen and Järvelin 2005).Logic-based approaches to IR (van Rijsbergen 1989).Natural Language Processing techniques (Manning and Schütze 1999), (Jurafsky and Martin 2008), and (Lewis and Jones 1996).Online edition (c) 2009 Cambridge UPPrefacexxxvPrerequisitesIntroductory courses in data structures and algorithms, in linear algebra andin probability theory suffice as prerequisites for all 21 chapters.
We now givemore detail for the benefit of readers and instructors who wish to tailor theirreading to some of the chapters.Chapters 1–5 assume as prerequisite a basic course in algorithms and datastructures. Chapters 6 and 7 require, in addition, a knowledge of basic linear algebra including vectors and dot products. No additional prerequisitesare assumed until Chapter 11, where a basic course in probability theory isrequired; Section 11.1 gives a quick review of the concepts necessary in Chapters 11–13.
Chapter 15 assumes that the reader is familiar with the notion ofnonlinear optimization, although the chapter may be read without detailedknowledge of algorithms for nonlinear optimization. Chapter 18 demands afirst course in linear algebra including familiarity with the notions of matrixrank and eigenvectors; a brief review is given in Section 18.1. The knowledgeof eigenvalues and eigenvectors is also necessary in Chapter 21.Book layout✎✄?Worked examples in the text appear with a pencil sign next to them in the leftmargin. Advanced or difficult material appears in sections or subsectionsindicated with scissors in the margin.
Exercises are marked in the marginwith a question mark. The level of difficulty of exercises is indicated as easy(⋆), medium (⋆⋆), or difficult (⋆ ⋆ ⋆).AcknowledgmentsWe would like to thank Cambridge University Press for allowing us to makethe draft book available online, which facilitated much of the feedback wehave received while writing the book. We also thank Lauren Cowles, whohas been an outstanding editor, providing several rounds of comments oneach chapter, on matters of style, organization, and coverage, as well as detailed comments on the subject matter of the book.
To the extent that wehave achieved our goals in writing this book, she deserves an important partof the credit.We are very grateful to the many people who have given us comments,suggestions, and corrections based on draft versions of this book. We thankfor providing various corrections and comments: Cheryl Aasheim, Josh Attenberg, Daniel Beck, Luc Bélanger, Georg Buscher, Tom Breuel, Daniel Burckhardt, Fazli Can, Dinquan Chen, Stephen Clark, Ernest Davis, Pedro Domingos, Rodrigo Panchiniak Fernandes, Paolo Ferragina, Alex Fraser, NorbertOnline edition (c) 2009 Cambridge UPxxxviPrefaceFuhr, Vignesh Ganapathy, Elmer Garduno, Xiubo Geng, David Gondek, Sergio Govoni, Corinna Habets, Ben Handy, Donna Harman, Benjamin Haskell,Thomas Hühn, Deepak Jain, Ralf Jankowitsch, Dinakar Jayarajan, Vinay Kakade,Mei Kobayashi, Wessel Kraaij, Rick Lafleur, Florian Laws, Hang Li, DavidLosada, David Mann, Ennio Masi, Sven Meyer zu Eissen, Alexander Murzaku,Gonzalo Navarro, Frank McCown, Paul McNamee, Christoph Müller, ScottOlsson, Tao Qin, Megha Raghavan, Michal Rosen-Zvi, Klaus Rothenhäusler,Kenyu L.
![](https://s.studizba.com/z.php?f=/templates/si/i/noavatar.png&w=100&h=100&t=1"){kind=avatar}
