An introduction to information retrieval. Manning_ Raghavan (2009) (An introduction to information retrieval. Manning_ Raghavan (2009).pdf), страница 5

Runner, Alexander Salamanca, Grigory Sapunov, Evgeny Shadchnev, Tobias Scheffer, Nico Schlaefer, Ian Soboroff, Benno Stein, MarcinSydow, Andrew Turner, Jason Utt, Huey Vo, Travis Wade, Mike Walsh, ChangliangWang, Renjing Wang, and Thomas Zeume.Many people gave us detailed feedback on individual chapters, either atour request or through their own initiative. For this, we’re particularly grateful to: James Allan, Omar Alonso, Ismail Sengor Altingovde, Vo Ngoc Anh,Roi Blanco, Eric Breck, Eric Brown, Mark Carman, Carlos Castillo, JunghooCho, Aron Culotta, Doug Cutting, Meghana Deodhar, Susan Dumais, Johannes Fürnkranz, Andreas Heß, Djoerd Hiemstra, David Hull, ThorstenJoachims, Siddharth Jonathan J. B., Jaap Kamps, Mounia Lalmas, Amy Langville,Nicholas Lester, Dave Lewis, Daniel Lowd, Yosi Mass, Jeff Michels, Alessandro Moschitti, Amir Najmi, Marc Najork, Giorgio Maria Di Nunzio, PaulOgilvie, Priyank Patel, Jan Pedersen, Kathryn Pedings, Vassilis Plachouras,Daniel Ramage, Ghulam Raza, Stefan Riezler, Michael Schiehlen, HelmutSchmid, Falk Nicolas Scholer, Sabine Schulte im Walde, Fabrizio Sebastiani,Sarabjeet Singh, Valentin Spitkovsky, Alexander Strehl, John Tait, Shivakumar Vaithyanathan, Ellen Voorhees, Gerhard Weikum, Dawid Weiss, YimingYang, Yisong Yue, Jian Zhang, and Justin Zobel.And finally there were a few reviewers who absolutely stood out in termsof the quality and quantity of comments that they provided.

We thank themfor their significant impact on the content and structure of the book. Weexpress our gratitude to Pavel Berkhin, Stefan Büttcher, Jamie Callan, ByronDom, Torsten Suel, and Andrew Trotman.Parts of the initial drafts of Chapters 13–15 were based on slides that weregenerously provided by Ray Mooney. While the material has gone throughextensive revisions, we gratefully acknowledge Ray’s contribution to thethree chapters in general and to the description of the time complexities oftext classification algorithms in particular.The above is unfortunately an incomplete list: we are still in the process ofincorporating feedback we have received.

And, like all opinionated authors,we did not always heed the advice that was so freely given. The publishedversions of the chapters remain solely the responsibility of the authors.The authors thank Stanford University and the University of Stuttgart forproviding a stimulating academic environment for discussing ideas and theopportunity to teach courses from which this book arose and in which itsOnline edition (c) 2009 Cambridge UPPrefacexxxviicontents were refined. CM thanks his family for the many hours they’ve lethim spend working on this book, and hopes he’ll have a bit more free time onweekends next year.

PR thanks his family for their patient support throughthe writing of this book and is also grateful to Yahoo! Inc. for providing afertile environment in which to work on this book. HS would like to thankhis parents, family, and friends for their support while writing this book.Web and contact informationThis book has a companion website at http://informationretrieval.org. As well aslinks to some more general resources, it is our intent to maintain on this website a set of slides for each chapter which may be used for the correspondinglecture.

We gladly welcome further feedback, corrections, and suggestionson the book, which may be sent to all the authors at informationretrieval (at) yahoogroups (dot) com.Online edition (c) 2009 Cambridge UPDRAFT! © April 1, 2009 Cambridge University Press. Feedback welcome.1INFORMATIONRETRIEVAL1Boolean retrievalThe meaning of the term information retrieval can be very broad. Just gettinga credit card out of your wallet so that you can type in the card numberis a form of information retrieval. However, as an academic field of study,information retrieval might be defined thus:Information retrieval (IR) is finding material (usually documents) ofan unstructured nature (usually text) that satisfies an information needfrom within large collections (usually stored on computers).As defined in this way, information retrieval used to be an activity that onlya few people engaged in: reference librarians, paralegals, and similar professional searchers.

Now the world has changed, and hundreds of millionsof people engage in information retrieval every day when they use a websearch engine or search their email.1 Information retrieval is fast becomingthe dominant form of information access, overtaking traditional databasestyle searching (the sort that is going on when a clerk says to you: “I’m sorry,I can only look up your order if you can give me your Order ID”).IR can also cover other kinds of data and information problems beyondthat specified in the core definition above. The term “unstructured data”refers to data which does not have clear, semantically overt, easy-for-a-computerstructure.

It is the opposite of structured data, the canonical example ofwhich is a relational database, of the sort companies usually use to maintain product inventories and personnel records. In reality, almost no dataare truly “unstructured”. This is definitely true of all text data if you countthe latent linguistic structure of human languages.

But even accepting thatthe intended notion of structure is overt structure, most text has structure,such as headings and paragraphs and footnotes, which is commonly represented in documents by explicit markup (such as the coding underlying web1. In modern parlance, the word “search” has tended to replace “(information) retrieval”; theterm “search” is quite ambiguous, but in context we use the two synonymously.Online edition (c) 2009 Cambridge UP21 Boolean retrievalpages). IR is also used to facilitate “semistructured” search such as finding adocument where the title contains Java and the body contains threading.The field of information retrieval also covers supporting users in browsingor filtering document collections or further processing a set of retrieved documents.

Given a set of documents, clustering is the task of coming up with agood grouping of the documents based on their contents. It is similar to arranging books on a bookshelf according to their topic. Given a set of topics,standing information needs, or other categories (such as suitability of textsfor different age groups), classification is the task of deciding which class(es),if any, each of a set of documents belongs to. It is often approached by firstmanually classifying some documents and then hoping to be able to classifynew documents automatically.Information retrieval systems can also be distinguished by the scale atwhich they operate, and it is useful to distinguish three prominent scales.In web search, the system has to provide search over billions of documentsstored on millions of computers.

Distinctive issues are needing to gatherdocuments for indexing, being able to build systems that work efficientlyat this enormous scale, and handling particular aspects of the web, such asthe exploitation of hypertext and not being fooled by site providers manipulating page content in an attempt to boost their search engine rankings,given the commercial importance of the web. We focus on all these issuesin Chapters 19–21. At the other extreme is personal information retrieval. Inthe last few years, consumer operating systems have integrated informationretrieval (such as Apple’s Mac OS X Spotlight or Windows Vista’s InstantSearch).

Email programs usually not only provide search but also text classification: they at least provide a spam (junk mail) filter, and commonly alsoprovide either manual or automatic means for classifying mail so that it canbe placed directly into particular folders. Distinctive issues here include handling the broad range of document types on a typical personal computer,and making the search system maintenance free and sufficiently lightweightin terms of startup, processing, and disk space usage that it can run on onemachine without annoying its owner.

In between is the space of enterprise,institutional, and domain-specific search, where retrieval might be provided forcollections such as a corporation’s internal documents, a database of patents,or research articles on biochemistry. In this case, the documents will typically be stored on centralized file systems and one or a handful of dedicatedmachines will provide search over the collection. This book contains techniques of value over this whole spectrum, but our coverage of some aspectsof parallel and distributed search in web-scale search systems is comparatively light owing to the relatively small published literature on the detailsof such systems. However, outside of a handful of web search companies, asoftware developer is most likely to encounter the personal search and enterprise scenarios.Online edition (c) 2009 Cambridge UP1.1 An example information retrieval problem3In this chapter we begin with a very simple example of an informationretrieval problem, and introduce the idea of a term-document matrix (Section 1.1) and the central inverted index data structure (Section 1.2).