An introduction to information retrieval. Manning_ Raghavan (2009) (811397), страница 101
Текст из файла (страница 101)
Sellers of suchproperty and their agents, therefore, have a strong incentive to create webpages that rank highly on this query. In a search engine whose scoring wasbased on term frequencies, a web page with numerous repetitions of maui golfreal estate would rank highly. This led to the first generation of spam, which(in the context of web search) is the manipulation of web page content forthe purpose of appearing high up in search results for selected keywords.To avoid irritating users with these repetitions, sophisticated spammers resorted to such tricks as rendering these repeated terms in the same color asthe background.
Despite these words being consequently invisible to the human user, a search engine indexer would parse the invisible words out ofOnline edition (c) 2009 Cambridge UP42819 Web search basics◮ Figure 19.5 Cloaking as used by spammers.PAID INCLUSIONthe HTML representation of the web page and index these words as beingpresent in the page.At its root, spam stems from the heterogeneity of motives in content creation on the Web. In particular, many web content creators have commercialmotives and therefore stand to gain from manipulating search engine results.You might argue that this is no different from a company that uses large fontsto list its phone numbers in the yellow pages; but this generally costs thecompany more and is thus a fairer mechanism.
A more apt analogy, perhaps,is the use of company names beginning with a long string of A’s to be listedearly in a yellow pages category. In fact, the yellow pages’ model of companies paying for larger/darker fonts has been replicated in web search: inmany search engines, it is possible to pay to have one’s web page includedin the search engine’s index – a model known as paid inclusion. Differentsearch engines have different policies on whether to allow paid inclusion,and whether such a payment has any effect on ranking in search results.Search engines soon became sophisticated enough in their spam detectionto screen out a large number of repetitions of particular keywords.
Spammers responded with a richer set of spam techniques, the best known ofwhich we now describe. The first of these techniques is cloaking, shown inFigure 19.5. Here, the spammer’s web server returns different pages depending on whether the http request comes from a web search engine’s crawler(the part of the search engine that gathers web pages, to be described inChapter 20), or from a human user’s browser. The former causes the webpage to be indexed by the search engine under misleading keywords. Whenthe user searches for these keywords and elects to view the page, he receivesa web page that has altogether different content than that indexed by thesearch engine. Such deception of search indexers is unknown in the traditional world of information retrieval; it stems from the fact that the relationship between page publishers and web search engines is not completelycollaborative.A doorway page contains text and metadata carefully chosen to rank highlyOnline edition (c) 2009 Cambridge UP19.3 Advertising as the economic modelS EARCH E NGINEO PTIMIZERSADVERSARIALINFORMATIONRETRIEVALLINK SPAM?429on selected search keywords.
When a browser requests the doorway page, itis redirected to a page containing content of a more commercial nature. Morecomplex spamming techniques involve manipulation of the metadata relatedto a page including (for reasons we will see in Chapter 21) the links into aweb page. Given that spamming is inherently an economically motivatedactivity, there has sprung around it an industry of Search Engine Optimizers,or SEOs to provide consultancy services for clients who seek to have theirweb pages rank highly on selected keywords. Web search engines frown onthis business of attempting to decipher and adapt to their proprietary ranking techniques and indeed announce policies on forms of SEO behavior theydo not tolerate (and have been known to shut down search requests from certain SEOs for violation of these).
Inevitably, the parrying between such SEOs(who gradually infer features of each web search engine’s ranking methods)and the web search engines (who adapt in response) is an unending struggle;indeed, the research sub-area of adversarial information retrieval has sprung uparound this battle. To combat spammers who manipulate the text of theirweb pages is the exploitation of the link structure of the Web – a techniqueknown as link analysis.
The first web search engine known to apply link analysis on a large scale (to be detailed in Chapter 21) was Google, although allweb search engines currently make use of it (and correspondingly, spammers now invest considerable effort in subverting it – this is known as linkspam).Exercise 19.1If the number of pages with in-degree i is proportional to 1/i2.1 , what is the probability that a randomly chosen web page has in-degree 1?Exercise 19.2If the number of pages with in-degree i is proportional to 1/i2.1 , what is the averagein-degree of a web page?Exercise 19.3If the number of pages with in-degree i is proportional to 1/i2.1 , then as the largestin-degree goes to infinity, does the fraction of pages with in-degree i grow, stay thesame, or diminish? How would your answer change for values of the exponent otherthan 2.1?Exercise 19.4The average in-degree of all nodes in a snapshot of the web graph is 9.
What can wesay about the average out-degree of all nodes in this snapshot?19.3Advertising as the economic modelEarly in the history of the Web, companies used graphical banner advertisements on web pages at popular websites (news and entertainment sites suchas MSN, America Online, Yahoo! and CNN). The primary purpose of theseadvertisements was branding: to convey to the viewer a positive feeling aboutOnline edition (c) 2009 Cambridge UP43019 Web search basicsCPMCPCSPONSORED SEARCHSEARCH ADVERTISINGALGORITHMIC SEARCHthe brand of the company placing the advertisement.
Typically these advertisements are priced on a cost per mil (CPM) basis: the cost to the company ofhaving its banner advertisement displayed 1000 times. Some websites struckcontracts with their advertisers in which an advertisement was priced not bythe number of times it is displayed (also known as impressions), but ratherby the number of times it was clicked on by the user. This pricing model isknown as the cost per click (CPC) model. In such cases, clicking on the advertisement leads the user to a web page set up by the advertiser, where the useris induced to make a purchase. Here the goal of the advertisement is not somuch brand promotion as to induce a transaction.
This distinction betweenbrand and transaction-oriented advertising was already widely recognizedin the context of conventional media such as broadcast and print. The interactivity of the web allowed the CPC billing model – clicks could be meteredand monitored by the website and billed to the advertiser.The pioneer in this direction was a company named Goto, which changedits name to Overture prior to eventual acquisition by Yahoo! Goto was not,in the traditional sense, a search engine; rather, for every query term q it accepted bids from companies who wanted their web page shown on the queryq.
In response to the query q, Goto would return the pages of all advertiserswho bid for q, ordered by their bids. Furthermore, when the user clickedon one of the returned results, the corresponding advertiser would make apayment to Goto (in the initial implementation, this payment equaled theadvertiser’s bid for q).Several aspects of Goto’s model are worth highlighting.
First, a user typingthe query q into Goto’s search interface was actively expressing an interestand intent related to the query q. For instance, a user typing golf clubs is morelikely to be imminently purchasing a set than one who is simply browsingnews on golf. Second, Goto only got compensated when a user actually expressed interest in an advertisement – as evinced by the user clicking the advertisement.
Taken together, these created a powerful mechanism by whichto connect advertisers to consumers, quickly raising the annual revenues ofGoto/Overture into hundreds of millions of dollars. This style of search engine came to be known variously as sponsored search or search advertising.Given these two kinds of search engines – the “pure” search engines suchas Google and Altavista, versus the sponsored search engines – the logical next step was to combine them into a single user experience. Currentsearch engines follow precisely this model: they provide pure search results(generally known as algorithmic search results) as the primary response to auser’s search, together with sponsored search results displayed separatelyand distinctively to the right of the algorithmic results.
This is shown in Figure 19.6. Retrieving sponsored search results and ranking them in responseto a query has now become considerably more sophisticated than the simple Goto scheme; the process entails a blending of ideas from informationOnline edition (c) 2009 Cambridge UP19.3 Advertising as the economic model431◮ Figure 19.6 Search advertising triggered by query keywords.
![](https://s.studizba.com/z.php?f=/templates/si/i/noavatar.png&w=100&h=100&t=1"){kind=avatar}
