The Elements of Statistical Learning. Data Mining_ Inference_ and Prediction (The Elements of Statistical Learning. Data Mining_ Inference_ and Prediction.pdf), страница 5

Introductionooooooooooooooooooooo6.07.0oooo oooooooooooooooooooooooooooooooo ooooo oooooooooooooooooooooooooooooo oooooooooooooooooo oooooooo ooooooo ooooooooo oooooooo oo oo oo oooooooo ooooo o ooo o o oooooooooooooo o o oooooo ooooooooo oooooooooooooooooooo ooooo oo oooo oo ooooooo oo o ooooooo oo oooooo o o o ooo oo ooooooooo ooooooo oooooooooooooo0 1 2 3 4 5oo o ooo o o oooo o oooooooooooooooooooooooooooooo ooo ooooooooooooooooo oo oooo oo oooooooooo oo o oo ooooooooo ooo o o ooo ooo oo ooo oo oo oooooooooooooooooooooooo ooo oo o2.53.5o oo o o ooooooooooooooooooooooo ooo o oo ooooooooooooo ooo oo ooo ooo o ooooo oo ooo oooooo oo oooo oooooooooo oo ooo o oo oooo ooooooo ooooooooooooo4.5oooo o oooo o ooooooooooooooooooo o ooooooooooooooo o ooo oooooooooo ooooooo oo ooooooo ooo oooooooooo o o ooooooooooo012ooooooooooooooolcpoooooo oooo o−1o oooooooo o oooooo ooooooooooooooooooooooooooooooooooooooo o ooo o ooo ooooo oo oooooo oooooooooo o o ooooooooooo o ooooogleasonooooooooo o o ooooooooooo o o oo oo oo ooooooo o oo oooooooo o ooo oo ooo ooooo oo o o oooo0ooooooooo o o o o o oooo−1oo o oooo ooooooooooo oooooooooooo oo oooooooooooooooooooooooo13ooooo oooooooooo ooooooo oo ooooooo o oooooo oo oooooo oooooooooooooo o2ooo o oo ooooooo oooooooo ooooooooo oooo ooo ooo oooooo ooooooooooooo oooo ooo o ooooooo1ooooooo ooooooooo ooo ooo oooo o o oo ooooo oo o ooooooo ooooooooooooooooooooooooo oo ooooo0oooooooo ooo oooooooooo oooo ooooooooooo oo oooo oooooooooo oooooooo oooooooo o oo−1o oo o o ooooo ooooo ooo ooooo o oooo oooooo ooo ooooooooooooooooo oooooooooooooooooooooooooooooooo ooooooooooooooooooooo23oooooooooooooo100o ooooooo60oo oooo oooooooooooooooooooooooo oopgg450 20o oooooooooooooooooooooooooooooooooooooooooooooo o9.0ooooooo ooooooooooooooooooooooooooooo oo8.00.00.4sviooo oooooooooooooooooooooooooooooo0 2060100FIGURE 1.1.

Scatterplot matrix of the prostate cancer data. The first row showsthe response against each of the predictors in turn. Two of the predictors, svi andgleason, are categorical.For this problem not all errors are equal; we want to avoid filtering outgood email, while letting spam get through is not desirable but less seriousin its consequences. We discuss a number of different methods for tacklingthis learning problem in the book.Example 2: Prostate CancerThe data for this example, displayed in Figure 1.11 , come from a studyby Stamey et al. (1989) that examined the correlation between the level of1 There was an error in these data in the first edition of this book. Subject 32 hada value of 6.1 for lweight, which translates to a 449 gm prostate! The correct value is44.9 gm. We are grateful to Prof.

Stephen W. Link for alerting us to this error.41. IntroductionFIGURE 1.2. Examples of handwritten digits from U.S. postal envelopes.prostate specific antigen (PSA) and a number of clinical measures, in 97men who were about to receive a radical prostatectomy.The goal is to predict the log of PSA (lpsa) from a number of measurements including log cancer volume (lcavol), log prostate weight lweight,age, log of benign prostatic hyperplasia amount lbph, seminal vesicle invasion svi, log of capsular penetration lcp, Gleason score gleason, andpercent of Gleason scores 4 or 5 pgg45. Figure 1.1 is a scatterplot matrixof the variables.

Some correlations with lpsa are evident, but a good predictive model is difficult to construct by eye.This is a supervised learning problem, known as a regression problem,because the outcome measurement is quantitative.Example 3: Handwritten Digit RecognitionThe data from this example come from the handwritten ZIP codes onenvelopes from U.S. postal mail. Each image is a segment from a five digitZIP code, isolating a single digit. The images are 16×16 eight-bit grayscalemaps, with each pixel ranging in intensity from 0 to 255. Some sampleimages are shown in Figure 1.2.The images have been normalized to have approximately the same sizeand orientation.

The task is to predict, from the 16 × 16 matrix of pixelintensities, the identity of each image (0, 1, . . . , 9) quickly and accurately. Ifit is accurate enough, the resulting algorithm would be used as part of anautomatic sorting procedure for envelopes. This is a classification problemfor which the error rate needs to be kept very low to avoid misdirection of1. Introduction5mail.

In order to achieve this low error rate, some objects can be assignedto a “don’t know” category, and sorted instead by hand.Example 4: DNA Expression MicroarraysDNA stands for deoxyribonucleic acid, and is the basic material that makesup human chromosomes. DNA microarrays measure the expression of agene in a cell by measuring the amount of mRNA (messenger ribonucleicacid) present for that gene. Microarrays are considered a breakthroughtechnology in biology, facilitating the quantitative study of thousands ofgenes simultaneously from a single sample of cells.Here is how a DNA microarray works. The nucleotide sequences for a fewthousand genes are printed on a glass slide. A target sample and a referencesample are labeled with red and green dyes, and each are hybridized withthe DNA on the slide.

Through fluoroscopy, the log (red/green) intensitiesof RNA hybridizing at each site is measured. The result is a few thousandnumbers, typically ranging from say −6 to 6, measuring the expression levelof each gene in the target relative to the reference sample. Positive valuesindicate higher expression in the target versus the reference, and vice versafor negative values.A gene expression dataset collects together the expression values from aseries of DNA microarray experiments, with each column representing anexperiment. There are therefore several thousand rows representing individual genes, and tens of columns representing samples: in the particular example of Figure 1.3 there are 6830 genes (rows) and 64 samples (columns),although for clarity only a random sample of 100 rows are shown.

The figure displays the data set as a heat map, ranging from green (negative) tored (positive). The samples are 64 cancer tumors from different patients.The challenge here is to understand how the genes and samples are organized. Typical questions include the following:(a) which samples are most similar to each other, in terms of their expression profiles across genes?(b) which genes are most similar to each other, in terms of their expressionprofiles across samples?(c) do certain genes show very high (or low) expression for certain cancersamples?We could view this task as a regression problem, with two categoricalpredictor variables—genes and samples—with the response variable beingthe level of expression.

However, it is probably more useful to view it asunsupervised learning problem. For example, for question (a) above, wethink of the samples as points in 6830–dimensional space, which we wantto cluster together in some way.61. IntroductionBREASTRENALMELANOMAMELANOMAMCF7D-reproCOLONCOLONK562B-reproCOLONNSCLCLEUKEMIARENALMELANOMABREASTCNSCNSRENALMCF7A-reproNSCLCK562A-reproCOLONCNSNSCLCNSCLCLEUKEMIACNSOVARIANBREASTLEUKEMIAMELANOMAMELANOMAOVARIANOVARIANNSCLCRENALBREASTMELANOMAOVARIANOVARIANNSCLCRENALBREASTMELANOMALEUKEMIACOLONBREASTLEUKEMIACOLONCNSMELANOMANSCLCPROSTATENSCLCRENALRENALNSCLCRENALLEUKEMIAOVARIANPROSTATECOLONBREASTRENALUNKNOWNSIDW299104SIDW380102SID73161GNALH.sapiensmRNASID325394RASGTPASESID207172ESTsSIDW377402HumanmRNASIDW469884ESTsSID471915MYBPROTOESTsChr.1SID377451DNAPOLYMERSID375812SIDW31489SID167117SIDW470459SIDW487261HomosapiensSIDW376586ChrMITOCHONDRIAL60SID47116ESTsChr.6SIDW296310SID488017SID305167ESTsChr.3SID127504SID289414PTPRCSIDW298203SIDW310141SIDW376928ESTsCh31SID114241SID377419SID297117SIDW201620SIDW279664SIDW510534HLACLASSISIDW203464SID239012SIDW205716SIDW376776HYPOTHETICALWASWiskottSIDW321854ESTsChr.15SIDW376394SID280066ESTsChr.5SIDW488221SID46536SIDW257915ESTsChr.2SIDW322806SID200394ESTsChr.15SID284853SID485148SID297905ESTsSIDW486740SMALLNUCESTsSIDW366311SIDW357197SID52979ESTsSID43609SIDW416621ERLUMENTUPLE1TUP1SIDW428642SID381079SIDW298052SIDW417270SIDW362471ESTsChr.15SIDW321925SID380265SIDW308182SID381508SID377133SIDW365099ESTsChr.10SIDW325120SID360097SID375990SIDW128368SID301902SID31984SID42354FIGURE 1.3.

DNA microarray data: expression matrix of 6830 genes (rows)and 64 samples (columns), for the human tumor data. Only a random sampleof 100 rows are shown. The display is a heat map, ranging from bright green(negative, under expressed) to bright red (positive, over expressed). Missing valuesare gray.

The rows and columns are displayed in a randomly chosen order.1. Introduction7Who Should Read this BookThis book is designed for researchers and students in a broad variety offields: statistics, artificial intelligence, engineering, finance and others. Weexpect that the reader will have had at least one elementary course instatistics, covering basic topics including linear regression.We have not attempted to write a comprehensive catalog of learningmethods, but rather to describe some of the most important techniques.Equally notable, we describe the underlying concepts and considerationsby which a researcher can judge a learning method. We have tried to writethis book in an intuitive fashion, emphasizing concepts rather than mathematical details.As statisticians, our exposition will naturally reflect our backgrounds andareas of expertise.

However in the past eight years we have been attendingconferences in neural networks, data mining and machine learning, and ourthinking has been heavily influenced by these exciting fields. This influenceis evident in our current research, and in this book.How This Book is OrganizedOur view is that one must understand simple methods before trying tograsp more complex ones. Hence, after giving an overview of the supervising learning problem in Chapter 2, we discuss linear methods for regressionand classification in Chapters 3 and 4. In Chapter 5 we describe splines,wavelets and regularization/penalization methods for a single predictor,while Chapter 6 covers kernel methods and local regression. Both of thesesets of methods are important building blocks for high-dimensional learning techniques.