The Elements of Statistical Learning. Data Mining_ Inference_ and Prediction (811377), страница 82
Текст из файла (страница 82)
Figure 10.16 shows the two-variable partialdependence of housing value on joint values of median age and average occupancy. An interaction between these two variables is apparent. For valuesof average occupancy greater than two, house value is nearly independentof median age, whereas for values less than two there is a strong dependenceon age.Figure 10.17 shows the two-variable partial dependence of the fittedmodel on joint values of longitude and latitude, displayed as a shadedcontour plot.
There is clearly a very strong dependence of median housevalue on the neighborhood location in California. Note that Figure 10.17 isnot a plot of house value versus location ignoring the effects of the otherpredictors (10.49). Like all partial dependence plots, it represents the effectof location after accounting for the effects of the other neighborhood andhouse attributes (10.47). It can be viewed as representing an extra premiumone pays for location. This premium is seen to be relatively large near thePacific coast especially in the Bay Area and Los Angeles–San Diego re-10. Boosting and Additive Trees1.51.00.50.0-1.0-0.5Partial Dependence1.51.00.50.0-0.5Partial Dependence2.03742468102345AveOccup1.00.50.0-1.0-0.5Partial Dependence0.50.0-0.5-1.0Partial Dependence1.01.5MedInc102030405046HouseAge810AveRoomsFIGURE 10.15. Partial dependence of housing value on the nonlocation variables for the California housing data.
The red ticks at the base of the plot aredeciles of the input variables.1.00.50.0504030HouseAge22034105AveOccupFIGURE 10.16. Partial dependence of house value on median age and average occupancy. There appears to be a strong interaction effect between these twovariables.4210.14 Illustrations375401.0Latitude380.5360.034−0.5−1.0−124−122−120−118−116−114LongitudeFIGURE 10.17. Partial dependence of median house value on location in California.
One unit is $100, 000, at 1990 prices, and the values plotted are relativeto the overall median of $180, 000.gions. In the northern, central valley, and southeastern desert regions ofCalifornia, location costs considerably less.10.14.2 New Zealand FishPlant and animal ecologists use regression models to predict species presence, abundance and richness as a function of environmental variables.Although for many years simple linear and parametric models were popular, recent literature shows increasing interest in more sophisticated models such as generalized additive models (Section 9.1, GAM), multivariateadaptive regression splines (Section 9.4, MARS) and boosted regressiontrees (Leathwick et al., 2005; Leathwick et al., 2006). Here we model the37610.
Boosting and Additive Treespresence and abundance of the Black Oreo Dory, a marine fish found in theoceanic waters around New Zealand.3Figure 10.18 shows the locations of 17,000 trawls (deep-water net fishing,with a maximum depth of 2km), and the red points indicate those 2353trawls for which the Black Oreo was present, one of over a hundred speciesregularly recorded.
The catch size in kg for each species was recorded foreach trawl. Along with the species catch, a number of environmental measurements are available for each trawl. These include the average depth ofthe trawl (AvgDepth), and the temperature and salinity of the water. Sincethe latter two are strongly correlated with depth, Leathwick et al. (2006)derived instead TempResid and SalResid, the residuals obtained when thesetwo measures are adjusted for depth (via separate non-parametric regressions).
SSTGrad is a measure of the gradient of the sea surface temperature,and Chla is a broad indicator of ecosytem productivity via satellite-imagemeasurements. SusPartMatter provides a measure of suspended particulatematter, particularly in coastal waters, and is also satellite derived.The goal of this analysis is to estimate the probability of finding BlackOreo in a trawl, as well as the expected catch size, standardized to takeinto account the effects of variation in trawl speed and distance, as wellas the mesh size of the trawl net. The authors used logistic regressionfor estimating the probability.
For the catch size, it might seem naturalto assume a Poisson distribution and model the log of the mean count,but this is often not appropriate because of the excessive number of zeros.Although specialized approaches have been developed, such as the zeroinflated Poisson (Lambert, 1992), they chose a simpler approach. If Y isthe (non-negative) catch size,E(Y |X) = E(Y |Y > 0, X) · Pr(Y > 0|X).(10.54)The second term is estimated by the logistic regression, and the first termcan be estimated using only the 2353 trawls with a positive catch.For the logistic regression the authors used a gradient boosted model(GBM)4 with binomial deviance loss function, depth-10 trees, and a shrinkage factor ν = 0.025.
For the positive-catch regression, they modeledlog(Y ) using a GBM with squared-error loss (also depth-10 trees, butν = 0.01), and un-logged the predictions. In both cases they used 10-foldcross-validation for selecting the number of terms, as well as the shrinkagefactor.3 The models, data, and maps shown here were kindly provided by Dr John Leathwickof the National Institute of Water and Atmospheric Research in New Zealand, and DrJane Elith, School of Botany, University of Melbourne. The collection of the researchtrawl data took place from 1979–2005, and was funded by the New Zealand Ministry ofFisheries.4 Version 1.5-7 of package gbm in R, ver. 2.2.0.10.14 Illustrations377FIGURE 10.18. Map of New Zealand and its surrounding exclusive economiczone, showing the locations of 17,000 trawls (small blue dots) taken between 1979and 2005.
The red points indicate trawls for which the species Black Oreo Dorywere present.10. Boosting and Additive Trees1.03780.6Sensitivity0.20.40.300.280.26Mean Deviance0.320.80.34GBM TestGBM CVGAM Test0.24AUC0.0GAM 0.97GBM 0.9805001000Number of Trees15000.00.20.40.60.81.0SpecificityFIGURE 10.19. The left panel shows the mean deviance as a function of thenumber of trees for the GBM logistic regression model fit to the presence/absencedata. Shown are 10-fold cross-validation on the training data (and 1 × s.e. bars),and test deviance on the test data.
Also shown for comparison is the test devianceusing a GAM model with 8 df for each term. The right panel shows ROC curveson the test data for the chosen GBM model (vertical line in left plot) and theGAM model.Figure 10.19 (left panel) shows the mean binomial deviance for the sequence of GBM models, both for 10-fold CV and test data. There is a modest improvement over the performance of a GAM model, fit using smoothingsplines with 8 degrees-of-freedom (df) per term.
The right panel shows theROC curves (see Section 9.2.5) for both models, which measures predictiveperformance. From this point of view, the performance looks very similar, with GBM perhaps having a slight edge as summarized by the AUC(area under the curve). At the point of equal sensitivity/specificity, GBMachieves 91%, and GAM 90%.Figure 10.20 summarizes the contributions of the variables in the logisticGBM fit. We see that there is a well-defined depth range over which BlackOreo are caught, with much more frequent capture in colder waters.
We donot give details of the quantitative catch model; the important variableswere much the same.All the predictors used in these models are available on a fine geographical grid; in fact they were derived from environmental atlases, satellite images and the like—see Leathwick et al. (2006) for details. This also meansthat predictions can be made on this grid, and imported into GIS mappingsystems. Figure 10.21 shows prediction maps for both presence and catchsize, with both standardized to a common set of trawl conditions; since thepredictors vary in a continuous fashion with geographical location, so dothe predictions.−2−6−725−42460TempResidf(SSTGrad)−1−3−7−7152000AvgDepth−5f(SalResid)−3−5f(SusPartMatter)−710500 1000−1Relative influence0−310−505−4f(AvgDepth)−3−5f(TempResid)TempResidAvgDepthSusPartMatterSalResidSSTGradChlaCase2SlopeTidalCurrPentadeCodendSizeDisOrgMatterDistanceSpeedOrbVel0379−110.14 Illustrations−0.8SusPartMatter−0.40.0SalResid0.40.000.050.100.15SSTGradFIGURE 10.20.
The top-left panel shows the relative influence computed fromthe GBM logistic regression model. The remaining panels show the partial dependence plots for the leading five variables, all plotted on the same scale forcomparison.Because of their ability to model interactions and automatically selectvariables, as well as robustness to outliers and missing data, GBM modelsare rapidly gaining popularity in this data-rich and enthusiastic community.10.14.3 Demographics DataIn this section we illustrate gradient boosting on a multiclass classification problem, using MART.
The data come from 9243 questionnaires filledout by shopping mall customers in the San Francisco Bay Area (ImpactResources, Inc., Columbus, OH). Among the questions are 14 concerningdemographics. For this illustration the goal is to predict occupation using the other 13 variables as predictors, and hence identify demographicvariables that discriminate between different occupational categories. Werandomly divided the data into a training set (80%) and test set (20%),and used J = 6 node trees with a learning rate ν = 0.1.Figure 10.22 shows the K = 9 occupation class values along with theircorresponding error rates.
![](https://s.studizba.com/z.php?f=/templates/si/i/noavatar.png&w=100&h=100&t=1"){kind=avatar}
