Building machine learning systems with Python (779436), страница 42

Просмтор этого файла доступен только зарегистрированным пользователям. Но у нас супер быстрая регистрация: достаточно только электронной почты!

Текст из файла (страница 42)

We won't go into details here, but it's sufficientto say that there are extensions of PCA, for example, Kernel PCA, which introduces anon-linear transformation so that we can still use the PCA approach.Another interesting weakness of PCA, which we will cover here, is when it's beingapplied to special classification problems. Let's replace good = (x1 > 5) | (x2 > 5)with good = x1 > x2 to simulate such a special case and we quickly see the problem:Here, the classes are not distributed according to the axis with the highest variance,but the second highest variance. Clearly, PCA falls flat on its face.

As we don'tprovide PCA with any cues regarding the class labels, it cannot do any better.Linear Discriminant Analysis (LDA) comes to the rescue here. It's a methodthat tries to maximize the distance of points belonging to different classes, whileminimizing the distances of points of the same class. We won't give any more detailsregarding how in particular the underlying theory works, just a quick tutorial onhow to use it:>>> from sklearn import lda>>> lda_inst = lda.LDA(n_components=1)>>> Xtrans = lda_inst.fit_transform(X, good)[ 257 ]Dimensionality ReductionThat's all.

Note that in contrast to the previous PCA example, we provide the classlabels to the fit_transform() method. Thus, PCA is an unsupervised featureextraction method, whereas LDA is a supervised one. The result looks as expected:So, why then consider PCA at all and not simply use LDA? Well, it's not that simple.With an increasing number of classes and fewer samples per class, LDA does notlook that well any more.

Also, PCA seems to be not as sensitive to different trainingsets as LDA. So, when we have to advise which method to use, we can only suggest aclear "it depends".Multidimensional scalingAlthough, PCA tries to use optimization for retained variance, multidimensionalscaling (MDS) tries to retain the relative distances as much as possible whenreducing the dimensions.

This is useful when we have a high-dimensionaldataset and want to get a visual impression.[ 258 ]Chapter 11MDS does not care about the data points themselves; instead, it's interested in thedissimilarities between pairs of data points and interprets these as distances. Thefirst thing the MDS algorithm is doing is, therefore, taking all the N datapoints ofdimension k and calculates a distance matrix using a distance function d o , whichmeasures the (most of the time, Euclidean) distance in the original feature space: X 11  X N 1   d o ( X 1 , X 1 )  d o ( X N , X 1 )   → X  d ( X , X )  d ( X , X )XNk oNN  1k o 1 NNow, MDS tries to position the individual datapoints in the lower dimensionalspace such that the new distance there resembles the distances in the original spaceas much as possible.

As MDS is often used for visualization, the choice of the lowerdimension is most of the time two or three.Let's have a look at the following simple data consisting of three datapoints infive-dimensional space. Two of the datapoints are close by and one is very distinctand we want to visualize this in three and two dimensions:>>> X = np.c_[np.ones(5), 2 * np.ones(5), 10 * np.ones(5)].T>>> print(X)[[1.1.1.1.1.][2.2.2.2.2.][ 10.10.10.10.10.]]Using the MDS class in scikit-learn's manifold package, we first specify that we wantto transform X into a three-dimensional Euclidean space:>>> from sklearn import manifold>>> mds = manifold.MDS(n_components=3)>>> Xtrans = mds.fit_transform(X)To visualize it in two dimensions, we would need to set n_components accordingly.[ 259 ]Dimensionality ReductionThe results can be seen in the following two graphs.

The triangle and circle are bothclose together, whereas the star is far away:Let's have a look at the slightly more complex Iris dataset. We will use it later tocontrast LDA with PCA. The Iris dataset contains four attributes per flower. Withthe preceding code, we would project it into three-dimensional space while keepingthe relative distances between the individual flowers as much as possible. In theprevious example, we did not specify any metric, so MDS will default to Euclidean.This means that flowers that were "different" according to their four attributesshould also be far away in the MDS-scaled three-dimensional space and flowers thatwere similar should be near together now, as shown in the following diagram:[ 260 ]Chapter 11Reducing the dimensional reduction to three and two dimensions with PCA instead,we see the expected bigger spread of the flowers belonging to the same class, asshown in the following diagram:Of course, using MDS requires an understanding of the individual feature's units;maybe we are using features that cannot be compared using the Euclidean metric.For instance, a categorical variable, even when encoded as an integer (0= circle,1= star, 2= triangle, and so on), cannot be compared using Euclidean (is circlecloser to star than to triangle?).However, once we are aware of this issue, MDS is a useful tool that reveals similaritiesin our data that otherwise would be difficult to see in the original feature space.Looking a bit deeper into MDS, we realize that it's not a single algorithm, but rathera family of different algorithms, of which we have used just one.

The same was truefor PCA. Also, in case you realize that neither PCA nor MDS solves your problem,just look at the many other manifold learning algorithms that are available in thescikit-learn toolkit.However, before you get overwhelmed by the many different algorithms, it's alwaysbest to start with the simplest one and see how far you get with it. Then, take thenext more complex one and continue from there.[ 261 ]Dimensionality ReductionSummaryYou learned that sometimes you can get rid of complete features using featureselection methods. We also saw that in some cases, this is not enough and we haveto employ feature extraction methods that reveal the real and the lower-dimensionalstructure in our data, hoping that the model has an easier game with it.For sure, we only scratched the surface of the huge body of available dimensionalityreduction methods.

Still, we hope that we got you interested in this whole field, asthere are lots of other methods waiting for you to be picked up. At the end, featureselection and extraction is an art, just like choosing the right learning method ortraining model.The next chapter covers the use of Jug, a little Python framework to managecomputations in a way that takes advantage of multiple cores or multiple machines.You will also learn about AWS, the Amazon Cloud.[ 262 ]Bigger DataIt's not easy to say what big data is.

We will adopt an operational definition: whendata is so large that it becomes cumbersome to work with, we will talk about bigdata. In some areas, this might mean petabytes of data or trillions of transactions:data which will not fit into a single hard drive. In other cases, it may be one hundredtimes smaller, but still difficult to work with.Why has data itself become an issue? While computers keep getting faster and havemore memory, the size of the data has grown as well.

In fact, data has grown fasterthan computational speed and few algorithms scale linearly with the size of theinput data—taken together, this means that data has grown faster than our abilityto process it.We will first build on some of the experience of the previous chapters and work withwhat we can call medium data setting (not quite big data, but not small either).

Forthis, we will use a package called jug, which allows us to perform the following tasks:• Break up your pipeline into tasks• Cache (memoize) intermediate results• Make use of multiple cores, including multiple computers on a gridThe next step is to move to true big data and we will see how to use the cloudfor computation purpose.

In particular, you will learn about the Amazon WebServices infrastructure. In this section, we introduce another Python packagecalled StarCluster to manage clusters.[ 263 ]Bigger DataLearning about big dataThe expression "big data" does not mean a specific amount of data, neither in thenumber of examples nor in the number of gigabytes, terabytes, or petabytes occupiedby the data. It means that data has been growing faster than processing power.

Thisimplies the following:• Some of the methods and techniques that worked well in the past nowneed to be redone or replaced as they do not scale well to the new sizeof the input data• Algorithms cannot assume that all the input data can fit in RAM• Managing data becomes a major task in itself• Using computer clusters or multicore machines becomes a necessity and nota luxuryThis chapter will focus on this last piece of the puzzle: how to use multiple cores(either on the same machine or on separate machines) to speed up and organizeyour computations.

This will also be useful in other medium-sized data tasks.Using jug to break up your pipeline into tasksOften, we have a simple pipeline: we preprocess the initial data, compute features,and then call a machine learning algorithm with the resulting features.Jug is a package developed by Luis Pedro Coelho, one of the authors of this book. It'sopen source (using the liberal MIT License) and can be useful in many areas, but wasdesigned specifically around data analysis problems. It simultaneously solves severalproblems, for example:• It can memoize results to disk (or a database), which means that if you ask it tocompute something you have already computed before, the result is insteadread from disk.• It can use multiple cores or even multiple computers on a cluster.

Характеристики

Список файлов книги