Building machine learning systems with Python (779436), страница 4
Текст из файла (страница 4)
This is the bestway to get to know what works and what doesn't.The only blog we want to highlight right here (more in the Appendix) is http://blog.kaggle.com, the blog of the Kaggle company, which is carrying out machinelearning competitions. Typically, they encourage the winners of the competitions towrite down how they approached the competition, what strategies did not work, andhow they arrived at the winning strategy.
Even if you don't read anything else, this isa must.Getting startedAssuming that you have Python already installed (everything at least as recent as 2.7should be fine), we need to install NumPy and SciPy for numerical operations, as wellas matplotlib for visualization.[5]Getting Started with Python Machine LearningIntroduction to NumPy, SciPy, and matplotlibBefore we can talk about concrete machine learning algorithms, we have to talkabout how best to store the data we will chew through. This is important as the mostadvanced learning algorithm will not be of any help to us if it will never finish.
Thismay be simply because accessing the data is too slow. Or maybe its representationforces the operating system to swap all day. Add to this that Python is an interpretedlanguage (a highly optimized one, though) that is slow for many numericallyheavy algorithms compared to C or FORTRAN. So we might ask why on earth somany scientists and companies are betting their fortune on Python even in highlycomputation-intensive areas?The answer is that, in Python, it is very easy to off-load number crunching tasks tothe lower layer in the form of C or FORTRAN extensions.
And that is exactly whatNumPy and SciPy do (http://scipy.org/Download). In this tandem, NumPyprovides the support of highly optimized multidimensional arrays, which are thebasic data structure of most state-of-the-art algorithms. SciPy uses those arrays toprovide a set of fast numerical recipes. Finally, matplotlib (http://matplotlib.org/) is probably the most convenient and feature-rich library to plot high-qualitygraphs using Python.Installing PythonLuckily, for all major operating systems, that is, Windows, Mac, and Linux, thereare targeted installers for NumPy, SciPy, and matplotlib. If you are unsure aboutthe installation process, you might want to install Anaconda Python distribution(which you can access at https://store.continuum.io/cshop/anaconda/), whichis driven by Travis Oliphant, a founding contributor of SciPy.
What sets Anacondaapart from other distributions such as Enthought Canopy (which you can downloadfrom https://www.enthought.com/downloads/) or Python(x,y) (accessible athttp://code.google.com/p/pythonxy/wiki/Downloads), is that Anaconda isalready fully Python 3 compatible—the Python version we will be using throughoutthe book.Chewing data efficiently with NumPy andintelligently with SciPyLet's walk quickly through some basic NumPy examples and then take a look atwhat SciPy provides on top of it. On the way, we will get our feet wet with plottingusing the marvelous Matplotlib package.[6]For an in-depth explanation, you might want to take a look at some of the moreinteresting examples of what NumPy has to offer at http://www.scipy.org/Tentative_NumPy_Tutorial.You will also find the NumPy Beginner's Guide - Second Edition, Ivan Idris, byPackt Publishing, to be very valuable.
Additional tutorial style guides can befound at http://scipy-lectures.github.com, and the official SciPy tutorialat http://docs.scipy.org/doc/scipy/reference/tutorial.In this book, we will use NumPy in version 1.8.1 andSciPy in version 0.14.0.Learning NumPySo let's import NumPy and play a bit with it. For that, we need to start the Pythoninteractive shell:>>> import numpy>>> numpy.version.full_version1.8.1As we do not want to pollute our namespace, we certainly should not use thefollowing code:>>> from numpy import *Because, for instance, numpy.array will potentially shadow the array package that isincluded in standard Python.
Instead, we will use the following convenient shortcut:>>> import numpy as np>>> a = np.array([0,1,2,3,4,5])>>> aarray([0, 1, 2, 3, 4, 5])>>> a.ndim1>>> a.shape(6,)So, we just created an array like we would create a list in Python. However, theNumPy arrays have additional information about the shape. In this case, it is aone-dimensional array of six elements.
No surprise so far.[7]Getting Started with Python Machine LearningWe can now transform this array to a two-dimensional matrix:>>> b = a.reshape((3,2))>>> barray([[0, 1],[2, 3],[4, 5]])>>> b.ndim2>>> b.shape(3, 2)The funny thing starts when we realize just how much the NumPy package isoptimized. For example, doing this avoids copies wherever possible:>>> b[1][0] = 77>>> barray([[ 0,1],[77,3],[ 4,5]])>>> aarray([ 0,1, 77,3,4,5])In this case, we have modified value 2 to 77 in b, and immediately see the samechange reflected in a as well. Keep in mind that whenever you need a true copy,you can always perform:>>> c = a.reshape((3,2)).copy()>>> carray([[ 0,1],[77,3],[ 4,5]])>>> c[0][0] = -99>>> aarray([ 0,1, 77,3,4,5])>>> carray([[-99,1],[ 77,3],[5]])4,[8]Note that here, c and a are totally independent copies.Another big advantage of NumPy arrays is that the operations are propagated to theindividual elements.
For example, multiplying a NumPy array will result in an arrayof the same size with all of its elements being multiplied:>>> d = np.array([1,2,3,4,5])>>> d*2array([ 2,4,6,8, 10])Similarly, for other operations:>>> d**2array([ 1,4,9, 16, 25])Contrast that to ordinary Python lists:>>> [1,2,3,4,5]*2[1, 2, 3, 4, 5, 1, 2, 3, 4, 5]>>> [1,2,3,4,5]**2Traceback (most recent call last):File "<stdin>", line 1, in <module>TypeError: unsupported operand type(s) for ** or pow(): 'list' and 'int'Of course by using NumPy arrays, we sacrifice the agility Python lists offer.
Simpleoperations such as adding or removing are a bit complex for NumPy arrays. Luckily,we have both at our hands and we will use the right one for the task at hand.IndexingPart of the power of NumPy comes from the versatile ways in which its arrays canbe accessed.In addition to normal list indexing, it allows you to use arrays themselves as indicesby performing:>>> a[np.array([2,3,4])]array([77,3,4])[9]Getting Started with Python Machine LearningTogether with the fact that conditions are also propagated to individual elements,we gain a very convenient way to access our data:>>> a>4array([False, False,True, False, False,True], dtype=bool)>>> a[a>4]array([77,5])By performing the following command, this can be used to trim outliers:>>> a[a>4] = 4>>> aarray([0, 1, 4, 3, 4, 4])As this is a frequent use case, there is the special clip function for it, clipping thevalues at both ends of an interval with one function call:>>> a.clip(0,4)array([0, 1, 4, 3, 4, 4])Handling nonexisting valuesThe power of NumPy's indexing capabilities comes in handy when preprocessingdata that we have just read in from a text file.
Most likely, that will contain invalidvalues that we will mark as not being a real number using numpy.NAN:>>> c = np.array([1, 2, np.NAN, 3, 4]) # let's pretend we have read thisfrom a text file>>> carray([1.,2.,nan,3.,4.])>>> np.isnan(c)array([False, False,True, False, False], dtype=bool)>>> c[~np.isnan(c)]array([ 1.,2.,3.,4.])>>> np.mean(c[~np.isnan(c)])2.5[ 10 ]Comparing the runtimeLet's compare the runtime behavior of NumPy compared with normal Python lists.In the following code, we will calculate the sum of all squared numbers from 1 to1000 and see how much time it will take.
We perform it 10,000 times and report thetotal time so that our measurement is accurate enough.import timeitnormal_py_sec = timeit.timeit('sum(x*x for x in range(1000))',number=10000)naive_np_sec = timeit.timeit('sum(na*na)',setup="import numpy as np; na=np.arange(1000)",number=10000)good_np_sec = timeit.timeit('na.dot(na)',setup="import numpy as np; na=np.arange(1000)",number=10000)print("Normal Python: %f sec" % normal_py_sec)print("Naive NumPy: %f sec" % naive_np_sec)print("Good NumPy: %f sec" % good_np_sec)Normal Python: 1.050749 secNaive NumPy: 3.962259 secGood NumPy: 0.040481 secWe make two interesting observations.
Firstly, by just using NumPy as data storage(Naive NumPy) takes 3.5 times longer, which is surprising since we believe it mustbe much faster as it is written as a C extension. One reason for this is that the accessof individual elements from Python itself is rather costly. Only when we are ableto apply algorithms inside the optimized extension code is when we get speedimprovements. The other observation is quite a tremendous one: using the dot()function of NumPy, which does exactly the same, allows us to be more than 25 timesfaster.
In summary, in every algorithm we are about to implement, we should alwayslook how we can move loops over individual elements from Python to some of thehighly optimized NumPy or SciPy extension functions.[ 11 ]Getting Started with Python Machine LearningHowever, the speed comes at a price. Using NumPy arrays, we no longer have theincredible flexibility of Python lists, which can hold basically anything. NumPyarrays always have only one data type.>>> a = np.array([1,2,3])>>> a.dtypedtype('int64')If we try to use elements of different types, such as the ones shown in the followingcode, NumPy will do its best to coerce them to be the most reasonable commondata type:>>> np.array([1, "stringy"])array(['1', 'stringy'], dtype='<U7')>>> np.array([1, "stringy", set([1,2,3])])array([1, stringy, {1, 2, 3}], dtype=object)Learning SciPyOn top of the efficient data structures of NumPy, SciPy offers a magnitude ofalgorithms working on those arrays.
![](https://s.studizba.com/z.php?f=/templates/si/i/noavatar.png&w=100&h=100&t=1"){kind=avatar}
