Tom White - Hadoop The Definitive Guide_ 4 edition - 2015 (811394), страница 70
Текст из файла (страница 70)
For example, to grow the storage available to a cluster, you commissionnew nodes. Conversely, sometimes you may wish to shrink a cluster, and to do so, youdecommission nodes. Sometimes it is necessary to decommission a node if it is misbe‐having, perhaps because it is failing more often than it should or its performance isnoticeably slow.Nodes normally run both a datanode and a node manager, and both are typicallycommissioned or decommissioned in tandem.Commissioning new nodesAlthough commissioning a new node can be as simple as configuring the hdfssite.xml file to point to the namenode, configuring the yarn-site.xml file to point to theresource manager, and starting the datanode and resource manager daemons, it is gen‐erally best to have a list of authorized nodes.It is a potential security risk to allow any machine to connect to the namenode and actas a datanode, because the machine may gain access to data that it is not authorized tosee.
Furthermore, because such a machine is not a real datanode, it is not under yourcontrol and may stop at any time, potentially causing data loss. (Imagine what wouldhappen if a number of such nodes were connected and a block of data was present onlyon the “alien” nodes.) This scenario is a risk even inside a firewall, due to the possibilityof misconfiguration, so datanodes (and node managers) should be explicitly managedon all production clusters.Datanodes that are permitted to connect to the namenode are specified in a file whosename is specified by the dfs.hosts property. The file resides on the namenode’s localfilesystem, and it contains a line for each datanode, specified by network address (asreported by the datanode; you can see what this is by looking at the namenode’s webUI).
If you need to specify multiple network addresses for a datanode, put them on oneline, separated by whitespace.334|Chapter 11: Administering HadoopSimilarly, node managers that may connect to the resource manager are specified in afile whose name is specified by the yarn.resourcemanager.nodes.include-pathproperty. In most cases, there is one shared file, referred to as the include file, that bothdfs.hosts and yarn.resourcemanager.nodes.include-path refer to, since nodes inthe cluster run both datanode and node manager daemons.Thefile (or files) specified by the dfs.hosts andyarn.resourcemanager.nodes.include-path properties is differentfrom the slaves file.
The former is used by the namenode and re‐source manager to determine which worker nodes may connect. Theslaves file is used by the Hadoop control scripts to perform cluster-wide operations, such as cluster restarts. It is never used by the Ha‐doop daemons.To add new nodes to the cluster:1. Add the network addresses of the new nodes to the include file.2.
Update the namenode with the new set of permitted datanodes using thiscommand:% hdfs dfsadmin -refreshNodes3. Update the resource manager with the new set of permitted node managers using:% yarn rmadmin -refreshNodes4. Update the slaves file with the new nodes, so that they are included in future oper‐ations performed by the Hadoop control scripts.5.
Start the new datanodes and node managers.6. Check that the new datanodes and node managers appear in the web UI.HDFS will not move blocks from old datanodes to new datanodes to balance the cluster.To do this, you should run the balancer described in “Balancer” on page 329.Decommissioning old nodesAlthough HDFS is designed to tolerate datanode failures, this does not mean you canjust terminate datanodes en masse with no ill effect.
With a replication level of three,for example, the chances are very high that you will lose data by simultaneously shuttingdown three datanodes if they are on different racks. The way to decommission datanodesis to inform the namenode of the nodes that you wish to take out of circulation, so thatit can replicate the blocks to other datanodes before the datanodes are shut down.With node managers, Hadoop is more forgiving. If you shut down a node manager thatis running MapReduce tasks, the application master will notice the failure and resched‐ule the tasks on other nodes.Maintenance|335The decommissioning process is controlled by an exclude file, which is set for HDFSibythedfs.hosts.excludepropertyandforYARNbytheyarn.resourcemanager.nodes.exclude-path property.
It is often the case that theseproperties refer to the same file. The exclude file lists the nodes that are not permittedto connect to the cluster.The rules for whether a node manager may connect to the resource manager are simple:a node manager may connect only if it appears in the include file and does not appearin the exclude file. An unspecified or empty include file is taken to mean that all nodesare in the include file.For HDFS, the rules are slightly different. If a datanode appears in both the include andthe exclude file, then it may connect, but only to be decommissioned. Table 11-3 sum‐marizes the different combinations for datanodes. As for node managers, an unspecifiedor empty include file means all nodes are included.Table 11-3. HDFS include and exclude file precedenceNode appears in include file Node appears in exclude file InterpretationNoNoNode may not connect.NoYesNode may not connect.YesNoNode may connect.YesYesNode may connect and will be decommissioned.To remove nodes from the cluster:1.
Add the network addresses of the nodes to be decommissioned to the exclude file.Do not update the include file at this point.2. Update the namenode with the new set of permitted datanodes, using thiscommand:% hdfs dfsadmin -refreshNodes3. Update the resource manager with the new set of permitted node managers using:% yarn rmadmin -refreshNodes4. Go to the web UI and check whether the admin state has changed to “DecommissionIn Progress” for the datanodes being decommissioned.
They will start copying theirblocks to other datanodes in the cluster.5. When all the datanodes report their state as “Decommissioned,” all the blocks havebeen replicated. Shut down the decommissioned nodes.6. Remove the nodes from the include file, and run:% hdfs dfsadmin -refreshNodes% yarn rmadmin -refreshNodes336|Chapter 11: Administering Hadoop7. Remove the nodes from the slaves file.UpgradesUpgrading a Hadoop cluster requires careful planning.
The most important consider‐ation is the HDFS upgrade. If the layout version of the filesystem has changed, then theupgrade will automatically migrate the filesystem data and metadata to a format that iscompatible with the new version. As with any procedure that involves data migration,there is a risk of data loss, so you should be sure that both your data and the metadataare backed up (see “Routine Administration Procedures” on page 332).Part of the planning process should include a trial run on a small test cluster with a copyof data that you can afford to lose.
A trial run will allow you to familiarize yourself withthe process, customize it to your particular cluster configuration and toolset, and ironout any snags before running the upgrade procedure on a production cluster. A testcluster also has the benefit of being available to test client upgrades on.
You can readabout general compatibility concerns for clients in the following sidebar.CompatibilityWhen moving from one release to another, you need to think about the upgrade stepsthat are needed. There are several aspects to consider: API compatibility, data compat‐ibility, and wire compatibility.API compatibility concerns the contract between user code and the published HadoopAPIs, such as the Java MapReduce APIs.
Major releases (e.g., from 1.x.y to 2.0.0) areallowed to break API compatibility, so user programs may need to be modified andrecompiled. Minor releases (e.g., from 1.0.x to 1.1.0) and point releases (e.g., from 1.0.1to 1.0.2) should not break compatibility.Hadoop uses a classification scheme for API elements to denotetheir stability. The preceding rules for API compatibility coverthose elements that are marked InterfaceStability.Stable.Some elements of the public Hadoop APIs, however, are markedwith the InterfaceStability.Evolving or InterfaceStability.Unstable annotations (all these annotations are in theorg.apache.hadoop.classification package), which meanthey are allowed to break compatibility on minor and point re‐leases, respectively.Data compatibility concerns persistent data and metadata formats, such as the formatin which the HDFS namenode stores its persistent data.
The formats can change acrossminor or major releases, but the change is transparent to users because the upgrade willMaintenance|337automatically migrate the data. There may be some restrictions about upgrade paths,and these are covered in the release notes. For example, it may be necessary to upgradevia an intermediate release rather than upgrading directly to the later final release inone step.Wire compatibility concerns the interoperability between clients and servers via wireprotocols such as RPC and HTTP.
The rule for wire compatibility is that the client musthave the same major release number as the server, but may differ in its minor or pointrelease number (e.g., client version 2.0.2 will work with server 2.0.1 or 2.1.0, but notnecessarily with server 3.0.0).This rule for wire compatibility differs from earlier versions ofHadoop, where internal clients (like datanodes) had to be upgra‐ded in lockstep with servers. The fact that internal client andserver versions can be mixed allows Hadoop 2 to support roll‐ing upgrades.The full set of compatibility rules that Hadoop adheres to are documented at the ApacheSoftware Foundation’s website.Upgrading a cluster when the filesystem layout has not changed is fairlystraightforward: install the new version of Hadoop on the cluster (and on clients at thesame time), shut down the old daemons, update the configuration files, and then startup the new daemons and switch clients to use the new libraries.
![](https://s.studizba.com/z.php?f=/templates/si/i/noavatar.png&w=100&h=100&t=1"){kind=avatar}
