Blog of Samperson

Hadoop HA相关部署和配置

2019-11-01

集群的规划

[1] 集群规划

IP 角色
192.168.130.3 NameNode1主节点, ResourceManager1主节点, Journalnode
192.168.130.4 NameNode2备用主节点, ResourceManager2备用主节点, Journalnode
192.168.130.5 DataNode1, NodeManager1

[2] 相关版本

JDK Hadoop Zookeeper
/home/jdk1.8.0_211 /home/hadoop-2.8.5 /home/zookeeper3.4.14

[3] 登录方式

1
2
sudo su
ssh -p 22 user@192.168.130.*

安装Zookeeper集群

首先在192.168.130.3上:

[1] 修改配置文件/conf/zoo.cfg:

1
dataDir=/home/zookeeper3.4.14/tmp

在最后一行添加:

1
2
3
server.1=192.168.130.3:2888:3888
server.2=192.168.130.4:2888:3888
server.3=192.168.130.5:2888:3888

[2] 修改myid配置文件
/home/zookeeper3.4.14目录下创建一个/tmp/myid空文件

1
2
3
4
mkdir tmp
cd tmp
touch myid
echo 1 > myid

[3] 将配置好的zookeeper拷贝到其他节点

1
2
scp -r /home/zookeeper3.4.14/ user@192.168.130.4:/home/
scp -r /home/zookeeper3.4.14/ user@192.168.130.5:/home/

进入192.168.130.4和192.168.130.5两台机器中,找到myid文件,修改各自的myid文件:在192.168.130.4中输入2,192.168.130.5中输入3

安装Hadoop集群

首先在192.168.130.3上:

[1] 修改core-site.xml

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
<configuration>
<property>
<name>fs.defaultFS</name>
<value>hdfs://ns1</value>
</property>

<property>
<name>hadoop.tmp.dir</name>
<value>/home/sunzesong/hadoop-2.8.5/tmp</value>
</property>

<property>
<name>ha.zookeeper.quorum</name>
<value>192.168.130.3:2181, 192.168.130.4:2181, 192.168.130.5:2181</value>
</property>
</configuration>

[2] 修改hdfs-site.xml

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
<configuration> 
<property>
<name>dfs.nameservices</name>
<value>ns1</value>
</property>

<property>
<name>dfs.ha.namenodes.ns1</name>
<value>nn1,nn2</value>
</property>

<property>
<name>dfs.namenode.rpc-address.ns1.nn1</name>
<value>192.168.130.3:9000</value>
</property>

<property>
<name>dfs.namenode.http-address.ns1.nn1</name>
<value>192.168.130.3:50070</value>
</property>

<property>
<name>dfs.namenode.rpc-address.ns1.nn2</name>
<value>192.168.130.4:9000</value>
</property>

<property>
<name>dfs.namenode.http-address.ns1.nn2</name>
<value>192.168.130.4:50070</value>
</property>

<property>
<name>dfs.namenode.shared.edits.dir</name>
<value>qjournal://192.168.130.3:8485;192.168.130.4:8485;/ns1</value>
</property>

<property>
<name>dfs.journalnode.edits.dir</name>
<value>/home/sunzesong/hadoop-2.8.5/journal</value>
</property>

<property>
<name>dfs.ha.automatic-failover.enabled</name>
<value>true</value>
</property>

<property>
<name>dfs.client.failover.proxy.provider.ns1</name>
<value>org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider</value>
</property>

<!-- 配置隔离机制方法,多个机制用换行分割,即每个机制暂用一行-->
<property>
<name>dfs.ha.fencing.methods</name>
<value>
sshfence
shell(/bin/true)
</value>
</property>

<!-- 使用sshfence隔离机制时需要ssh免登陆 -->
<property>
<name>dfs.ha.fencing.ssh.private-key-files</name>
<value>/root/.ssh/id_rsa</value>
</property>

<!-- 配置sshfence隔离机制超时时间 -->
<property>
<name>dfs.ha.fencing.ssh.connect-timeout</name>
<value>30000</value>
</property>
</configuration>

[3] 将配置好的hadoop拷贝到其他节点

1
2
scp -r /home/hadoop-2.8.5/ user@192.168.130.4:/home/
scp -r /home/hadoop-2.8.5/ user@192.168.130.5:/home/

启动Zookeeper集群和Hadoop集群

[1] 启动Zookeeper集群
在每一台机器上:

1
./zkServer.sh start

[2] 启动journalnode
在192.168.130.3和192.168.130.4两台节点上启动journalnode节点:

1
hadoop-daemon.sh start journalnode

[3] 格式化
格式化HDFS:hdfs namenode -format
格式化zookeeper:hdfs zkfc -formatZK手动输入,不可以复制粘贴

[4] 启动Hadoop集群
在192.168.130.3:start-dfs.sh
输出:

1
2
3
4
5
6
7
8
9
10
Starting namenodes on [192.168.130.3 192.168.130.4]
192.168.130.3: starting namenode, logging to /home/hadoop-2.8.5/logs/hadoop-namenode-s03.out
192.168.130.4: starting namenode, logging to /home/hadoop-2.8.5/logs/hadoop-namenode-pc04.out
192.168.130.5: starting datanode, logging to /home/hadoop-2.8.5/logs/hadoop-datanode-s5.out
Starting journal nodes [192.168.130.3 192.168.130.4 ]
192.168.130.3: starting journalnode, logging to /home/hadoop-2.8.5/logs/hadoop-journalnode-s03.out
192.168.130.4: starting journalnode, logging to /home/hadoop-2.8.5/logs/hadoop-journalnode-pc04.out
Starting ZK Failover Controllers on NN hosts [192.168.130.3 192.168.130.4]
192.168.130.3: starting zkfc, logging to /home/hadoop-2.8.5/logs/hadoop-zkfc-s03.out
192.168.130.4: starting zkfc, logging to /home/hadoop-2.8.5/logs/hadoop-zkfc-pc04.out

参考资料

[1] https://yq.aliyun.com/articles/606740
[2] https://www.cnblogs.com/yousen/p/5191075.html
[3] https://www.cnblogs.com/shizhijie/p/9878078.html