# 环境介绍
|ip |主机名 |角色|
|-------|-------|-------|
|192.168.200.161 |master |server|
|192.168.200.162 |slave1 |agent|
|192.168.200.163 |slave2 |agent|
# 安装环境准备
## JDK安装配置
```bash
mkdir -p /usr/share/java
cd /usr/share/java/
tar -zxvf ./jdk-8u201-linux-x64.tar.gz
vim /etc/profile
#jdk配置
export JAVA_HOME=/usr/share/java/jdk1.8.0_201
export CLASSPATH=.:${JAVA_HOME}/jre/lib/rt.jar:${JAVA_HOME}/lib/dt.jar:${JAVA_HOME}/lib/tools.jar
export PATH=$PATH:${JAVA_HOME}/bin
```
## 修改主机名，主机名映射(所有节点）
```bash
hostnamectl set-hostname cm-server #更改个主机名
vim /etc/hosts
192.168.200.161 master
192.168.200.162 slave1
192.168.200.163 slave2
```
## 关闭防火墙 selinux （所有节点）
```bash
systemctl stop firewalld
systemctl disable firewalld
#关闭selinux
sed -i 's/SELINUX=enforcing/SELINUX=disable/g' /etc/selinux/config
```
## 禁止交互（所有节点）
```bash
vim /etc/sysctl.conf
sudo sysctl vm.swappiness=0
```
## 禁止大页面（所有节点）
```bash
echo never > /sys/kernel/mm/transparent_hugepage/defrag
echo never > /sys/kernel/mm/transparent_hugepage/enabled
vim /etc/rc.local
#增加两行：
echo never > /sys/kernel/mm/transparent_hugepage/defrag
echo never > /sys/kernel/mm/transparent_hugepage/enabled
chmod +x /etc/rc.d/rc.local
```
## 免密登录
```bash
#在cdh-master生成密钥对
ssh-keygen -t rsa
#将公钥发送到其它节点
for i in `seq 1 2`;do ssh-copy-id -i /root/.ssh/id_rsa.pub root@slave$i;done
```
## NTP服务安装（所有节点）
```bash
#master节点配置
#查看是否安装的ntp时间服务器
rpm -qa | grep ntp
#安装ntp服务
yum install -y ntp ntpdate
#修改ntp配置文件
vim /etc/ntp.conf
# 中国这边最活跃的时间服务器 : http://www.pool.ntp.org/zone/cn
server 0.cn.pool.ntp.org
server 0.asia.pool.ntp.org
server 3.asia.pool.ntp.org
# allow update time by the upper server
# 允许上层时间服务器主动修改本机时间
restrict 0.cn.pool.ntp.org nomodify notrap noquery
restrict 0.asia.pool.ntp.org nomodify notrap noquery
restrict 3.asia.pool.ntp.org nomodify notrap noquery
# Undisciplined Local Clock. This is a fake driver intended for backup
# and when no outside source of synchronized time is available.
# 外部时间服务器不可用时，以本地时间作为时间服务
server 127.127.1.0 # local clock
fudge 127.127.1.0 stratum 10
#启动ntp服务
service ntpd start
service ntpd status
#查看是否已经打开服务
systemctl list-unit-files | grep ntp
#查看服务连接和监听
netstat -tlunp | grep ntp
#查看网络中的NTP服务器，同时显示客户端和每个服务器的关系
ntpq -p
#查看时间同步状态，这个一般需要5-10分钟后才能成功连接和同步。所以，服务器启动后需要稍等下
ntpstat
#slave节点配置
#配置/etc/ntp.conf文件，在此文件中添加如下配置：
vim /etc/ntp.conf
server 192.168.200.161 prefer
#执行chkconfig ntpd on，设置开机启动
chkconfig ntpd on
#执行service ntpd start启动ntpd服务
service ntpd start
#同步NTP-Server时间
ntpdate -u 192.168.200.161
6 Aug 13:44:12 ntpdate[6616]: adjust time server 192.168.200.161 offset 0.003185 sec
systemctl enable ntpd
```
## 安装mysql
```bash
rpm -qa | grep mariadb
#卸载自带数据库
rpm -e --nodeps mariadb-libs-5.5.56-2.el7.x86_64
cd /usr/local/src
wget http://repo.mysql.com/mysql57-community-release-el7-8.noarch.rpm
rpm -ivh mysql57-community-release-el7-8.noarch.rpm
yum -y install mysql-server
systemctl start mysqld
systemctl enable mysqld
```
关于mysql5.7密码策略问题：ERROR 1819 (HY000): Your password does not satisfy the current policy requirements
```bash
#进入数据库，修改只验证密码强度等级
set global validate_password_policy=LOW;
#设置验证密码长度
set global validate_password_length=6;
```
## 创建数据库
```bash
mysql -uroot -p123456
mysql> create database hive DEFAULT CHARSET utf8 COLLATE utf8_general_ci;
Query OK, 1 row affected (0.00 sec)
mysql> create database amon DEFAULT CHARSET utf8 COLLATE utf8_general_ci;
Query OK, 1 row affected (0.00 sec)
mysql> create database hue DEFAULT CHARSET utf8 COLLATE utf8_general_ci;
Query OK, 1 row affected (0.00 sec)
mysql> create database monitor DEFAULT CHARSET utf8 COLLATE utf8_general_ci;
Query OK, 1 row affected (0.00 sec)
mysql> create database oozie DEFAULT CHARSET utf8 COLLATE utf8_general_ci;
Query OK, 1 row affected (0.00 sec)
mysql> create database report DEFAULT CHARSET utf8 COLLATE utf8_general_ci;
Query OK, 1 row affected (0.00 sec)
mysql> grant all on *.* to root@"%" Identified by "123456";
Query OK, 0 rows affected (0.00 sec)
mysql> exit
```
# CM安装和CDH的准备
## 把所有准备好的相应包上传到master节点
```bash
[root@master home]# ls
CDH-5.16.1-1.cdh5.16.1.p0.3-el7.parcel mysql-community-libs-compat-5.7.24-1.el7.x86_64.rpm
CDH-5.16.1-1.cdh5.16.1.p0.3-el7.parcel.sha1 mysql-connector-java-5.1.46
cloudera-manager-centos7-cm5.16.1_x86_64.tar.gz mysql-connector-java-5.1.46.zip
manifest.json mysql-connector-java.jar
[root@master home]#tar zxvf cloudera-manager-centos7-cm5.16.1_x86_64.tar.gz -C /opt/
[root@master home]# ls /opt/
cloudera cm-5.16.1
```
## 拷贝jar包（所有节点）
```bash
ls
CDH-5.16.1-1.cdh5.16.1.p0.3-el7.parcel mysql-community-libs-compat-5.7.24-1.el7.x86_64.rpm
CDH-5.16.1-1.cdh5.16.1.p0.3-el7.parcel.sha1 mysql-connector-java-5.1.46
cloudera-manager-centos7-cm5.16.1_x86_64.tar.gz mysql-connector-java-5.1.46.zip
manifest.json mysql-connector-java.jar
cp mysql-connector-java.jar /usr/share/java/mysql-connector-java.jar
```
## 创建用户（所有节点）
```bash
useradd --system --home=/opt/cm-5.16.1/run/cloudera-scm-server --shell=/bin/false --comment "Cloudera SCM User" cloudera-scm
```
## 初始化数据库
```bash
/opt/cm-5.16.1/share/cmf/schema/scm_prepare_database.sh mysql cm -hlocalhost -uroot -p123456 --scm-host localhost scm scm scm
......
All done, your SCM database is configured correctly!
```
## 制作本地源(master节点）
```bash
ls
CDH-5.16.1-1.cdh5.16.1.p0.3-el7.parcel mysql-community-libs-compat-5.7.24-1.el7.x86_64.rpm
CDH-5.16.1-1.cdh5.16.1.p0.3-el7.parcel.sha1 mysql-connector-java-5.1.46
cloudera-manager-centos7-cm5.16.1_x86_64.tar.gz mysql-connector-java-5.1.46.zip
manifest.json mysql-connector-java.jar
mv CDH-5.16.1-1.cdh5.16.1.p0.3-el7.parcel.sha1 CDH-5.16.1-1.cdh5.16.1.p0.3-el7.parcel.sha
cp CDH-5.16.1-1.cdh5.16.1.p0.3-el7.parcel CDH-5.16.1-1.cdh5.16.1.p0.3-el7.parcel.sha1 manifest.json /opt/cloudera/parcel-repo/
#需将sha1改成sha
```
## 修改配置
```bash
#修改server_host为server节点的主机名
sed -i "s/server_host=localhost/server_host=192.168.200.161/" /opt/cm-5.16.1/etc/cloudera-scm-agent/config.ini
mkdir /opt/cm-5.16.1/run/cloudera-scm-agent
```
## 将文件cm-5.16.1发送到其它各节点
```bash
for i in `seq 1 2`;do scp -r /opt/cm-5.16.1 slave$i:/opt/;done
```
## 启动服务
```bash
#启动服务 master节点需要server agent都启 其它节点只启agent
/opt/cm-5.16.1/etc/init.d/cloudera-scm-server start
/opt/cm-5.16.1/etc/init.d/cloudera-scm-agent start
#关闭服务
/opt/cm-5.16.1/etc/init.d/cloudera-scm-server stop
#slave节点启动服务
/opt/cm-5.16.1/etc/init.d/cloudera-scm-agent start
```
浏览器页面开始安装
访问地址：master节点IP:7180
用户名和密码admin、admin
进入一步一步安装
上述不显示slave节点是因为agent文件是从master机器上拷贝过来的，该进程在运行时候会产生一个uuid文件，拷贝时这个文件一起拷贝过来了造成了uuid冲突
解决方法：删除 /opt/cm-5.16.1/lib/cloudera-scm-agent/所有目录 （刚搭建的新机器未存放数据）
参考链接
https://www.cnblogs.com/fujiangong/p/5620050.html https://blog.csdn.net/qq_39572733/article/details/89469741
CDH5.16.1集群部署
Copyright: 采用 知识共享署名4.0 国际许可协议进行许可
