环境准备

准备节点

node1: yum源，httpd，mariadb和java8环境
node2: java8环境
node3: java8环境
node4: java8环境

将主机名-ip映射配置到每个节点

/etc/hosts

127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6
192.168.107.74 node1
192.168.107.83 node2
192.168.107.105 node3
192.168.107.131 node4

免秘钥登录

在每个节点上生成ssh秘钥
-t rsa -P '' -f ~/.ssh/id_rsa
1
将主节点公钥依次发送到其余节点
.ssh/id_rsa.pub root@node2:~/
1
将主节点上的公钥追加到其他节点~/.ssh/authorized_keys
cat id_rsa.pub >> ~/.ssh/authorized_keys && cat id_rsa.pub >> ~/.ssh/authorized_keys && chmod 0600 ~/.ssh/authorized_keys
1
测试免秘钥登录
root@node2
1

关闭Selinux和防火墙

vim /etc/sysconfig/selinux
selinux=disalbed

systemctl stop firewalld
systemctl disable firewalld

时间同步

如果节点上时间不正确，则：

# 安装ntpdate工具 
yum -y install ntp ntpdate  
# 设置系统时间与网络时间同步   
ntpdate cn.pool.ntp.org
# 将系统时间写入硬件时间
hwclock --systohc

安装nscd

由于ambari server和client之间要通信，为了减轻dns压力使用nscd缓存

yum install -y nscd
systemctl enable nscd && systemctl start nscd

安装jdk

略

配置本地仓库

安装yum-utils
install yum-utils createrepo
1
创建HTTP服务
install httpd -y
1

下载源，根据自己想要的版本更改

将repo文件放在/etc/yum.repos.d/ 目录下，等会配置，tgz文件放在http服务器下：/var/www/html下

wget http://public-repo-1.hortonworks.com/ambari/centos7/2.x/updates/2.6.1.0/ambari.repo
wget http://public-repo-1.hortonworks.com/HDP/centos7/2.x/updates/2.6.4.0/hdp.repo
wget http://public-repo-1.hortonworks.com/ambari/centos7/2.x/updates/2.6.1.0/ambari-2.6.1.0-centos7.tar.gz
wget http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.22/repos/centos7/HDP-UTILS-1.1.0.22-centos7.tar.gz
wget http://public-repo-1.hortonworks.com/HDP-GPL/centos7/2.x/updates/2.6.4.0/HDP-GPL-2.6.4.0-centos7-rpm.tar.gz
wget http://public-repo-1.hortonworks.com/HDP/centos7/2.x/updates/2.6.4.0/HDP-2.6.4.0-centos7-rpm.tar.gz

将以上tgz文件解压在/var/www/html下

启动httpd测试访问

systemctl start httpd 访问：http://node1/ambari, http://node1/HDP, http://node1/HDP-UTILS...
配置yum源
- 更改/etc/yum.repos.d/下刚才下载的文件，将源路径指向node1
  
  vi /etc/yum.repos.d/hdp.repo
  
  [HDP-2.6.4.0] name=HDP Version - HDP-2.6.4.0 baseurl=http://node1/HDP/centos7/2.6.4.0-91 gpgcheck=1 gpgkey=http://node1/HDP/centos7/2.6.4.0-91/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins enabled=1 priority=1
  
  [HDP-UTILS-1.1.0.22] name=HDP-UTILS Version - HDP-UTILS-1.1.0.22 baseurl=http://node1/HDP-UTILS/centos7/1.1.0.22 gpgcheck=1 gpgkey=http://node1/HDP-UTILS/centos7/1.1.0.22/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins enabled=1 priority=1
  
  [HDP-GPL-2.6.4.0] name=HDP-GPL Version - HDP-GPL-2.6.4.0 baseurl=http://node1/HDP-GPL/centos7/2.6.4.0-91 gpgcheck=1 gpgkey=http://node1/HDP-GPL/centos7/2.6.4.0-91/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins enabled=1 priority=1
  
  vi /etc/yum.repos.d/hdp.repo
  
  [ambari-2.6.1.0] name=ambari Version - ambari-2.6.1.0 baseurl=http://node1/ambari/centos7/2.6.1.0-143 gpgcheck=1 gpgkey=http://node1/ambari/centos7/2.6.1.0-143/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins enabled=1 priority=1
- 缓存yum源
  clean all && yum makecache fast
  1

安装ambar

安装mariadb

yum install mariadb-server -y
systemctl start mariadb
mysql_secure_installation # 直接确定，然后设置root密码

# 创建数据库：

MariaDB [(none)]> create database ambari default character set utf8;
Query OK, 1 row affected (0.00 sec)

MariaDB [(none)]> grant all on ambari.* to ambari@localhost identified by 'ambari';
Query OK, 0 rows affected (0.00 sec)

MariaDB [(none)]> grant all on ambari.* to ambari@'%' identified by 'ambari';
Query OK, 0 rows affected (0.00 sec)

MariaDB [(none)]> create database hive default character set utf8;
Query OK, 1 row affected (0.00 sec)

MariaDB [(none)]> grant all on hive.* to hive@localhost identified by 'ambari';
Query OK, 0 rows affected (0.00 sec)

MariaDB [(none)]> grant all on hive.* to hive@'%' identified by 'ambari';

安装ambari-server

yum install ambari-server -y
ambari-server setup

安装过程中要选择数据库，如果出现如下警告：按照提示放入jdbc驱动jar包，并配置路径即可 WARNING: Before starting Ambari Server, you must copy the MySQL JDBC driver JAR file to /usr/> share/java and set property “server.jdbc.driver.path=[path/to/custom_jdbc_driver]” in ambari.properties.

vim /etc/ambari-server/conf/ambari.properties 
server.jdbc.driver.path=/usr/share/java/mysql-connector-java-5.1.34.jar

Using python  /usr/bin/python
Setup ambari-server
Checking SELinux...
SELinux status is 'enabled'
SELinux mode is 'permissive'
WARNING: SELinux is set to 'permissive' mode and temporarily disabled.
OK to continue [y/n] (y)? y
Customize user account for ambari-server daemon [y/n] (n)? y
Enter user account for ambari-server daemon (root):ambari
Adjusting ambari-server permissions and ownership...
Checking firewall status...
Checking JDK...
Do you want to change Oracle JDK [y/n] (n)? y
[1] Oracle JDK 1.8 + Java Cryptography Extension (JCE) Policy Files 8
[2] Oracle JDK 1.7 + Java Cryptography Extension (JCE) Policy Files 7
[3] Custom JDK
==============================================================================
Enter choice (1): 3
WARNING: JDK must be installed on all hosts and JAVA_HOME must be valid on all hosts.
WARNING: JCE Policy files are required for configuring Kerberos security. If you plan to use Kerberos,please make sure JCE Unlimited Strength Jurisdiction Policy Files are valid on all hosts.
Path to JAVA_HOME: /home/java/jre
Validating JDK on Ambari Server...done.
Checking GPL software agreement...
GPL License for LZO: https://www.gnu.org/licenses/old-licenses/gpl-2.0.en.html
Enable Ambari Server to download and install GPL Licensed LZO packages [y/n] (n)? n
Completing setup...
Configuring database...
Enter advanced database configuration [y/n] (n)? y
Configuring database...
==============================================================================
Choose one of the following options:
[1] - PostgreSQL (Embedded)
[2] - Oracle
[3] - MySQL / MariaDB
[4] - PostgreSQL
[5] - Microsoft SQL Server (Tech Preview)
[6] - SQL Anywhere
[7] - BDB
==============================================================================
Enter choice (3): 3
Hostname (localhost): 
Port (3306): 
Database name (ambari): 
Username (ambari): 
Enter Database Password (ambari): 
Configuring ambari database...
Configuring remote database connection properties...
WARNING: Before starting Ambari Server, you must run the following DDL against the database to create the schema: /var/lib/ambari-server/resources/Ambari-DDL-MySQL-CREATE.sql
Proceed with configuring remote database connection properties [y/n] (y)? n^Hy
input not recognized, please try again: 
Proceed with configuring remote database connection properties [y/n] (y)? y
Extracting system views...
ambari-admin-2.6.1.0.143.jar
...........
Adjusting ambari-server permissions and ownership...
Ambari Server 'setup' completed successfully.

启动ambari-server

ambari-server start

启动失败，查看日志：tail /var/log/ambari-server.log Caused by: com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Table ‘ambari.metainfo’ doesn’t exist

导入表后再次启动
mysql -uroot -p ambari < /var/lib/ambari-server/resources/Ambari-DDL-MySQL-CREATE.sql

测试访问：node1:8080

默认监听8080端口，如果失败，查看日志。使用默认账户admin/admin登录，界面如下

创建数据平台

选择Launch Install Wizard

![step0](https://i.imgur.com/ly0mo2a.png)

配置为本地仓库

![配置本地仓库](https://i.imgur.com/AAJH4Tv.png)

添加主机节点

    The user account used to install the Ambari Agent on the target host(s) via SSH. This user must be set up with passwordless SSH and sudo access on all the target host(s)
    添加主节点node1私钥id_rsa，必须能够免秘钥登录其他节点安装ambari agent
![配置节点](https://i.imgur.com/yY2LyF4.png)

Confirm Hosts

在这一步可能碰见很多问题，执行之前一定要用tail -f实时查看日志，注意查看的是

-flink
1
	


> 错误示例一：

> 查看openssl版本没有问题，更改所有ambari-client配置：在[security]单元下添加：force_https_protocol=PROTOCOL_TLSv1_2

/etc/ambari-agent/conf/ambari-agent.ini
1
	


    ERROR 2018-06-17 12:50:36,164 NetUtil.py:96 - EOF occurred in violation of protocol (_ssl.c:765)
    ERROR 2018-06-17 12:50:36,164 NetUtil.py:97 - SSLError: Failed to connect. Please check openssl library versions. 
    ......
    ......
    INFO 2018-06-17 12:50:35,295 main.py:437 - Connecting to Ambari server at https://node1:8440 (192.168.107.74)
    INFO 2018-06-17 12:50:35,295 NetUtil.py:70 - Connecting to https://node1:8440/ca
    ERROR 2018-06-17 12:50:36,164 NetUtil.py:96 - EOF occurred in violation of protocol (_ssl.c:765)
    ERROR 2018-06-17 12:50:36,164 NetUtil.py:97 - SSLError: Failed to connect. Please check openssl library versions. 
    Refer to: https://bugzilla.redhat.com/show_bug.cgi?id=1022468 for more details.

> 错误示例二：

> 不能创建文件夹，赋予权限，ambari-server在确认主机和分配服务均需要被操作

-R ambarilink
1
	


    17 Jun 2018 12:20:23,259 ERROR [Thread-35] BSRunner:441 - java.io.FileNotFoundException: /var/run/ambari-server/bootstrap/1/node1.done (No such file or directory)
    17 Jun 2018 12:20:23,260  WARN [Thread-35] BSRunner:401 - File does not exist: /var/run/ambari-server/bootstrap/1/sshKey
    17 Jun 2018 12:41:20,217  INFO [Thread-37] BSRunner:189 - Kicking off the scheduler for polling on logs in /var/run/ambari-server/bootstrap/1

> 错误示例三：

> 注意主节点信任主机.ssh/known_hosts以及免秘钥登录除了客户机也必须要有自己，如果成功则如下：

确认主机

选择服务

根据自己系统配置，建议可以一个个添加，比如先添加HDFS、YARN和MapReduce

选择服务

assign master

相当于做集群配置指派master

主从分配

自定义配置

红色的表示还需要进一步配置，数据库登录，根据提示来即可之前已经添加了jdbc驱动，因此这里只要执行：

setup --jdbc-db

```

![自定义配置](https://i.imgur.com/ZnrSK6B.png)

## 测试MapReduce
**注意：root用户是不能操作hdfs文件系统的，当然可以更改hdfs-site.xml关闭权限验证，不过建议切换hdfs用户来操作**

su hdfs && cd ~

创建文件wc.txt，同时将文件上传到hdfs

vi wc.txt
    hadoop hello world
    zk hello hadoop
    hbase hive
    zk spark
hadoop fs -mkdir /tmp/input
hadoop fs -put wc.txt /tmp/input
# 执行MapReduce，根据自己的版本和文件选择hdp的示例，或者使用自己的jar包
hadoop jar /usr/hdp/2.6.4.0-91/hadoop-mapreduce/hadoop-mapreduce-examples-2.7.3.2.6.4.0-91.jar wordcount /tmp/input/wc.txt /tmp/output

官方示例源码

http://hadoop.apache.org/docs/current/hadoop-mapreduce-client/hadoop-mapreduce-client-core/MapReduceTutorial.html#Source_Code