zoukankan      html  css  js  c++  java
  • CDH6安装文档

    1、准备工作

    1.1 环境

    centos7、jdk8、mysql5.7、python2.7、CDH6
    

    1.2文件下载

    1.2.1 cloudmanger地址

    https://archive.cloudera.com/cm6/6.0.0/redhat7/yum/RPMS/x86_64/ 
    
    • cloudera-manager-agent-6.0.0-530873.el7.x86_64.rpm
    • cloudera-manager-daemons-6.0.0-530873.el7.x86_64.rpm
    • cloudera-manager-server-6.0.0-530873.el7.x86_64.rpm

    1.2.2 资源下载地址

    https://archive.cloudera.com/cdh6/6.0.0/parcels/
    
    • CDH-6.0.0-1.cdh6.0.0.p0.537114-el7.parcel
    • CDH-6.0.0-1.cdh6.0.0.p0.537114-el7.parcel.sha256
    • manifest.json

    2、开始安装

    2.1 服务器配置

    2.1.1 修改HOST和主机名

    1. vim /etc/hosts修改host,例如

      198.168.12.1 cdh1

      192.168.12.2 cdh2

      192.168.12.3 cdh3

      不能有大写字母和下划线
    2. vim /etc/hostname修改主机名,修改完后三台机器分别执行hostname ***(当前节点的主机名)

    2.1.2 配置免密登录

    1. ssh-keygen -t rsa -P '' -f ~/.ssh/id_rsa b、ssh-copy-id 每台服务器地址
    

    2.1.3 关闭防火墙

    1. 临时关闭 systemctl stop firewalld
    2. 禁止开机启动 systemctl disable firewalld
    

    2.1.4 安装jdk,配置环境变量

    这个大家都会,如果不会可以参考这个教程

    https://www.cnblogs.com/ocean-sky/p/8392444.html

    2.1.5 其他依赖

    每台服务器yum安装依赖包:redhat-lsb、portmap、httpd、mod_ssl、openssl-devel、python-psycopg2、MySQL-python、chkconfig、python、chkconfig、bind-utils、libxslt、zlib、sqlite、cyrus-sasl-plain、cyrus-sasl-gssapi、fuse、fuse-libs

    2.2 Mysql5.7安装(在主节点安装)

    2.2.1 安装

    1.  wget -i -c http://dev.mysql.com/get/mysql57-community-release-el7-10.noarch.rpm
    2.  yum -y install mysql57-community-release-el7-10.noarch.rpm
    3.  yum -y install mysql-community-server
    4.  启动	systemctl start  mysqld.service
    5.  查看运行状态 systemctl status mysqld.service
    6.  查看初始密码 grep "password" /var/log/mysqld.log
    7.  登录后修改密码(尽量复杂一点,太简单了验证不通过,也可以查看如何关闭mysql校验规则) ALTER USER 'root'@'localhost' IDENTIFIED BY 'new password';
    8.  卸载yum源防止自动更新 yum -y remove mysql57-community-release-el7-10.noarch</br>
    9.  安装MySQL-shared-compat或MySQL-shared软件包(这是Cloudera Manager Agent软件包安装所必需的)yum install MySQL-shared-advanced
    

    2.2.2 创建CDH相关数据库

    • create database scm default character set utf8 default collate utf8_general_ci;
    • grant all on scm.* to 'scm'@'%' identified by 'password';
    • create database amon default character set utf8 default collate utf8_general_ci;
    • grant all on amon.* to 'amon'@'%' identified by 'password';
    • create database rman default character set utf8 default collate utf8_general_ci;
    • grant all on rman.* to 'rman'@'%' identified by 'password';
    • create database hue default character set utf8 default collate utf8_general_ci;
    • grant all on hue.* to 'hue'@'%' identified by 'password';
    • create database metastore default character set utf8 default collate utf8_general_ci;
    • grant all on metastore.* to 'hive'@'%' identified by 'password';
    • create database sentry default character set utf8 default collate utf8_general_ci;
    • grant all on sentry.* to 'sentry'@'%' identified by 'password';
    • create database nav default character set utf8 default collate utf8_general_ci;
    • grant all on nav.* to 'nav'@'%' identified by 'password';
    • create database oozie default character set utf8 default collate utf8_general_ci;
    • grant all on oozie.* to 'oozie'@'%' identified by 'password';

    2.3 CDH安装

     1.  mkdir /opt/cloudera-manager
     2.  将cloudera-manager-agent-6.0.0-530873.el7.x86_64.rpm、cloudera-manager-daemons-6.0.0-530873.el7.x86_64.rpm	、cloudera-manager-server-6.0.0-530873.el7.x86_64.rpm
    

    复制到/opt/cloudera-manager文件夹中
    3. 按 daemons、server、agent顺序解压
    4. 修改文件/etc/cloudera-scm-agent/config.ini中serverhost指定server节点
    5. 下载mysql-connector-java-5.1.47.jar包并复制为/usr/share/java/mysql-connector-java.jar
    6. 在主节点执行
    /opt/cloudera/cm/schema/scmpreparedatabase.sh mysql -h 主节点ip --scm-host 主节点ip scm scm
    7. mkdir -p /opt/cloudera/parcel-repo
    8. 将CDH-6.0.0-1.cdh6.0.0.p0.537114-el7.parcel、CDH-6.0.0-1.cdh6.0.0.p0.537114-el7.parcel.sha256、manifest.json复制到/opt/cloudera/parcel-repo
    9. 将CDH-6.0.0-1.cdh6.0.0.p0.537114-el7.parcel.sha256 重命名为 CDH-6.0.0-1.cdh6.0.0.p0.537114-el7.parcel.sha
    10. 在manifest.json文件中,找到对应版本的秘钥,复制到CDH-6.0.0-1.cdh6.0.0.p0.537114-el7.parcel.sha文件中。
    11. 启动systemctl start cloudera-scm-agent、systemctl start cloudera-scm-server

    2.4 集群配置

    2.4.1 禁用透明大页面压缩

    echo never > /sys/kernel/mm/transparent_hugepage/defrag

    echo never > /sys/kernel/mm/transparent_hugepage/enabled

    2.4.1 分配组件

    访问http://192.168.1.1:7180开始配置集群组件
    因为是先安装集群后编写文档,没有截图就没有详细写。

  • 相关阅读:
    Building Java Projects with Gradle
    Vert.x简介
    Spring及Spring Boot 国内快速开发框架
    dip vs di vs ioc
    Tools (StExBar vs Cmder)which can switch to command line window on context menu in windows OS
    SSO的定义、原理、组件及应用
    ModSecurity is an open source, cross-platform web application firewall (WAF) module.
    TDD中测试替身学习总结
    Spring事务银行转账示例
    台式机(华硕主板)前面板音频接口(耳机和麦克风)均无声的解决办法
  • 原文地址:https://www.cnblogs.com/aitwhite/p/9970694.html
Copyright © 2011-2022 走看看