zoukankan      html  css  js  c++  java
  • 迁移数据库mysql

     文档 <http://site.clairvoyantsoft.com/migrate-cloudera-scm-database-from-postgresql-to-mysql/> 

    http://node0:7180/api/v14/cm/deployment          -------------正确网址。
    #####1.停止服务;
        1.在UI界面,停止集群;
        2.在UI界面,停止cm的service,monitor等全部服务;
        3.在Linux,停止agent服务:service cloudera-scm-agent stop
    #####2.备份CM配置,通过API ------
        $ curl -v -u admin:admin http://node0:7180/api/<version>/cm/deployment > /home/user/deployment
        本机:
        $ curl -v -u admin:admin http://node0:7180/api/v14/cm/deployment > /home/user/deployment
    注意:这个文件被保留和保护至关重要,因为它将需要重新导入功能配置到新数据库中。(v14)
    Obtain version with: http://manager_host:7180/api/version
    #####3.对postgre的db和server目录做备份。
        #cd /var/lib/cloudera-scm-server-db/
        #tar -cvf /home/user/cloudera-scm-server-db.tar .
        #cd /etc/cloudera-scm-server/
        #tar -cvf /home/user/cloudera-scm-server.tar .
    文件归档在 /home/user/ 下。
    #####4.在server端,停止server,停止server-db,抛弃server-db off
        $ sudo service cloudera-scm-server stop 
        $ sudo service cloudera-scm-server-db stop 
        $ sudo chkconfig cloudera-scm-server-db off 
    #####5.保证mysql server正确安装。--------------------<font color=red>[1]</font>
    #####6.安装、配置mysql软件------<font color=red>[2]</font>
    #####7.创建"temp" db 并且赋权限,记录“示例1:运行脚本当MySQL安装在另一个主机”<font color=red>[3]</font>
        #mysql -h mysql_hostname -u user -p
        mysql> create database temp; 
        mysql> grant all on *.* to 'temp'@'%' identified by 'temp' with grant option; 
    #####8.执行/usr/share/cmf/schema/scm_prepare_database.sh(可重置)
        $/usr/share/cmf/schema/scm_prepare_database.sh mysql -h mysql_hostname  -u user -p --scm-host manager_host <scm_db_name> <scm_user> <scm_password>
    Note:  run $ /usr/share/cmf/schema/scm_prepare_database.sh 
    with no arguments to get syntax assistance
    #####9.开启server服务。
        service httpd start 
        $ sudo service cloudera-scm-server start 
    #####10.Cloudera Manager web用户界面登录和上传企业许可证密钥。
    #####11.重启scm-server
        $ sudo service cloudera-scm-server restart 
    #####12.修复CM配置(step2),通过API---
        $ curl --upload-file /home/user/deployment -u admin:admin http://node0:7180/api/v14/cm/deployment?deleteCurrentDeployment=true -H "Content-Type: application/json"
        //add http header option(恢复集群的命令): -H "Content-Type: application/json" ..if required
        //verify that there are no errors, the command prints the api even on connection failure.
    #####13.登录CM,验证当前的配置选项。所有服务现在应该可用但是停止状态。

        尝试,所有主机-重新运行升级向导。

    #####14.cleanup: 
        $ yum erase cloudera-manager-server-sb
    #####15.如果scm-agent启动失败,你必须删除或者一出pid ./var/lib/cloudera-scm-agent/ 并且重启agent。
    ###参考网址
    [1] <http://www.cloudera.com/documentation/enterprise/latest/topics/cm_ig_mysql.html#cmig_topic_5_5_2>
    [2] <http://dev.mysql.com/downloads/mysql/>
    [3] <http://www.cloudera.com/documentation/enterprise/latest/topics/cm_ig_installing_configuring_dbs.html#concept_i2r_m3m_hn__example_fsj_cyp_bm>
    [4] ref: https://gist.github.com/gwenshap/7044525</br>
    <font color = 'red'>[5] <http://site.clairvoyantsoft.com/migrate-cloudera-scm-database-from-postgresql-to-mysql/>【译】</br></font>
    http://cwiki.apachecn.org/pages/viewpage.action?pageId=2888090

    参考网址1-----配置mysql server和jdbc driver:
    1.



    参考网址3:在另一台主机配置mysql2,连接mysql1。


    ---
    3、 查看postgresql 的scm用户的密码
    grep password /etc/cloudera-scm-server/db.properties
    会得到如下信息:
    [root@hadoop1 /]# grep password /etc/cloudera-scm-server/db.properties
    com.cloudera.cmf.db.password=pfxlUudKJo

    4、 登陆postgresql 数据库
    psql -h localhost -p 7432 -U scm
    输入第三步查到的密码

    /usr/share/cmf/schema/scm_prepare_database.sh postgresql scm scm scm

    /etc/init.d/cloudera-manager-server-db restart

    http://blog.csdn.net/dracotianlong/article/details/7907633
    <http://blog.csdn.net/mirabelle521/article/details/71169631>



    **guid删除,重新运行升级向导**
    /var/lib/cloudera-scm-agent/cm_guid 删除guid--主机状况不良(由于原集群的guid存在唯一性,过时)
    ---http://blog.csdn.net/wonder4/article/details/58585180
    ---http://blog.csdn.net/u011031430/article/details/74057601
    ---https://www.cloudera.com/documentation/enterprise/release-notes/topics/cm_rn_known_issues.html#rn_OPSAPS-34847

    ---**重新运行升级向导<br>**

    ---parcel没有分发,但是是活动的<BR>

    **取消parcel异常分发状态--激活卡主**
    > use curl -X POST with Cloudera API.
    > http://node0:7180/api/v14/cm/deployment中找到相应的配置。
    > 格式API文档:http://node0:7180/static/apidocs/

        curl -u admin:admin -X POST http://node0:7180/api/v14/clusters/cluster/parcels/products/KAFKA/versions/2.0.2-1.2.0.2.p0.5/commands/deactivate

        curl -u admin:admin -X POST http://node0:7180/api/v14/clusters/cluster/parcels/products/CDH/versions/5.8.0-1.cdh5.8.0.p0.42/commands/deactivate

    <http://community.cloudera.com/t5/Cloudera-Manager-Installation/Parcel-not-distributed-but-have-active-state-ACTIVATING/m-p/32474/highlight/true><br>
    **ctrl + B** 

    拒绝连接-- /etc/hosts

    **官方文档**

        https://www.cloudera.com/documentation/enterprise/latest/topics/installation.html

  • 相关阅读:
    论文笔记:SRCNN
    4.2 CNN实例探究
    4.1 卷积神经网络
    3 ML策略
    2.3 超参数调试,batch正则化和程序框架
    2.2 优化算法
    2.1 深度学习的实用层面
    Lecture4 反向传播算法
    Lecture3 神经网络学习
    java基础部分
  • 原文地址:https://www.cnblogs.com/hexu105/p/7654623.html
Copyright © 2011-2022 走看看