zoukankan      html  css  js  c++  java
  • MHA+Atlas+mysql一主一从开启gtid安装配置与实验

    各节点架构

    (说明:生产环境有两个节点可以组成一套完整集群,我是测试环境,因此对于manager以及atlas和binlog server都是单点,如果生产环境,相应的将manager以及atlas和binlog server每个节点都部署即可)

    10.80.8.89 mysql-master manager,node atlas
    10.80.8.90 mysql-slave node binlog server

    安装步骤

    10.80.8.89操作命令

    1.#增加mha用户

    useradd mha
    
    passwd mha

    2.#增加mysql用户

    useradd mysql
    
    passwd mysql

    3.#生成私钥

    ssh-keygen -t rsa

    4.#各节点建立互信

    su -mha
    
    ssh-copy-id -i ./.ssh/id_rsa.pub 10.80.8.89
    
    ssh-copy-id -i ./.ssh/id_rsa.pub 10.80.8.90

    5.#安装mysql,二进制安装

    mkdir -p /soft
    
    cd /soft && wget http://cdn.mysql.com//Downloads/MySQL-5.6/mysql-5.6.30-linux-glibc2.5-x86_64.tar.gz
    
    tar -zxf mysql-5.6.30-linux-glibc2.5-x86_64.tar.gz
    
    ln -s /soft/mysql-5.6.30-linux-glibc2.5-x86_64 /usr/local/mysql
    
    echo "export $PATH=/usr/local/mysql/bin:$PATH">>/etc/profile
    
    source /etc/profile
    
    mkdir -p /data/mysql/3306/{data,logs,etc,tmp}
    
    chown -R mysql:mysql /data/mysql/3306

    6.#上传配置文件my.cnf到/data/mysql/3306/etc下,配置文件见附件my.cnf,两个节点只需要修改其中的server-id为不一样即可

     

    7.#初始化mysql

    /usr/local/mysql/scripts/mysql_install_db --defaults-file=/data/mysql/3306/etc/my.cnf

    8.#启动mysql   

    mysqld_safe --defaults-file=/data/mysql/3306/etc/my.cnf&

    9.#安装mha

    yum -y install perl-DBD-MySQL  perl-Config-Tiny perl-Log-Dispatch perl-Parallel-ForkManager perl
    
    yum localinstall mha4mysql-manager-0.56-0.el6.noarch.rpm mha4mysql-node-0.56-0.el6.noarch.rpm

    10.#配置文件,注意由于我的主从复制启用了gtid,所以binlog server必须配置,如果未启用gtid那么可以不必配置

    /etc/masterha_default.cnf

    [server default]
    
    user=mha
    
    password=hellomha
    
    ssh_user=mha
    
    ping_interval=3
    
    ping_type=INSERT
    
    log_level=debug

    /etc/app1.cnf

    [server default]
    
    master_binlog_dir= /data/mysql/3306/logs
    
    remote_workdir=/var/log/masterha/app1
    
    manager_workdir=/var/log/masterha/app1
    
    manager_log=/var/log/masterha/app1/app1.log
    
    master_ip_failover_script=/bin/master_ip_failover
    
    [server1]
    
    hostname=10.80.8.89
    
    ip=10.80.8.89
    
    port=3306
    
    master_binlog_dir=/data/mysql/3306/logs
    
    candidate_master=1
    
    #check_repl_delay=0
    
    
    [server2]
    
    hostname=10.80.8.90
    
    ip=10.80.8.90
    
    port=3306
    
    master_binlog_dir=/data/mysql/3306/logs
    
    candidate_master=1
    
    [binlog1]
    
    hostname=10.80.8.90
    
    ip=10.80.8.90
    
    master_binlog_dir=/data/binlog

    11.#创建manager所需目录

    mkdir -p /var/log/masterha/app1
    
    chown -R mha:mha /var/log/masterha/app1

    12.#安装并配置atlas

    wget https://github.com/Qihoo360/Atlas/releases/download/2.2.1/Atlas-2.2.1.el6.x86_64.rpm
    
    rpm -ivh Atlas-2.2.1.el6.x86_64.rpm

    13.#启动atlas

    /usr/local/mysql-proxy/bin/mysql-proxyd test start

    14.#配置atlas实例,由于本实验主要是配合mha在atlas上线下线mysql master,所以其他参数默认

    mysql -h 10.80.8.89 -P2345 -uuser -ppwd -e "add master 10.80.8.89:3306;"
    
    mysql -h 10.80.8.89 -P2345 -uuser -ppwd -e "add slave 10.80.8.90:3306;"

    15.#创建mha用户和复制用户

    grant all privileges on *.* to 'mha'@'%' identified by 'hellomha';
    
    grant replication slave on *.* to 'repl'@'%' identified by 'hellorepl';

    16. #更改自动切换atlas里的master    /bin/master_ip_failover

    use strict;
    use warnings FATAL => 'all';
    
    use Getopt::Long;
    use MHA::DBHelper;
    
    my $proxy_ip_1 = "10.80.8.89";
    
    my (
    $command, $ssh_user, $orig_master_host,
    $orig_master_ip, $orig_master_port, $new_master_host,
    $new_master_ip, $new_master_port, $new_master_user,
    $new_master_password
    );
    
    GetOptions(
    'command=s' => $command,
    'ssh_user=s' => $ssh_user,
    'orig_master_host=s' => $orig_master_host,
    'orig_master_ip=s' => $orig_master_ip,
    'orig_master_port=i' => $orig_master_port,
    'new_master_host=s' => $new_master_host,
    'new_master_ip=s' => $new_master_ip,
    'new_master_port=i' => $new_master_port,
    'new_master_user=s' => $new_master_user,
    'new_master_password=s' => $new_master_password,
    );
    
    sub add_vip {
    my $output1 = `/usr/local/mysql/bin/mysql -h $proxy_ip_1 -P 2345 -uuser -ppwd -e "remove backend 1;"` ; 
    my $output2 = `/usr/local/mysql/bin/mysql -h $proxy_ip_1 -P 2345 -uuser -ppwd -e "remove backend 1;"` ;
    my $output3 = `/usr/local/mysql/bin/mysql -h $proxy_ip_1 -P 2345 -uuser -ppwd -e "save config;"` ;
    my $output4 = `/usr/local/mysql/bin/mysql -h $proxy_ip_1 -P 2345 -uuser -ppwd -e "add master $new_master_host:3306;"` ;
    my $output5 = `/usr/local/mysql/bin/mysql -h $proxy_ip_1 -P 2345 -uuser -ppwd -e "save config;"` ;
    # my $output1 = `ssh -o ConnectTimeout=15 -o ConnectionAttempts=3 $orig_master_host /sbin/ip addr del $vip/32 dev $if`;
    # my $output2 = `ssh -o ConnectTimeout=15 -o ConnectionAttempts=3 $new_master_host /sbin/ip addr add $vip/32 dev $if`;
    
    }
    exit &main();
    
    sub main {
    if ( $command eq "stop" || $command eq "stopssh" ) {
    
    # $orig_master_host, $orig_master_ip, $orig_master_port are passed.
    # If you manage master ip address at global catalog database,
    # invalidate orig_master_ip here.
    my $exit_code = 1;
    eval {
    
    # updating global catalog, etc
    $exit_code = 0;
    };
    if ($@) {
    warn "Got Error: $@
    ";
    exit $exit_code;
    }
    exit $exit_code;
    }
    elsif ( $command eq "start" ) {
    
    # all arguments are passed.
    # If you manage master ip address at global catalog database,
    # activate new_master_ip here.
    # You can also grant write access (create user, set read_only=0, etc) here.
    my $exit_code = 10;
    eval {
    my $new_master_handler = new MHA::DBHelper();
    
    # args: hostname, port, user, password, raise_error_or_not
    $new_master_handler->connect( $new_master_ip, $new_master_port,
    $new_master_user, $new_master_password, 1 );
    
    ## Set read_only=0 on the new master
    $new_master_handler->disable_log_bin_local();
    print "Set read_only=0 on the new master.
    ";
    $new_master_handler->disable_read_only();
    
    ## Creating an app user on the new master
    #print "Creating app user on the new master..
    ";
    #FIXME_xxx_create_user( $new_master_handler->{dbh} );
    $new_master_handler->enable_log_bin_local();
    $new_master_handler->disconnect();
    
    ## Update master ip on the catalog database, etc
    &add_vip();
    $exit_code = 0;
    };
    if ($@) {
    warn $@;
    
    # If you want to continue failover, exit 10.
    exit $exit_code;
    }
    exit $exit_code;
    }
    elsif ( $command eq "status" ) {
    
    # do nothing
    exit 0;
    }
    else {
    &usage();
    exit 1;
    }
    }
    
    sub usage {
    print
    "Usage: master_ip_failover --command=start|stop|stopssh|status --orig_master_host=host --orig_master_ip=ip --orig_master_port=port --new_master_host=host --new_master_ip=ip --new_master_port=port
    ";
    }

    10.80.8.90操作命令

    1-9步参见89操作命令

    10.#创建manager所需目录

    mkdir -p /var/log/masterha/app1
    
    chown -R mha:mha /var/log/masterha/app1

    11.#与89建立主从复制关系

    mysql>CHANGE MASTER TO
    
    MASTER_HOST="10.80.8.89",
    MASTER_USER="repl",
    MASTER_PASSWORD="hellomrepl",
    MASTER_AUTO_POSITION = 1;
    
    mysql>start slave;
    
    mysql>set global read_only=on;

    12.建立binlog server

    mkdir -p /data/binlog
    
    chown -R mha:mha /data/binlog
    
    su - mha
    
    /usr/local/mysql/bin/mysqlbinlog -R --raw --host=10.80.8.89 --user=repl --port=3306 --password=hellorepl --stop-never -t -r /data/binlog mysql_bin.000001 &

    10.80.8.89操作命令

    1.#测试mha ssh

    su - mha
    
    [mha@iZ250pd1qtuZ ~]$ masterha_check_ssh --conf=/etc/app1.cnf
    Wed May 18 11:28:54 2016 - [info] Reading default configuration from /etc/masterha_default.cnf..
    Wed May 18 11:28:54 2016 - [info] Reading application default configuration from /etc/app1.cnf..
    Wed May 18 11:28:54 2016 - [info] Reading server configuration from /etc/app1.cnf..
    Wed May 18 11:28:54 2016 - [info] Starting SSH connection tests..
    Wed May 18 11:28:55 2016 - [debug] 
    Wed May 18 11:28:54 2016 - [debug] Connecting via SSH from mha@10.80.8.89(10.80.8.89:22) to mha@10.80.8.90(10.80.8.90:22)..
    Wed May 18 11:28:55 2016 - [debug] ok.
    Wed May 18 11:28:55 2016 - [debug] 
    Wed May 18 11:28:55 2016 - [debug] Connecting via SSH from mha@10.80.8.90(10.80.8.90:22) to mha@10.80.8.89(10.80.8.89:22)..
    Wed May 18 11:28:55 2016 - [debug] ok.
    Wed May 18 11:28:55 2016 - [info] All SSH connection tests passed successfully.

    2.#测试mha repl

    [mha@iZ250pd1qtuZ ~]$ masterha_check_repl --conf=/etc/app1.cnf
    Wed May 18 11:32:38 2016 - [info] Reading default configuration from /etc/masterha_default.cnf..
    Wed May 18 11:32:38 2016 - [info] Reading application default configuration from /etc/app1.cnf..
    Wed May 18 11:32:38 2016 - [info] Reading server configuration from /etc/app1.cnf..
    Wed May 18 11:32:38 2016 - [info] MHA::MasterMonitor version 0.56.
    Wed May 18 11:32:39 2016 - [debug] Connecting to servers..
    Wed May 18 11:32:39 2016 - [debug] Connected to: 10.110.18.89(10.110.18.89:3306), user=mha
    Wed May 18 11:32:39 2016 - [debug] Number of slave worker threads on host 10.110.18.89(10.110.18.89:3306): 0
    Wed May 18 11:32:39 2016 - [debug] Connected to: 10.110.18.90(10.110.18.90:3306), user=mha
    Wed May 18 11:32:39 2016 - [debug] Number of slave worker threads on host 10.110.18.90(10.110.18.90:3306): 0
    Wed May 18 11:32:39 2016 - [debug] Comparing MySQL versions..
    Wed May 18 11:32:39 2016 - [debug] Comparing MySQL versions done.
    Wed May 18 11:32:39 2016 - [debug] Connecting to servers done.
    Wed May 18 11:32:39 2016 - [info] GTID failover mode = 1
    Wed May 18 11:32:39 2016 - [info] Dead Servers:
    Wed May 18 11:32:39 2016 - [info] Alive Servers:
    Wed May 18 11:32:39 2016 - [info] 10.110.18.89(10.110.18.89:3306)
    Wed May 18 11:32:39 2016 - [info] 10.110.18.90(10.110.18.90:3306)
    Wed May 18 11:32:39 2016 - [info] Alive Slaves:
    Wed May 18 11:32:39 2016 - [info] 10.110.18.90(10.110.18.90:3306) Version=5.6.29-76.2-log (oldest major version between slaves) log-bin:enabled
    Wed May 18 11:32:39 2016 - [info] GTID ON
    Wed May 18 11:32:39 2016 - [debug] Relay log info repository: TABLE
    Wed May 18 11:32:39 2016 - [info] Replicating from 10.110.18.89(10.110.18.89:3306)
    Wed May 18 11:32:39 2016 - [info] Primary candidate for the new Master (candidate_master is set)
    Wed May 18 11:32:39 2016 - [info] Current Alive Master: 10.110.18.89(10.110.18.89:3306)
    Wed May 18 11:32:39 2016 - [info] Checking slave configurations..
    Wed May 18 11:32:39 2016 - [info] read_only=1 is not set on slave 10.110.18.90(10.110.18.90:3306).
    Wed May 18 11:32:39 2016 - [info] Checking replication filtering settings..
    Wed May 18 11:32:39 2016 - [info] binlog_do_db= , binlog_ignore_db= information_schema,monitor,performance_schema
    Wed May 18 11:32:39 2016 - [info] Replication filtering check ok.
    Wed May 18 11:32:39 2016 - [info] GTID (with auto-pos) is supported. Skipping all SSH and Node package checking.
    Wed May 18 11:32:39 2016 - [debug] SSH connection test to 10.110.18.90, option -o StrictHostKeyChecking=no -o PasswordAuthentication=no -o BatchMode=yes -o ConnectTimeout=5, timeout 5
    Wed May 18 11:32:39 2016 - [info] HealthCheck: SSH to 10.110.18.90 is reachable.
    Wed May 18 11:32:39 2016 - [info] Binlog server 10.110.18.90 is reachable.
    Wed May 18 11:32:39 2016 - [info] Checking recovery script configurations on 10.110.18.90(10.110.18.90:3306)..
    Wed May 18 11:32:39 2016 - [info] Executing command: save_binary_logs --command=test --start_pos=4 --binlog_dir=/data/binlog --output_file=/var/log/masterha/app1/save_binary_logs_test --manager_version=0.56 --start_file=mysql_bin.000006 --debug 
    Wed May 18 11:32:39 2016 - [info] Connecting to mha@10.110.18.90(10.110.18.90:22).. 
    Creating /var/log/masterha/app1 if not exists.. ok.
    Checking output directory is accessible or not..
    ok.
    Binlog found at /data/binlog, up to mysql_bin.000006
    Wed May 18 11:32:39 2016 - [info] Binlog setting check done.
    Wed May 18 11:32:39 2016 - [info] Checking SSH publickey authentication settings on the current master..
    Wed May 18 11:32:39 2016 - [debug] SSH connection test to 10.110.18.89, option -o StrictHostKeyChecking=no -o PasswordAuthentication=no -o BatchMode=yes -o ConnectTimeout=5, timeout 5
    Wed May 18 11:32:39 2016 - [info] HealthCheck: SSH to 10.110.18.89 is reachable.
    Wed May 18 11:32:39 2016 - [info] 
    10.110.18.89(10.110.18.89:3306) (current master)
    +--10.110.18.90(10.110.18.90:3306)
    
    Wed May 18 11:32:39 2016 - [info] Checking replication health on 10.110.18.90..
    Wed May 18 11:32:39 2016 - [info] ok.
    Wed May 18 11:32:39 2016 - [info] Checking master_ip_failover_script status:
    Wed May 18 11:32:39 2016 - [info] /bin/master_ip_failover --command=status --ssh_user=mha --orig_master_host=10.110.18.89 --orig_master_ip=10.110.18.89 --orig_master_port=3306 
    Wed May 18 11:32:39 2016 - [info] OK.
    Wed May 18 11:32:39 2016 - [warning] shutdown_script is not defined.
    Wed May 18 11:32:39 2016 - [debug] Disconnected from 10.110.18.89(10.110.18.89:3306)
    Wed May 18 11:32:39 2016 - [debug] Disconnected from 10.110.18.90(10.110.18.90:3306)
    Wed May 18 11:32:39 2016 - [info] Got exit code 0 (Not master dead).
    
    MySQL Replication Health is OK.

    3.#启动masterha_manager

    nohup masterha_manager --conf=/etc/app1.cnf 

    实验阶段

    实验一.关闭master,观察从库有没有切成主库,观察atlas里的backend是不是已经下线了旧主并且上线了新主

    1.#查看atlas中的backends信息

    [mha@iZ250pd1qtuZ ~]$ mysql -h 10.80.8.89 -P2345 -uuser -p -e "select * from backends;"
    Enter password: 
    +-------------+-------------------+-------+------+
    | backend_ndx | address | state | type |
    +-------------+-------------------+-------+------+
    | 1 | 10.80.8.89:3306 | up | rw |
    
    | 2 | 10.80.8.90:3306 | up | ro |
    +-------------+-------------------+-------+------+

    2.#关闭master

    mysqladmin -uroot -p shutdown

    3.#查看atlas中的从库信息

    [mha@iZ250pd1qtuZ ~]$ mysql -h 10.80.8.89 -P2345 -uuser -p -e "select * from backends;"
    Enter password: 
    +-------------+-------------------+-------+------+
    | backend_ndx | address | state | type |
    +-------------+-------------------+-------+------+
    
    | 1 | 10.80.8.90:3306 | up | ro |
    +-------------+-------------------+-------+------+

    4.#查看mha切换日志

    ----- Failover Report -----
    
    app1: MySQL Master failover 10.80.8.89(10.80.8.89:3306) to 10.80.8.90(10.80.8.90:3306) succeeded
    
    Master 10.80.8.89(10.80.8.89:3306) is down!
    
    Check MHA Manager logs at iZ250pd1qtuZ:/var/log/masterha/app1/app1.log for details.
    
    Started automated(non-interactive) failover.
    Invalidated master IP address on 10.80.8.89(10.80.8.89:3306)
    Selected 10.80.8.90(10.80.8.90:3306) as a new master.
    10.80.8.90(10.80.8.90:3306): OK: Applying all logs succeeded.
    10.80.8.90(10.80.8.90:3306): OK: Activated master IP address.
    10.80.8.90(10.80.8.90:3306): Resetting slave info succeeded.
    Master failover to 10.80.8.90(10.80.8.90:3306) completed successfully.

    5.#将下线的主作为从挂到新主

    mysqld_safe --defaults-file=/data/mysql/3306/etc/my.cnf &
    mysql -uroot -p
    mysql>CHANGE MASTER TO
                 MASTER_HOST="10.80.8.90",
                 MASTER_USER="repl",
                 MASTER_PASSWORD="hellorepl",
                 MASTER_AUTO_POSITION = 1;  
    mysql>start slave;
    mysql>set global read_only=on;

    6.#启动binlog server,指向新master90

    su - mha
    
    /usr/local/mysql/bin/mysqlbinlog -R --raw --host=10.80.8.90 --user=repl --port=3306 --password=hellorepl --stop-never -t -r /data/binlog mysql_bin.000001 &

    7.#清理mha目录下所有内容,发生切换后,状态文件还存在,会影响下次的切换,如果不删除日志,那么mha在8小时内如果再次发生宕机,那么不会发生自动切换

    rm -rf /var/log/masterha/app1/*

    8.#启动mha

    nohup masterha_manager --conf=/etc/app1.cnf &

    实验二.从库关闭slave 的io thread线程,主库写入数据,此时kill -9 mysql实例,观察主库写入的数据有没有正确被mha补全到新主

    1.#从停止slave io_thread

    mysql> stop slave io_thread;
    Query OK, 0 rows affected (0.00 sec)

    mysql> show databases;
    +--------------------+
    | Database |
    +--------------------+
    | information_schema |
    | gaoquan |
    | infra |
    | mysql |
    | performance_schema |
    | test |
    +--------------------+
    6 rows in set (0.00 sec)

    2.#主创建数据库

    mysql> create database sbtest;
    Query OK, 1 row affected (0.00 sec)
    
    mysql> show databases;
    +--------------------+
    | Database           |
    +--------------------+
    | information_schema |
    | gaoquan            |
    | infra              |
    | mysql              |
    | performance_schema |
    | sbtest             |
    | test               |
    +--------------------+
    7 rows in set (0.01 sec)

    3.#从查看数据库

    mysql> show databases;
    +--------------------+
    | Database           |
    +--------------------+
    | information_schema |
    | gaoquan            |
    | infra              |
    | mysql              |
    | performance_schema |
    | test               |
    +--------------------+
    6 rows in set (0.00 sec)

    4.#杀掉主的mysql进程,mysqld_safe和mysqld进程一起干掉

    kill -9 27229 26253

    5.#从查看数据库中的sbtest是否已经正常拉取过来

    mysql> show databases;
    +--------------------+
    | Database           |
    +--------------------+
    | information_schema |
    | gaoquan            |
    | infra              |
    | mysql              |
    | performance_schema |
    | sbtest             |
    | test               |
    +--------------------+
    7 rows in set (0.00 sec)

    实验三.交互式切换主从数据库,适用于升级Master,主从都存活,但是需要升级内存或者更换控制器等必须重启服务器时。(注意,master_ip_online_change脚本是在切换到新主之前执行的,因此从库必须制定read_only)

    1.#确保masterha_manager不存活,如果存活则停止

    masterha_stop --conf=/etc/app1.cnf

    2.#将主由90切换到89

    [mha@iZ250pd1qtuZ ~]$ masterha_master_switch --master_state=alive --conf=/etc/app1.cnf --new_master_host=10.80.8.89 --new_master_port=3306
    Wed May 18 20:07:17 2016 - [info] MHA::MasterRotate version 0.56.
    Wed May 18 20:07:17 2016 - [info] Starting online master switch..
    Wed May 18 20:07:17 2016 - [info]
    Wed May 18 20:07:17 2016 - [info] * Phase 1: Configuration Check Phase..
    Wed May 18 20:07:17 2016 - [info]
    Wed May 18 20:07:17 2016 - [info] Reading default configuration from /etc/masterha_default.cnf..
    Wed May 18 20:07:17 2016 - [info] Reading application default configuration from /etc/app1.cnf..
    Wed May 18 20:07:17 2016 - [info] Reading server configuration from /etc/app1.cnf..
    Wed May 18 20:07:17 2016 - [debug] Connecting to servers..
    Wed May 18 20:07:17 2016 - [debug] Connected to: 10.80.8.89(10.80.8.89:3306), user=mha
    Wed May 18 20:07:17 2016 - [debug] Number of slave worker threads on host 10.80.8.89(10.80.8.89:3306): 0
    Wed May 18 20:07:17 2016 - [debug] Connected to: 10.80.8.90(10.80.8.90:3306), user=mha
    Wed May 18 20:07:17 2016 - [debug] Number of slave worker threads on host 10.80.8.90(10.80.8.90:3306): 0
    Wed May 18 20:07:17 2016 - [debug] Comparing MySQL versions..
    Wed May 18 20:07:17 2016 - [debug] Comparing MySQL versions done.
    Wed May 18 20:07:17 2016 - [debug] Connecting to servers done.
    Wed May 18 20:07:17 2016 - [info] GTID failover mode = 1
    Wed May 18 20:07:17 2016 - [info] Current Alive Master: 10.80.8.90(10.80.8.90:3306)
    Wed May 18 20:07:17 2016 - [info] Alive Slaves:
    Wed May 18 20:07:17 2016 - [info] 10.80.8.89(10.80.8.89:3306) Version=5.6.29-76.2-log (oldest major version between slaves) log-bin:enabled
    Wed May 18 20:07:17 2016 - [info] GTID ON
    Wed May 18 20:07:17 2016 - [debug] Relay log info repository: TABLE
    Wed May 18 20:07:17 2016 - [info] Replicating from 10.80.8.90(10.80.8.90:3306)
    Wed May 18 20:07:17 2016 - [info] Primary candidate for the new Master (candidate_master is set)

    It is better to execute FLUSH NO_WRITE_TO_BINLOG TABLES on the master before switching. Is it ok to execute on 10.80.8.90(10.80.8.90:3306)? (YES/no): yes
    Wed May 18 20:07:20 2016 - [info] Executing FLUSH NO_WRITE_TO_BINLOG TABLES. This may take long time..
    Wed May 18 20:07:20 2016 - [info] ok.
    Wed May 18 20:07:20 2016 - [info] Checking MHA is not monitoring or doing failover..
    Wed May 18 20:07:20 2016 - [info] Checking replication health on 10.80.8.89..
    Wed May 18 20:07:20 2016 - [info] ok.
    Wed May 18 20:07:20 2016 - [info] 10.80.8.89 can be new master.
    Wed May 18 20:07:20 2016 - [info]
    From:
    10.80.8.90(10.80.8.90:3306) (current master)
    +--10.80.8.89(10.80.8.89:3306)

    To:
    10.80.8.89(10.80.8.89:3306) (new master)

    Starting master switch from 10.80.8.90(10.80.8.90:3306) to 10.80.8.89(10.80.8.89:3306)? (yes/NO): yes
    Wed May 18 20:07:21 2016 - [info] Checking whether 10.80.8.89(10.80.8.89:3306) is ok for the new master..
    Wed May 18 20:07:21 2016 - [info] ok.
    Wed May 18 20:07:21 2016 - [info] ** Phase 1: Configuration Check Phase completed.
    Wed May 18 20:07:21 2016 - [info]
    Wed May 18 20:07:21 2016 - [debug] Disconnected from 10.80.8.90(10.80.8.90:3306)
    Wed May 18 20:07:21 2016 - [info] * Phase 2: Rejecting updates Phase..
    Wed May 18 20:07:21 2016 - [info]
    Wed May 18 20:07:21 2016 - [info] Executing master ip online change script to disable write on the current master:
    Wed May 18 20:07:21 2016 - [info] /bin/master_ip_online_change --command=stop --orig_master_host=10.80.8.90 --orig_master_ip=10.80.8.90 --orig_master_port=3306 --orig_master_user='mha' --orig_master_password='yz6xFOkF1mh3Wmkr1Rsz' --new_master_host=10.80.8.89 --new_master_ip=10.80.8.89 --new_master_port=3306 --new_master_user='mha' --new_master_password='yz6xFOkF1mh3Wmkr1Rsz' --orig_master_ssh_user=mha --new_master_ssh_user=mha
    Wed May 18 20:07:21 2016 556919 Set read_only on the new master.. ok.
    Wed May 18 20:07:21 2016 559821 drop vip 192.168.1.100..
    Warning: Using a password on the command line interface can be insecure.
    ERROR 1105 (07000) at line 1: invalid backend_id
    Warning: Using a password on the command line interface can be insecure.
    ERROR 1105 (07000) at line 1: invalid backend_id
    Warning: Using a password on the command line interface can be insecure.
    Wed May 18 20:07:21 2016 582940 Waiting all running 1 threads are disconnected.. (max 1500 milliseconds)
    {'Time' => '2598','db' => undef,'Id' => '51','User' => 'mha','State' => 'Master has sent all binlog to slave; waiting for binlog to be updated','Command' => 'Binlog Dump GTID','Rows_examined' => '0','Info' => undef,'Rows_sent' => '0','Host' => '10.80.8.89:11427'}
    Wed May 18 20:07:22 2016 083378 Waiting all running 1 threads are disconnected.. (max 1000 milliseconds)
    {'Time' => '2599','db' => undef,'Id' => '51','User' => 'mha','State' => 'Master has sent all binlog to slave; waiting for binlog to be updated','Command' => 'Binlog Dump GTID','Rows_examined' => '0','Info' => undef,'Rows_sent' => '0','Host' => '10.80.8.89:11427'}
    Wed May 18 20:07:22 2016 584152 Waiting all running 1 threads are disconnected.. (max 500 milliseconds)
    {'Time' => '2599','db' => undef,'Id' => '51','User' => 'mha','State' => 'Master has sent all binlog to slave; waiting for binlog to be updated','Command' => 'Binlog Dump GTID','Rows_examined' => '0','Info' => undef,'Rows_sent' => '0','Host' => '10.80.8.89:11427'}
    Wed May 18 20:07:23 2016 084569 Set read_only=1 on the orig master.. ok.
    Wed May 18 20:07:23 2016 085676 Waiting all running 1 queries are disconnected.. (max 500 milliseconds)
    {'Time' => '2600','db' => undef,'Id' => '51','User' => 'mha','State' => 'Master has sent all binlog to slave; waiting for binlog to be updated','Command' => 'Binlog Dump GTID','Rows_examined' => '0','Info' => undef,'Rows_sent' => '0','Host' => '10.80.8.89:11427'}
    Wed May 18 20:07:23 2016 585008 Killing all application threads..
    Wed May 18 20:07:23 2016 585529 done.
    Wed May 18 20:07:23 2016 - [info] ok.
    Wed May 18 20:07:23 2016 - [info] Locking all tables on the orig master to reject updates from everybody (including root):
    Wed May 18 20:07:23 2016 - [info] Executing FLUSH TABLES WITH READ LOCK..
    Wed May 18 20:07:23 2016 - [info] ok.
    Wed May 18 20:07:23 2016 - [info] Orig master binlog:pos is mysql_bin.000007:193297.
    Wed May 18 20:07:23 2016 - [debug] Fetching current slave status..
    Wed May 18 20:07:23 2016 - [debug] Fetching current slave status done.
    Wed May 18 20:07:23 2016 - [info] Waiting to execute all relay logs on 10.80.8.89(10.80.8.89:3306)..
    Wed May 18 20:07:23 2016 - [info] master_pos_wait(mysql_bin.000007:193297) completed on 10.80.8.89(10.80.8.89:3306). Executed 0 events.
    Wed May 18 20:07:23 2016 - [info] done.
    Wed May 18 20:07:23 2016 - [debug] Stopping SQL thread on 10.80.8.89(10.80.8.89:3306)..
    Wed May 18 20:07:23 2016 - [debug] done.
    Wed May 18 20:07:23 2016 - [info] Getting new master's binlog name and position..
    Wed May 18 20:07:23 2016 - [info] mysql_bin.000007:202995
    Wed May 18 20:07:23 2016 - [info] All other slaves should start replication from here. Statement should be: CHANGE MASTER TO MASTER_HOST='10.80.8.89', MASTER_PORT=3306, MASTER_AUTO_POSITION=1, MASTER_USER='mha', MASTER_PASSWORD='xxx';
    Wed May 18 20:07:23 2016 - [info] Executing master ip online change script to allow write on the new master:
    Wed May 18 20:07:23 2016 - [info] /bin/master_ip_online_change --command=start --orig_master_host=10.80.8.90 --orig_master_ip=10.80.8.90 --orig_master_port=3306 --orig_master_user='mha' --orig_master_password='yz6xFOkF1mh3Wmkr1Rsz' --new_master_host=10.80.8.89 --new_master_ip=10.80.8.89 --new_master_port=3306 --new_master_user='mha' --new_master_password='yz6xFOkF1mh3Wmkr1Rsz' --orig_master_ssh_user=mha --new_master_ssh_user=mha
    Wed May 18 20:07:23 2016 695963 Set read_only=0 on the new master.
    Wed May 18 20:07:23 2016 696375Add vip 192.168.1.100 on eth0..
    Warning: Using a password on the command line interface can be insecure.
    Warning: Using a password on the command line interface can be insecure.
    Wed May 18 20:07:23 2016 - [info] ok.
    Wed May 18 20:07:23 2016 - [info]
    Wed May 18 20:07:23 2016 - [info] * Switching slaves in parallel..
    Wed May 18 20:07:23 2016 - [info]
    Wed May 18 20:07:23 2016 - [info] Unlocking all tables on the orig master:
    Wed May 18 20:07:23 2016 - [info] Executing UNLOCK TABLES..
    Wed May 18 20:07:23 2016 - [info] ok.
    Wed May 18 20:07:23 2016 - [info] All new slave servers switched successfully.
    Wed May 18 20:07:23 2016 - [info]
    Wed May 18 20:07:23 2016 - [info] * Phase 5: New master cleanup phase..
    Wed May 18 20:07:23 2016 - [info]
    Wed May 18 20:07:23 2016 - [debug] Clearing slave info..
    Wed May 18 20:07:23 2016 - [debug] Stopping slave IO/SQL thread on 10.80.8.89(10.80.8.89:3306)..
    Wed May 18 20:07:23 2016 - [debug] done.
    Wed May 18 20:07:23 2016 - [debug] SHOW SLAVE STATUS shows new master does not replicate from anywhere. OK.
    Wed May 18 20:07:23 2016 - [info] 10.80.8.89: Resetting slave info succeeded.
    Wed May 18 20:07:23 2016 - [info] Switching master to 10.80.8.89(10.80.8.89:3306) completed successfully.
    Wed May 18 20:07:23 2016 - [debug] Disconnected from 10.80.8.89(10.80.8.89:3306)
    Wed May 18 20:07:23 2016 - [debug] Disconnected from 10.80.8.90(10.80.8.90:3306)
    [mha@iZ250pd1qtuZ ~]$ mysql -h10.80.8.89 -uuser -ppwd -P2345 -e "select * from backends;";
    Warning: Using a password on the command line interface can be insecure.
    +-------------+-------------------+-------+------+
    | backend_ndx | address | state | type |
    +-------------+-------------------+-------+------+
    | 1 | 10.80.8.89:3306 | up | rw |
    +-------------+-------------------+-------+------+

    实验四:master挂掉,手动进行切换

    [mha@iZ250pd1qtuZ ~]$ mysqladmin -uroot -p shutdown
    Enter password: 
    160518 20:19:18 mysqld_safe mysqld from pid file /data/mysql/3306/logs/mysqld.3306.pid ended
    [mha@iZ250pd1qtuZ ~]$ masterha_master_switch --master_state=dead --conf=/etc/conf/masterha/app1.cnf --dead_master_host=10.80.8.89 --new_master_host=10.80.8.90 --interactive=0
    --dead_master_ip=<dead_master_ip> is not set. Using 10.80.8.89.
    --dead_master_port=<dead_master_port> is not set. Using 3306.
    Wed May 18 20:19:26 2016 - [info] Reading default configuration from /etc/masterha_default.cnf..
    Wed May 18 20:19:26 2016 - [error][/usr/share/perl5/vendor_perl/MHA/ManagerUtil.pm, ln177] Got ERROR: /etc/conf/masterha/app1.cnf:No such file or directory
     at /usr/share/perl5/vendor_perl/MHA/MasterFailover.pm line 104.
    [mha@iZ250pd1qtuZ ~]$ masterha_master_switch --master_state=dead --conf=/etc/app1.cnf --dead_master_host=10.80.8.89 --new_master_host=10.80.8.90 --interactive=0
    --dead_master_ip=<dead_master_ip> is not set. Using 10.80.8.89.
    --dead_master_port=<dead_master_port> is not set. Using 3306.
    Wed May 18 20:19:39 2016 - [info] Reading default configuration from /etc/masterha_default.cnf..
    Wed May 18 20:19:39 2016 - [info] Reading application default configuration from /etc/app1.cnf..
    Wed May 18 20:19:39 2016 - [info] Reading server configuration from /etc/app1.cnf..
    [mha@iZ250pd1qtuZ ~]$ mysql -h 10.80.8.89 -uuser -ppwd -P2345 -e "select * from backends";
    Warning: Using a password on the command line interface can be insecure.
    +-------------+-------------------+-------+------+
    | backend_ndx | address           | state | type |
    +-------------+-------------------+-------+------+
    |           1 | 10.80.8.90:3306 | up    | rw   |
    +-------------+-------------------+-------+------+

    总结:

    1.由于用的是阿里云,因此没法模拟主机直接掉电关闭的情况

    2.mha很重要的一个功能是补全slave的差异并与新主建立主从关系,由于我们的环境并发及重要程度不是特别大,一主挂多从的成本过高,因此一主一从已经足够,如果是业务相对比较重要,那么还是建议一主多从。

    3.secondary_check_script没有开启,有需要的可以开启试下,即通过多个节点去访问master,防止由于网络抖动而误判master异常而导致master切换

  • 相关阅读:
    OpenCascade Ray Tracing Rendering
    Create New Commands in Tcl
    OpenCascade Modeling Algorithms Fillets and Chamfers
    OpenCascade Modeling Algorithms Boolean Operations
    Construction of Primitives in Open Cascade
    Open Cascade Data Exchange STL
    Tcl Tk Introduction
    Open Cascade DataExchange IGES
    Netgen mesh library : nglib
    Hello Netgen
  • 原文地址:https://www.cnblogs.com/gaoquan/p/5503232.html
Copyright © 2011-2022 走看看