zoukankan      html  css  js  c++  java
  • MySQL Flashback 工具介绍

    MySQL Flashback 工具介绍

    • DML Flashback

      • 独立工具,通过伪装成slave拉取binlog来进行处理
      • patch形式集成到官方工具mysqlbinlog
      • 简单脚本。先用mysqlbinlog解析出文本格式的binlog,再根据回滚原理用正则进行匹配并替换
    • DDL Flashback

      DDL Falshback 要求离线,比 DML Flashback 要求更多,生产环境数据可能覆盖导致恢复不全

      因为 5.6 开始,有单独的 purge 线程,数据恢复可能性更低了

    部署位置

    • IP1

      /data1/dbatools/mysqlflashback

      • flashback_57 「mysqlbinlog」(因为系统原因,mysqlbinlog 放在 CentOS 7机器上)
      • binlog2sql 「建议使用」
      • MyFlash「建议使用」(测试暂时还不能在 CentOS 7上使用)
    • IP2

      /data1/dbatools/mysqlflashback

      • binlog2sql 「建议使用」
      • MyFlash「建议使用」

    MyFlash

    MyFlash——美团点评的开源MySQL闪回工具

    github:MyFlash

    MyFlash是由美团点评公司技术工程部开发维护的一个回滚DML操作的工具。该工具通过解析v4版本的binlog,完成回滚操作。相对已有的回滚工具,其增加了更多的过滤选项,让回滚更加容易。

    版本兼容

    • MySQL 5.6、5.7

    限制

    • binlog格式必须为row,且binlog_row_image=full
    • 目前只支持单个 binlog 文件解析
    • 只能回滚DML(增、删、改)

    依赖

    • glibc 版本

    编译

    git clone https://github.com/Meituan-Dianping/MyFlash.git
    cd MyFlash
    gcc -w  `pkg-config --cflags --libs glib-2.0` source/binlogParseGlib.c  -o binary/flashback
    

    使用

    基本用法

    cd binary
    
    # 回滚整个文件 && 应用
    ./flashback --binlogFileNames=mysql-bin.000005
    mysqlbinlog binlog_output_base.flashback | mysql -h<host> -u<user> -p
    
    # 回滚文件中 INSERT && 应用
    ./flashback --sqlTypes='INSERT'--binlogFileNames=mysql-bin.000005
    mysqlbinlog binlog_output_base.flashback | mysql -h<host> -u<user> -p
    
    # 回滚大文件 && 应用 (大于20M建议分割回滚文件)
    ./flashback --binlogFileNames=mysql-bin.000005
    ./flashback --maxSplitSize=1 --binlogFileNames=binlog_output_base.flashback
    
    mysqlbinlog binlog_output_base.flashback.000001 | mysql -h<host> -u<user> -p
    ...
    mysqlbinlog binlog_output_base.flashback.<N> | mysql -h<host> -u<user> -p
    
    • flashback 回滚的文件,还是mysqlbinlog格式的,如果想看 sql,还是需要原生 mysqlbinlog 进行解析

      /usr/local/mysql-5.7.18/bin/mysqlbinlog binlog_output_base.flashback

    • 开启 gtid MySQL 应用flashback报错

    ERROR 1782 (HY000) at line 16: @@SESSION.GTID_NEXT cannot be set to ANONYMOUS when @@GLOBAL.GTID_MODE = ON.

    在导入时加入--skip-gtids mysqlbinlog --skip-gtids | mysql -uxxx -pxxx

    选项

    ./flashback --help
    Usage:
      flashback [OPTION...]
    
    Help Options:
      -?, --help                  Show help options
    
    Application Options:
      --databaseNames             databaseName to apply. if multiple, seperate by comma(,)
      --tableNames                tableName to apply. if multiple, seperate by comma(,)
      --start-position            start position
      --stop-position             stop position
      --start-datetime            start time (format %Y-%m-%d %H:%M:%S)
      --stop-datetime             stop time (format %Y-%m-%d %H:%M:%S)
      --sqlTypes                  sql type to filter . support INSERT, UPDATE ,DELETE. if multiple, seperate by comma(,)
      --maxSplitSize              max file size after split, the uint is M
      --binlogFileNames           binlog files to process. if multiple, seperate by comma(,)  
      --outBinlogFileNameBase     output binlog file name base
      --logLevel                  log level, available option is debug,warning,error
      --include-gtids             gtids to process
      --exclude-gtids             gtids to skip
    
     参数解释
    1 databaseNames 指定需要回滚的数据库名。多个用“,”隔开。不指定该参数,相当于指定了所有数据库
    2 tableNames 指定需要回滚的表名。多个表可以用“,”隔开。如果不指定该参数,相当于指定了所有表
    3 start-position 指定回滚开始的位置。如不指定,从文件的开始处回滚。请指定正确的有效的位置,否则无法回滚
    4 stop-position 指定回滚结束的位置。如不指定,回滚到文件结尾。请指定正确的有效的位置,否则无法回滚
    5 start-datetime 指定回滚的开始时间。格式必须是 %Y-%m-%d %H:%M:%S。 如不指定,则不限定时间
    6 stop-datetime 定回滚的结束时间。格式必须是 %Y-%m-%d %H:%M:%S。 如不指定,则不限定时间
    7 sqlTypes 指定回滚的sql类型。支持的类型是INSERT, UPDATE ,DELETE。多个类型可以用“,”隔开。
    8 maxSplitSize 一旦指定该参数,对文件进行固定尺寸的分割(单位为M),过滤条件有效,但不进行回滚操作。该参数主要用来将大的binlog文件切割,防止单次应用的binlog尺寸过大,对线上造成压力
    9 binlogFileNames 需要回滚的binlog文件,目前只支持单个文件,后续会增加多个文件支持
    10 outBinlogFileNameBase 输出的binlog文件前缀,如不指定,则默认为binlog_output_base.flashback
    11 logLevel 仅供开发者使用,默认级别为error级别。在生产环境中不要修改这个级别,否则输出过多
    12 include-gtids 指定需要回滚的gtid,支持gtid的单个和范围两种形式。
    13 exclude-gtids 指定不需要回滚的gtid,用法同include-gtids

    binlog2sql

    伪装成slave拉取binlog,从MySQL binlog解析出你要的SQL。根据不同选项,你可以得到原始SQL、回滚SQL、去除主键的INSERT SQL等,建议表有主键。

    github: binlog2sql

    MySQL闪回原理与实战

    用途

    • 数据快速回滚(闪回)

    • 主从切换后新master丢数据的修复

    • 从binlog生成标准SQL,带来的衍生功能

    版本兼容

    • Python 2.6、2.7
    • MySQL 5.6 (简单测试在 5.5、5.7 gtid 均可用,不排除无 Bug,建议 5.6 以后版本使用)

    依赖

    限制 (对比mysqlbinlog)

    • mysql server必须开启,离线模式下不能解析

    • 参数 binlog_format 必须为ROW,参数 binlog_row_image 必须为FULL,暂不支持MINIMAL (5.6.2引入)

    • GEOMETRY 不能解析 (因为python-mysql-replication Limitations)

    • 解析速度不如 mysqlbinlog

    • 表有关联表,关联表并不会被回滚,需与业务方沟通清楚

    • 不支持DDL

    • 建议回滚的表有主键

    优点 (对比mysqlbinlog)

    • 纯Python开发,安装与使用都很简单
    • 自带flashback、no-primary-key解析模式,无需再装补丁
    • flashback模式下,更适合闪回实战
    • 解析为标准SQL,方便理解、调试
    • 代码容易改造,可以支持更多个性化解析

    安装

    git clone https://github.com/danfengcao/binlog2sql.git && cd binlog2sql
    pip install -r requirements.txt
    

    升级依赖模块

    # cat requirements.txt 
    PyMySQL==0.7.11
    wheel==0.29.0
    mysql-replication==0.13
    
    wget https://codeload.github.com/noplay/python-mysql-replication/tar.gz/0.15
    tar zxf 0.15
    cd python-mysql-replication-0.15
    python2.7 setup.py install
    

    使用

    MySQL server必须设置以下参数

    [mysqld]
    server_id = 1
    log_bin = /var/log/mysql/mysql-bin.log
    max_binlog_size = 1G
    binlog_format = row
    binlog_row_image = full
    

    user需要的最小权限集合

    select, super/replication client, replication slave
    
    建议授权
    GRANT SELECT, REPLICATION SLAVE, REPLICATION CLIENT ON *.* TO 
    

    权限说明

    • select:需要读取server端information_schema.COLUMNS表,获取表结构的元信息,拼接成可视化的sql
    • super/replication client:两个权限都可以,需要执行'SHOW MASTER STATUS', 获取server端的binlog列表
    • replication slave:通过BINLOG_DUMP协议获取binlog内容的权限

    基本用法

    解析标准 SQL (<u>会有 DDL SQL,回滚模式是没有 DDL的</u>)

    python2.7 binlog2sql/binlog2sql.py -h127.0.0.1 -P6000 -umyadmin_common -p'password' -dtest -tt1 --start-file='mysql-bin.000192'
    
    INSERT INTO `test`.`t1`(`id`, `name`) VALUES (5, 'name5'); #start 4 end 259 time 2017-09-14 19:59:31
    DELETE FROM `test`.`t1` WHERE `id`=4 AND `name`='name4' LIMIT 1; #start 286 end 438 time 2017-09-14 19:59:42
    UPDATE `test`.`t1` SET `id`=3, `name`='name300' WHERE `id`=3 AND `name`='name3' LIMIT 1; #start 465 end 631 time 2017-09-14 19:59:54
    

    解析回滚SQL (在上面命令基础上加上 --flashback 或 -B)

    python2.7 binlog2sql/binlog2sql.py -h127.0.0.1 -P6000 -umyadmin_common -p'ZTlh916@mxq' -dtest -tt1 --start-file='mysql-bin.000192' -B
    
    UPDATE `test`.`t1` SET `id`=3, `name`='name3' WHERE `id`=3 AND `name`='name300' LIMIT 1; #start 465 end 631 time 2017-09-14 19:59:54
    INSERT INTO `test`.`t1`(`id`, `name`) VALUES (4, 'name4'); #start 286 end 438 time 2017-09-14 19:59:42
    DELETE FROM `test`.`t1` WHERE `id`=5 AND `name`='name5' LIMIT 1; #start 4 end 259 time 2017-09-14 19:59:31
    

    选项

    mysql连接配置

    -h host; -P port; -u user; -p password

    解析模式

    --stop-never 持续同步binlog。可选。不加则同步至执行命令时最新的binlog位置。

    -K, --no-primary-key 对INSERT语句去除主键。可选。

    -B, --flashback 生成回滚语句,可解析大文件,不受内存限制,每打印一千行加一句SLEEP SELECT(1)。可选。与stop-never或no-primary-key不能同时添加。

    解析范围控制

    --start-file 起始解析文件。必须。

    --start-position/--start-pos start-file的起始解析位置。可选。默认为start-file的起始位置。

    --stop-file/--end-file 末尾解析文件。可选。默认为start-file同一个文件。若解析模式为stop-never,此选项失效。

    --stop-position/--end-pos stop-file的末尾解析位置。可选。默认为stop-file的最末位置;若解析模式为stop-never,此选项失效。

    --start-datetime 从哪个时间点的binlog开始解析,格式必须为datetime,如'2016-11-11 11:11:11'。可选。默认不过滤。

    --stop-datetime 到哪个时间点的binlog停止解析,格式必须为datetime,如'2016-11-11 11:11:11'。可选。默认不过滤。

    对象过滤

    -d, --databases 只输出目标db的sql。可选。默认为空。

    -t, --tables 只输出目标tables的sql。可选。默认为空。

    恢复数据

    误操作大概时间范围

    python binlog2sql/binlog2sql.py -h127.0.0.1 -P3306 -uadmin -p'admin' -dtest -ttbl --start-file='mysql-bin.000052' --start-datetime='2016-12-13 20:25:00' --stop-datetime='2016-12-13 20:30:00'
    

    找到具体位点进行恢复

    python binlog2sql/binlog2sql.py -h127.0.0.1 -P3306 -uadmin -p'admin' -dtest -ttbl --start-file='mysql-bin.000052' --start-position=3346 --stop-position=3556 -B > rollback.sql | cat
    

    <u>回滚前,一定找业务确认回滚数据</u>

    TIPS

    • 闪回的目标:快速筛选出真正需要回滚的数据。

    • 先根据库、表、时间做一次过滤,再根据位置做更准确的过滤。

    • 由于数据一直在写入,要确保回滚sql中不包含其他数据。可根据是否是同一事务、误操作行数、字段值的特征等等来帮助判断。

    • 执行回滚sql时如有报错,需要查实具体原因,一般是因为对应的数据已发生变化。由于是严格的行模式,只要有唯一键(包括主键)存在,就只会报某条数据不存在的错,不必担心会更新不该操作的数据。业务如果有特殊逻辑,数据回滚可能会带来影响。

    • 如果只回滚某张表,并且该表有关联表,关联表并不会被回滚,需与业务方沟通清楚。

    mysqlbinlog_flashback

    跟 binlog2sql 一样,伪装成slave拉取binlog,从MySQL binlog解析出你要的SQL,目前已经稳定运行在阿里RDS、db字符集为 utf8 的产生环境,<u>其他环境要谨慎使用</u>

    github: mysqlbinlog_flashback

    版本兼容

    • 阿里RDS
    • 默认 utf8 DB (gbk等字符集的 DB 理论上支持)

    限制

    • mysql server必须开启,离线模式下不能解析
    • 参数 binlog_format 必须为ROW,参数 binlog_row_image 必须为FULL,暂不支持MINIMAL (5.6.2引入)
    • GEOMETRY 不能解析 (因为python-mysql-replication Limitations)
    • 解析速度不如 mysqlbinlog
    • 表有关联表,关联表并不会被回滚,需与业务方沟通清
    • 不支持DDL
    • <u>表必须有主键</u> (只限制在 mysqlbinlog_flashback,虽然 binlog2sql 不强制,但建议表有主键)
    • <u>阿里 RDS、db 字符集为 utf8 的环境稳定使用,其他环境要谨慎使用</u> (只限制在 mysqlbinlog_flashback)

    • <u>修改 python-mysql-replication</u> (只限制在 mysqlbinlog_flashback)

      • 某些数据被封装,没有暴露出来
      • Ali-rds导致的bug
    • <u>字段类型限制</u> (只限制在 mysqlbinlog_flashback)

      ALLOW_TYPE={ "varchar":True, "char":True, "datetime":True, "date":True, "time":True, "timestamp":True, "bigint":True, "mediumint":True, "smallint":True, "tinyint":True, "int":True, "smallint":True, "decimal":True, "float":True, "double":True, "longtext":True, "tinytext":True, "text":True, "mediumtext":True }

    • mysql 导入一定要指定 字符集 --default-character-set=utf8

    安装

    git clone https://github.com/58daojia-dba/mysqlbinlog_flashback.git
    cd mysqlbinlog_flashback
    

    使用

    MySQL server必须设置以下参数

    [mysqld]
    server_id = 1
    log_bin = /var/log/mysql/mysql-bin.log
    max_binlog_size = 1G
    binlog_format = row
    binlog_row_image = full
    

    基本用法

    python2.7 mysqlbinlog_back.py --host="127.0.0.1" --port=7000 --user="mysql_user" --password='password' --schema=test --table=t2
    
    ls -l log/*
    -rw-r--r-- 1 root root 1.9K Sep 19 17:40 save_data_dml_test_20170919_174001.sql
    -rw-r--r-- 1 root root  179 Sep 19 17:40 save_data_create_table_test_20170919_174001.sql
    -rw-r--r-- 1 root root 1.1K Sep 19 17:40 flashback_test_20170919_174001.sql <--- 反向sq文件
    
    mysql -uroot -p -h -P --default-character-set=utf8 < flashback_test_20170919_174001.sql
    

    选项

    -s SCHEMA, --schema=SCHEMA 必须的,指定 DB

    -t TABLES, --tables=TABLES 必须的,指定 Table,用逗号分隔

    -N BINLOG_END_TIME, --binlog_end_time=BINLOG_END_TIME

    format yyyy-mm-dd hh24:mi:ss,default is current time

    -S BINLOG_START_FILE_NAME, --binlog_start_file_name=BINLOG_START_FILE_NAME

    binlog start file name,default is current logfile of db

    -L BINLOG_START_FILE_POSITION, --binlog_start_file_position=BINLOG_START_FILE_POSITION

    binlog start file name

    -E BINLOG_START_TIME, --binlog_start_time=BINLOG_START_TIME

    binlog start time,format yyyy-mm-dd hh24:mi:ss

    -l OUTPUT_FILE_PATH, --output_file_path=OUTPUT_FILE_PATH

    file path that sql generated,,default ./log

    -I, --skip_insert skip insert(WriteRowsEvent) event
    -U, --skip_update skip update(UpdateRowsEvent) event
    -D, --skip_delete skip delete(DeleteRowsEvent) event

    -a, --add_schema_name add schema name for flashback sql

    mysqlbinlog

    以patch形式集成到官方工具mysqlbinlog中。以彭立勋提交的patch为代表 MySQL下实现闪回的设计思路 (MySQL Flashback Feature)

    mysqlbinlog 闪回工具

    优点

    • 上手成本低,原 mysqlbinlog 选项可以直接利用,只是多一个闪回选项

    • 支持离线解析

    • 支持事务闪回,语句逆转、顺序也颠倒

    缺点

    • 兼容性差、项目活跃度不高,mysql5.5,5.6,5.7 分别开发了patch
    • 难以添加新功能,实战效果欠佳,例如过滤一张表 (姜承尧版本也是支持的)
    • 安装稍显麻烦。需要对mysql源码打补丁再编译生成

    限制

    • 参数 binlog_format 必须为ROW,参数 binlog_row_image 必须为FULL,暂不支持MINIMAL

    • 不支持DDL

    使用

    MySQL server必须设置以下参数

    [mysqld]
    server_id = 1
    log_bin = /var/log/mysql/mysql-bin.log
    max_binlog_size = 1G
    binlog_format = row
    binlog_row_image = full
    

    基本用法

    解析 mysqlbinlog

    ./mysqlbinlog -vv  mysql-bin.000067 --start-datetime="2017-09-19 15:00:00" --stop-datetime="2017-09-19 15:38:10"
    

    回滚

    ./mysqlbinlog -vv  mysql-bin.000067 --start-position=1427 --stop-position=2588 -B | mysql -umysqlha_common -p'password' -h10.13.2.29 -P 7000 test
    

    position 位点找需要回滚事务 BEGIN 前面那个 position.

    选项

    -B, --flashback Flashback data to start_postition or start_datetime.

    -A, --skip-database=name 解析BinLog时过滤掉该数据库 (local log only)

    -a, --skip-table=name 解析BinLog时过滤掉该表,一般与skip_datebase配套使用 (local log only)

    --skip-database 可以单独使用,--skip-table需要与 skip-database配置使用,<u>多个db或table不生效</u>

    --skip-database=test --skip-table=t1

    -O, --split-size-interval=# 将BinLog文件按照指定的大小拆分为多个段,解析结果为打印每个段的起始offset位置

    注意,当进行flashback时,flashback的内容先保存在内存中。若你的binlog大小为10G,那么需要额外的10G内存先暂时保存这部分信息。在某些情况下,如云环境、或服务器内存较小,会导致无法输出flashback的日志。这时可以通过此参数来设置内存保存文件的大小,例如将此值设置为100M,那么每100M就会刷新到一个文件

    -D, --datetime-to-pos=name 基于输入的时间信息,解析出该时间对应的第一个BinLog event偏移位置,格式参照start-datetime

    flashback时要先找到起始的偏移量,DBA可以先通过此参数定位到具体位置,然后再进行flashback操作

    「不是很好用啊,少了一个」

    -T, --table=name 仅解析该表,一般与database配套使用 (local log only)

    -E, --fb-event=name 仅解析该类型的Log event,一般与database、table选项配套使用。(local log only)

    可选的值有:

    • DELETE
    • INSERT
    • UPDATE

    DDL Flashbackup

    DDL Falshback 要求离线,比 DML Flashback 要求更多,生产环境数据可能覆盖导致恢复不全。

    MySQL 对于空间的重用机制与Oracle 有很大区别,对于Oracle 而言,如果是delete的数据,还是很难被覆盖掉的,对于drop 和truncate 则领导别论。然而MySQL则有所不同,MySQL 默认会启动一些purge 进程来进行空间重用,这是MySQL 5.6的情况

    localhost.(none)>show variables like '%innodb_purge_threads';
    +----------------------+-------+
    | Variable_name        | Value |
    +----------------------+-------+
    | innodb_purge_threads | 1     |
    +----------------------+-------+
    1 row in set (0.00 sec)
    

    在MySQL 5.7 版本中更为坑爹,MySQL 默认会启动4个purge 线程,因此很容易就会导致空间被重用,最终导致数据无法恢复,如下是MySQL 5.7的purge相关参数

    localhost.(none)>show variables like '%innodb_purge_threads';
    +----------------------+-------+
    | Variable_name        | Value |
    +----------------------+-------+
    | innodb_purge_threads | 4     |
    +----------------------+-------+
    1 row in set (0.00 sec)
    

    Percona Data Recovery Tool for InnoDB

    code wiki

    限制

    • 只对InnoDB/XtraDB表有效,而无法恢复MyISAM表

    • 离线恢复,MySQL Server 是关闭的

    • 不能保证数据总一定可被恢复。

      被重写的数据不能被恢复,这种情况下可能需要针对系统或物理的方式来恢复,不属于本工具的范畴。

    • 恢复的最好时机是当你发现数据丢失时,尽快备份MySQL数据文件

    • innodb_file_per_table = 0

    安装

    现在可以下载还是 0.5,2011-08-28,0.6 还未发布

    yum install glibc glibc-static -y
    
    wget https://launchpad.net/percona-data-recovery-tool-for-innodb/trunk/release-0.5/+download/percona-data-recovery-tool-for-innodb-0.5.tar.gz
    
    tar -xvf percona-data-recovery-tool-for-innodb-0.5.tar.gz
    cd percona-data-recovery-tool-for-innodb-0/mysql-source/
    ./configure
    cd ..
    make
    
    # ll -rth
    total 3.0M
    -rw-r--r--  1  510 wheel 2.0K Aug 28  2011 tables_dict.c
    -rwxr-xr-x  1  510 wheel  302 Aug 28  2011 split_dump.pl
    -rw-r--r--  1  510 wheel  11K Aug 28  2011 print_data.c
    -rw-r--r--  1  510 wheel  15K Aug 28  2011 page_parser.c
    -rw-r--r--  1  510 wheel 2.7K Aug 28  2011 Makefile
    -rw-r--r--  1  510 wheel   74 Aug 28  2011 INSTALL
    -rw-r--r--  1  510 wheel 9.0K Aug 28  2011 innochecksum.c
    -rw-r--r--  1  510 wheel 8.1K Aug 28  2011 incrementalupdate.c
    -rw-r--r--  1  510 wheel  12K Aug 28  2011 ibdconnect.c
    -rwxr-xr-x  1  510 wheel 2.0K Aug 28  2011 fetch_data.sh
    -rwxr-xr-x  1  510 wheel  12K Aug 28  2011 create_defs.pl
    -rw-r--r--  1  510 wheel  22K Aug 28  2011 constraints_parser.c
    -rw-r--r--  1  510 wheel 6.2K Aug 28  2011 check_data.c
    drwxr-xr-x  2  510 wheel 4.0K Aug 28  2011 include
    drwxr-xr-x  2  510 wheel 4.0K Aug 28  2011 docs
    drwxr-xr-x 40  510 wheel 4.0K Sep 26 17:53 mysql-source
    drwxr-xr-x  2 root root  4.0K Sep 26 17:55 lib
    -rwxr-xr-x  1 root root  725K Sep 26 17:55 constraints_parser
    -rwxr-xr-x  1 root root  1.2M Sep 26 17:55 page_parser
    -rwxr-xr-x  1 root root   15K Sep 26 17:55 innochecksum
    -rwxr-xr-x  1 root root  973K Sep 26 17:55 ibdconnect
    

    page_parser工具将根据InnoDB的底层实现原理,解析表的页和行结构。constraints_parser工具暂时不使用,后续还需要在定义表结构之后,重新编译生成它。

    如果MySQL是5.0之前的版本,InnoDB采取的是REDUNDANT格式,运行以下命令:

    ./page_parser -4 -f /path/to/ibdata1
    

    如果MySQL是5.0以后的版本,InnoDB采取的是COMPACT格式,运行以下命令:

    ./page_parser -5 -f /path/to/ibdata1
    

    运行后,page_parser工具会创建一个pages-<TIMESTAMP>的目录,其中TIMESTAMP是UNIX系统时间戳。在这个目录下,为每个index ID,以页的index ID创建一个子目录

    ...

    ...

    ...

    这个工具限制很多,没有测试,具体步骤可以参考 "其他参考"

    其他参考:

    Percona Data Recovery Tool 单表恢复

    使用Percona Data Recovery Tool for InnoDB恢复数据

    无备份情况下恢复MySQL误删的表

    Undrop MySQL InnoDB 中恢复被drop的表,当 innodb_file_per_table=off时

    undrop-for-innodb

    github percona live ppt wiki

    「2017-01-01 已经闭源并收费」,来源wiki

    MySQL · 数据恢复 · undrop-for-innodb 「数据库内核月报 - 2017 / 11」

    如果 innodb_file_per_table = on 恢复可能性更低

    • idb 文件被脏写
    • 有单独的 purge 线程

    限制

    • innodb_purge_threads = 0

    innodb_purge_threads <=5.6.1,默认为0,范围 [0-1]

    ​ = 5.6.2, <= 5.6.4 默认为0,范围 [0-32]

    ​ = 5.6.5 默认为1,范围 [1-32]

    ​ = 5.7.8 默认为4,范围 [4-32]

    5.6 以后,DDL 恢复还是比较困难的

    安装

    git clone https://github.com/chhabhaiya/undrop-for-innodb
    cd undrop-for-innodb
    make
    

    使用 「MySQL 5.6.24」

    1、truncate table

    localhost.test>select * from t_recover;
    +----+--------+
    
    | id | name   |
    
    +----+--------+
    |  1 | test1  |
    |  2 | test2  |
    |  3 | test3  |
    |  4 | test4  |
    |  5 | test5  |
    |  6 | test6  |
    |  7 | test7  |
    |  8 | test8  |
    |  9 | test9  |
    | 10 | test10 |
    +----+--------+
    
    10 rows in set (0.00 sec)
    
    mysql> truncate table t_recover;
    
    

    2、获取数据字典

    # ./stream_parser -f /data1/mysql7000/ibdata1 
    
    Opening file: /data1/mysql7000/ibdata1
    File information:
    
    ID of device containing file:         2055
    inode number:                      7012359
    protection:                         100640 (regular file)
    number of hard links:                    1
    user ID of owner:                      647
    group ID of owner:                     553
    device ID (if special file):             0
    blocksize for filesystem I/O:         4096
    number of blocks allocated:         205008
    Opening file: /data1/mysql7000/ibdata1
    File information:
    
    time of last access:            1506651425 Fri Sep 29 10:17:05 2017
    time of last modification:      1506651465 Fri Sep 29 10:17:45 2017
    ID of device containing file:         2055
    time of last status change:     1506651465 Fri Sep 29 10:17:45 2017
    inode number:                      7012359
    protection:                         100640 (regular file)
    number of hard links:                    1
    user ID of owner:                      647
    group ID of owner:                     553
    device ID (if special file):             0
    blocksize for filesystem I/O:         4096
    Opening file: /data1/mysql7000/ibdata1
    total size, in bytes:            104857600 (100.000 MiB)
    
    number of blocks allocated:         205008
    File information:
    
    Size to process:                 104857600 (100.000 MiB)
    ID of device containing file:         2055
    inode number:                      7012359
    protection:                         100640 (regular file)
    number of hard links:                    1
    user ID of owner:                      647
    group ID of owner:                     553
    device ID (if special file):             0
    blocksize for filesystem I/O:         4096
    number of blocks allocated:         205008
    Opening file: /data1/mysql7000/ibdata1
    File information:
    
    ID of device containing file:         2055
    inode number:                      7012359
    protection:                         100640 (regular file)
    time of last access:            1506651425 Fri Sep 29 10:17:05 2017
    number of hard links:                    1
    user ID of owner:                      647
    group ID of owner:                     553
    device ID (if special file):             0
    time of last modification:      1506651465 Fri Sep 29 10:17:45 2017
    blocksize for filesystem I/O:         4096
    number of blocks allocated:         205008
    time of last status change:     1506651465 Fri Sep 29 10:17:45 2017
    time of last access:            1506651425 Fri Sep 29 10:17:05 2017
    time of last modification:      1506651465 Fri Sep 29 10:17:45 2017
    time of last status change:     1506651465 Fri Sep 29 10:17:45 2017
    total size, in bytes:            104857600 (100.000 MiB)
    
    Opening file: /data1/mysql7000/ibdata1
    total size, in bytes:            104857600 (100.000 MiB)
    
    File information:
    
    Size to process:                 104857600 (100.000 MiB)
    ID of device containing file:         2055
    Size to process:                 104857600 (100.000 MiB)
    inode number:                      7012359
    protection:                         100640 (regular file)
    number of hard links:                    1
    user ID of owner:                      647
    group ID of owner:                     553
    device ID (if special file):             0
    blocksize for filesystem I/O:         4096
    number of blocks allocated:         205008
    time of last access:            1506651425 Fri Sep 29 10:17:05 2017
    time of last modification:      1506651465 Fri Sep 29 10:17:45 2017
    time of last status change:     1506651465 Fri Sep 29 10:17:45 2017
    total size, in bytes:            104857600 (100.000 MiB)
    
    Opening file: /data1/mysql7000/ibdata1
    File information:
    
    Size to process:                 104857600 (100.000 MiB)
    ID of device containing file:         2055
    inode number:                      7012359
    protection:                         100640 (regular file)
    number of hard links:                    1
    Opening file: /data1/mysql7000/ibdata1
    user ID of owner:                      647
    time of last access:            1506651425 Fri Sep 29 10:17:05 2017
    File information:
    
    group ID of owner:                     553
    device ID (if special file):             0
    time of last modification:      1506651465 Fri Sep 29 10:17:45 2017
    blocksize for filesystem I/O:         4096
    ID of device containing file:         2055
    time of last status change:     1506651465 Fri Sep 29 10:17:45 2017
    number of blocks allocated:         205008
    inode number:                      7012359
    protection:                         100640 (regular file)
    number of hard links:                    1
    total size, in bytes:            104857600 (100.000 MiB)
    
    user ID of owner:                      647
    group ID of owner:                     553
    device ID (if special file):             0
    blocksize for filesystem I/O:         4096
    number of blocks allocated:         205008
    Size to process:                 104857600 (100.000 MiB)
    Opening file: /data1/mysql7000/ibdata1
    File information:
    
    ID of device containing file:         2055
    inode number:                      7012359
    time of last access:            1506651425 Fri Sep 29 10:17:05 2017
    protection:                         100640 (regular file)
    time of last access:            1506651425 Fri Sep 29 10:17:05 2017
    time of last modification:      1506651465 Fri Sep 29 10:17:45 2017
    number of hard links:                    1
    user ID of owner:                      647
    time of last modification:      1506651465 Fri Sep 29 10:17:45 2017
    group ID of owner:                     553
    device ID (if special file):             0
    time of last status change:     1506651465 Fri Sep 29 10:17:45 2017
    blocksize for filesystem I/O:         4096
    number of blocks allocated:         205008
    total size, in bytes:            104857600 (100.000 MiB)
    
    Size to process:                 104857600 (100.000 MiB)
    Opening file: /data1/mysql7000/ibdata1
    File information:
    
    ID of device containing file:         2055
    inode number:                      7012359
    protection:                         100640 (regular file)
    number of hard links:                    1
    user ID of owner:                      647
    group ID of owner:                     553
    device ID (if special file):             0
    blocksize for filesystem I/O:         4096
    number of blocks allocated:         205008
    time of last access:            1506651425 Fri Sep 29 10:17:05 2017
    time of last modification:      1506651465 Fri Sep 29 10:17:45 2017
    time of last status change:     1506651465 Fri Sep 29 10:17:45 2017
    total size, in bytes:            104857600 (100.000 MiB)
    
    time of last access:            1506651425 Fri Sep 29 10:17:05 2017
    Size to process:                 104857600 (100.000 MiB)
    time of last modification:      1506651465 Fri Sep 29 10:17:45 2017
    time of last status change:     1506651465 Fri Sep 29 10:17:45 2017
    total size, in bytes:            104857600 (100.000 MiB)
    
    Opening file: /data1/mysql7000/ibdata1
    File information:
    
    Size to process:                 104857600 (100.000 MiB)
    ID of device containing file:         2055
    inode number:                      7012359
    protection:                         100640 (regular file)
    number of hard links:                    1
    user ID of owner:                      647
    group ID of owner:                     553
    device ID (if special file):             0
    blocksize for filesystem I/O:         4096
    number of blocks allocated:         205008
    time of last access:            1506651425 Fri Sep 29 10:17:05 2017
    time of last modification:      1506651465 Fri Sep 29 10:17:45 2017
    time of last status change:     1506651465 Fri Sep 29 10:17:45 2017
    total size, in bytes:            104857600 (100.000 MiB)
    
    Size to process:                 104857600 (100.000 MiB)
    time of last status change:     1506651465 Fri Sep 29 10:17:45 2017
    total size, in bytes:            104857600 (100.000 MiB)
    
    Size to process:                 104857600 (100.000 MiB)
    Opening file: /data1/mysql7000/ibdata1
    File information:
    
    ID of device containing file:         2055
    inode number:                      7012359
    protection:                         100640 (regular file)
    number of hard links:                    1
    user ID of owner:                      647
    group ID of owner:                     553
    device ID (if special file):             0
    blocksize for filesystem I/O:         4096
    number of blocks allocated:         205008
    time of last access:            1506651425 Fri Sep 29 10:17:05 2017
    time of last modification:      1506651465 Fri Sep 29 10:17:45 2017
    time of last status change:     1506651465 Fri Sep 29 10:17:45 2017
    total size, in bytes:            104857600 (100.000 MiB)
    
    Size to process:                 104857600 (100.000 MiB)
    Opening file: /data1/mysql7000/ibdata1
    File information:
    
    ID of device containing file:         2055
    inode number:                      7012359
    protection:                         100640 (regular file)
    number of hard links:                    1
    user ID of owner:                      647
    group ID of owner:                     553
    device ID (if special file):             0
    blocksize for filesystem I/O:         4096
    number of blocks allocated:         205008
    time of last access:            1506651425 Fri Sep 29 10:17:05 2017
    time of last modification:      1506651465 Fri Sep 29 10:17:45 2017
    time of last status change:     1506651465 Fri Sep 29 10:17:45 2017
    total size, in bytes:            104857600 (100.000 MiB)
    
    Size to process:                 104857600 (100.000 MiB)
    All workers finished in 0 sec
    
    

    stream_parser这个工具发现在ibdata1中,找到了存储页面类型(FIL_PAGE_INDEX或FIL_PAGE_TYPE_BLOB)由index_id的整理InnoDB的页面

    # cd pages-ibdata1/
    
    # ll -rth
    total 8.0K
    drwxr-xr-x 2 root root 4.0K Sep 29 10:18 FIL_PAGE_TYPE_BLOB
    drwxr-xr-x 2 root root 4.0K Sep 29 10:18 FIL_PAGE_INDEX
    
    

    SYS_TABLES

    SYS_TABLES本身是一个表,该表用于规范innodb各种表定义和保存innodb中各种表的基本信息,结合SYS_COLUMNS,SYS_INDEXES和SYS_FOREIGN系统表定义了特定某个表的所有信息。

    undrop-for-innodb]# ll pages-ibdata1/FIL_PAGE_INDEX/0000000000000001.page 
    -rw-r--r-- 1 root root 32768 Sep 29 10:18 pages-ibdata1/FIL_PAGE_INDEX/0000000000000001.page
    

    SYS_INDEXES

    SYS_INDEXES用于保存innodb中每个表定义的每个索引对象

    undrop-for-innodb]# ll pages-ibdata1/FIL_PAGE_INDEX/0000000000000003.page 
    -rw-r--r-- 1 root root 16384 Sep 29 10:18 pages-ibdata1/FIL_PAGE_INDEX/0000000000000003.page
    

    SYS_COLUMNS

    SYS_COLUMNS用于保存innodb引擎每个表定义的列,与SYS_TABLES相似。

    undrop-for-innodb]# ll pages-ibdata1/FIL_PAGE_INDEX/0000000000000002.page 
    -rw-r--r-- 1 root root 32768 Sep 29 10:18 pages-ibdata1/FIL_PAGE_INDEX/0000000000000002.page
    

    SYS_FIELDS

    SYS_FIELDS用于保存innodb中每个索引的每个列对象

    undrop-for-innodb]# ll pages-ibdata1/FIL_PAGE_INDEX/0000000000000004.page 
    -rw-r--r-- 1 root root 32768 Sep 29 10:18 pages-ibdata1/FIL_PAGE_INDEX/0000000000000004.page
    

    3、扫描逻辑卷

    ./stream_parser -f /dev/sda7 -t 105000000k
    
    Opening file: /dev/sda7
    File information:
    
    ID of device containing file:            5
    inode number:                         6047
    protection:                          60660 (block device)
    number of hard links:                    1
    user ID of owner:                        0
    group ID of owner:                       6
    device ID (if special file):          2055
    blocksize for filesystem I/O:         4096
    number of blocks allocated:              0
    Opening file: /dev/sda7
    File information:
    
    ID of device containing file:            5
    inode number:                         6047
    protection:                          60660 (block device)
    number of hard links:                    1
    user ID of owner:                        0
    group ID of owner:                       6
    device ID (if special file):          2055
    blocksize for filesystem I/O:         4096
    number of blocks allocated:              0
    time of last access:            1490933835 Fri Mar 31 12:17:15 2017
    time of last modification:      1477372695 Tue Oct 25 13:18:15 2016
    time of last status change:     1477372695 Tue Oct 25 13:18:15 2016
    total size, in bytes:                    0 (0.000 exp(+0))
    
    Opening file: /dev/sda7
    File information:
    
    Size to process:              107520000000 (100.136 GiB)
    time of last access:            1490933835 Fri Mar 31 12:17:15 2017
    time of last modification:      1477372695 Tue Oct 25 13:18:15 2016
    time of last status change:     1477372695 Tue Oct 25 13:18:15 2016
    ID of device containing file:            5
    inode number:                         6047
    protection:                          60660 total size, in bytes:                    0 (0.000 exp(+0))
    
    (block device)
    Opening file: /dev/sda7
    number of hard links:                    1
    Size to process:              107520000000 (100.136 GiB)
    File information:
    
    user ID of owner:                        0
    group ID of owner:                       6
    device ID (if special file):          2055
    blocksize for filesystem I/O:         4096
    number of blocks allocated:              0
    ID of device containing file:            5
    inode number:                         6047
    protection:                          60660 (block device)
    number of hard links:                    1
    user ID of owner:                        0
    group ID of owner:                       6
    device ID (if special file):          2055
    blocksize for filesystem I/O:         4096
    number of blocks allocated:              0
    Opening file: /dev/sda7
    File information:
    
    ID of device containing file:            5
    inode number:                         6047
    protection:                          60660 (block device)
    number of hard links:                    1
    Opening file: /dev/sda7
    user ID of owner:                        0
    group ID of owner:                       6
    File information:
    
    device ID (if special file):          2055
    blocksize for filesystem I/O:         4096
    number of blocks allocated:              0
    time of last access:            1490933835 Fri Mar 31 12:17:15 2017
    time of last modification:      1477372695 Tue Oct 25 13:18:15 2016
    time of last status change:     1477372695 Tue Oct 25 13:18:15 2016
    ID of device containing file:            5
    time of last access:            1490933835 Fri Mar 31 12:17:15 2017
    inode number:                         6047
    protection:                          60660 (block device)
    number of hard links:                    1
    user ID of owner:                        0
    time of last modification:      1477372695 Tue Oct 25 13:18:15 2016
    group ID of owner:                       6
    total size, in bytes:                    0 (0.000 exp(+0))
    
    time of last status change:     1477372695 Tue Oct 25 13:18:15 2016
    device ID (if special file):          2055
    blocksize for filesystem I/O:         4096
    number of blocks allocated:              0
    total size, in bytes:                    0 (0.000 exp(+0))
    
    Size to process:              107520000000 (100.136 GiB)
    Size to process:              107520000000 (100.136 GiB)
    time of last access:            1490933835 Fri Mar 31 12:17:15 2017
    Opening file: /dev/sda7
    time of last modification:      1477372695 Tue Oct 25 13:18:15 2016
    File information:
    
    time of last status change:     1477372695 Tue Oct 25 13:18:15 2016
    ID of device containing file:            5
    total size, in bytes:                    0 (0.000 exp(+0))
    
    inode number:                         6047
    protection:                          60660 (block device)
    number of hard links:                    1
    user ID of owner:                        0
    group ID of owner:                       6
    device ID (if special file):          2055
    blocksize for filesystem I/O:         4096
    Size to process:              107520000000 (100.136 GiB)
    number of blocks allocated:              0
    time of last access:            1490933835 Fri Mar 31 12:17:15 2017
    time of last modification:      1477372695 Tue Oct 25 13:18:15 2016
    Opening file: /dev/sda7
    time of last status change:     1477372695 Tue Oct 25 13:18:15 2016
    File information:
    
    total size, in bytes:                    0 (0.000 exp(+0))
    
    ID of device containing file:            5
    inode number:                         6047
    Size to process:              107520000000 (100.136 GiB)
    protection:                          60660 (block device)
    number of hard links:                    1
    user ID of owner:                        0
    group ID of owner:                       6
    device ID (if special file):          2055
    blocksize for filesystem I/O:         4096
    number of blocks allocated:              0
    time of last access:            1490933835 Fri Mar 31 12:17:15 2017
    time of last modification:      1477372695 Tue Oct 25 13:18:15 2016
    time of last status change:     1477372695 Tue Oct 25 13:18:15 2016
    Opening file: /dev/sda7
    total size, in bytes:                    0 (0.000 exp(+0))
    
    File information:
    
    ID of device containing file:            5
    Size to process:              107520000000 (100.136 GiB)
    inode number:                         6047
    protection:                          60660 (block device)
    number of hard links:                    1
    user ID of owner:                        0
    group ID of owner:                       6
    device ID (if special file):          2055
    blocksize for filesystem I/O:         4096
    number of blocks allocated:              0
    time of last access:            1490933835 Fri Mar 31 12:17:15 2017
    time of last modification:      1477372695 Tue Oct 25 13:18:15 2016
    time of last status change:     1477372695 Tue Oct 25 13:18:15 2016
    total size, in bytes:                    0 (0.000 exp(+0))
    
    Size to process:              107520000000 (100.136 GiB)
    time of last access:            1490933835 Fri Mar 31 12:17:15 2017
    time of last modification:      1477372695 Tue Oct 25 13:18:15 2016
    time of last status change:     1477372695 Tue Oct 25 13:18:15 2016
    total size, in bytes:                    0 (0.000 exp(+0))
    
    Size to process:              107520000000 (100.136 GiB)
    Opening file: /dev/sda7
    File information:
    
    Opening file: /dev/sda7
    File information:
    
    ID of device containing file:            5
    inode number:                         6047
    protection:                          60660 (block device)
    number of hard links:                    1
    ID of device containing file:            5
    user ID of owner:                        0
    inode number:                         6047
    group ID of owner:                       6
    protection:                          60660 device ID (if special file):          2055
    (block device)
    blocksize for filesystem I/O:         4096
    number of hard links:                    1
    number of blocks allocated:              0
    user ID of owner:                        0
    group ID of owner:                       6
    device ID (if special file):          2055
    blocksize for filesystem I/O:         4096
    number of blocks allocated:              0
    time of last access:            1490933835 Fri Mar 31 12:17:15 2017
    time of last access:            1490933835 Fri Mar 31 12:17:15 2017
    time of last modification:      1477372695 Tue Oct 25 13:18:15 2016
    time of last modification:      1477372695 Tue Oct 25 13:18:15 2016
    time of last status change:     1477372695 Tue Oct 25 13:18:15 2016
    time of last status change:     1477372695 Tue Oct 25 13:18:15 2016
    total size, in bytes:                    0 (0.000 exp(+0))
    
    total size, in bytes:                    0 (0.000 exp(+0))
    
    Size to process:              107520000000 (100.136 GiB)
    Size to process:              107520000000 (100.136 GiB)
    Opening file: /dev/sda7
    File information:
    
    ID of device containing file:            5
    inode number:                         6047
    protection:                          60660 (block device)
    number of hard links:                    1
    user ID of owner:                        0
    group ID of owner:                       6
    device ID (if special file):          2055
    blocksize for filesystem I/O:         4096
    number of blocks allocated:              0
    time of last access:            1490933835 Fri Mar 31 12:17:15 2017
    time of last modification:      1477372695 Tue Oct 25 13:18:15 2016
    time of last status change:     1477372695 Tue Oct 25 13:18:15 2016
    total size, in bytes:                    0 (0.000 exp(+0))
    
    Size to process:              107520000000 (100.136 GiB)
    Worker(2): 1.12% done. 2017-09-29 10:22:56 ETA(in 00:03:12). Processing speed: 44.000 MiB/sec
    Worker(3): 1.12% done. 2017-09-29 10:22:56 ETA(in 00:03:12). Processing speed: 44.000 MiB/sec
    
    ....
    ....
    ....
    
    Worker(11): 97.77% done. 2017-09-29 10:23:21 ETA(in 00:00:02). Processing speed: 87.828 MiB/sec
    Worker(0): 97.74% done. 2017-09-29 10:23:21 ETA(in 00:00:02). Processing speed: 87.844 MiB/sec
    Worker(9): 97.74% done. 2017-09-29 10:23:21 ETA(in 00:00:02). Processing speed: 87.844 MiB/sec
    Worker(8): 99.80% done. 2017-09-29 10:23:20 ETA(in 00:00:00). Processing speed: 87.828 MiB/sec
    Worker(11): 98.80% done. 2017-09-29 10:23:21 ETA(in 00:00:01). Processing speed: 87.828 MiB/sec
    Worker(0): 98.77% done. 2017-09-29 10:23:21 ETA(in 00:00:01). Processing speed: 87.828 MiB/sec
    Worker(9): 98.77% done. 2017-09-29 10:23:21 ETA(in 00:00:01). Processing speed: 87.828 MiB/sec
    Worker(0): 99.80% done. 2017-09-29 10:23:21 ETA(in 00:00:00). Processing speed: 87.828 MiB/sec
    Worker(11): 99.83% done. 2017-09-29 10:23:21 ETA(in 00:00:00). Processing speed: 87.828 MiB/sec
    Worker(9): 99.80% done. 2017-09-29 10:23:21 ETA(in 00:00:00). Processing speed: 87.844 MiB/sec
    All workers finished in 219 sec
    [root@idaross29 undrop-for-innodb]# ./stream_parser -f /dev/sda7 -t 105000000k
    Could not create directory pages-sda7
    mkdir(): File exists
    

    4、创建数据字典表

    recover_dictionary.sh
    
    修改 mysql 登陆相关命令 mysql -u -p -S
    
    # ./recover_dictionary.sh 
    Generating dictionary tables dumps... OK
    Creating test database ... OK
    Creating dictionary tables in database test:
    SYS_TABLES ... OK
    SYS_COLUMNS ... OK
    SYS_INDEXES ... OK
    SYS_FIELDS ... OK
    All OK
    Loading dictionary tables data:
    SYS_TABLES ... 24 recs OK
    SYS_COLUMNS ... 148 recs OK
    SYS_INDEXES ... 14 recs OK
    SYS_FIELDS ... 34 recs OK
    All OK
    
    localhost.test>show tables;
    +----------------+
    | Tables_in_test |
    +----------------+
    | SYS_COLUMNS    |
    | SYS_FIELDS     |
    | SYS_INDEXES    |
    | SYS_TABLES     |
    

    5、查询需要恢复的表的index_id信息

    localhost.test>select * from SYS_TABLES where name like 'test/t_recover';
    +----------------+----+--------+------+--------+---------+--------------+-------+
    | NAME           | ID | N_COLS | TYPE | MIX_ID | MIX_LEN | CLUSTER_NAME | SPACE |
    +----------------+----+--------+------+--------+---------+--------------+-------+
    | test/t_recover | 25 |      2 |    1 |      0 |      80 |              |    11 |
    +----------------+----+--------+------+--------+---------+--------------+-------+
    1 row in set (0.02 sec)
    
    localhost.test>select * from SYS_INDEXES where table_id = 25;
    +----------+----+---------+----------+------+-------+---------+
    | TABLE_ID | ID | NAME    | N_FIELDS | TYPE | SPACE | PAGE_NO |
    +----------+----+---------+----------+------+-------+---------+
    |       25 | 28 | PRIMARY |        1 |    3 |    11 |       3 |
    +----------+----+---------+----------+------+-------+---------+
    1 row in set (0.01 sec)
    

    可以看到被truncate的表的index_id 为28,我们应该进一步从28 的page中获取数据。

    6、拿到表结构,并检查数据是否可以恢复

    • 表结构
    cat t_recover.sql 
    create table t_recover(id int primary key,name varchar(20));
    
    • 确认数据是否存在
    undrop-for-innodb]# ./c_parser -6f pages-ibdata1/FIL_PAGE_INDEX/0000000000000028.page -t t_recover.sql | head -5
    SET FOREIGN_KEY_CHECKS=0;
    -- Page id: 3, Format: COMPACT, Records list: Valid, Expected records: (0 0)
    LOAD DATA LOCAL INFILE '-- Page id: 3, Found records: 0, Lost records: NO, Leaf page: YES
    -- Page id: 3, Format: COMPACT, Records list: Valid, Expected records: (0 0)
    /data1/jinlin3/undrop-for-innodb/dumps/default/t_recover' REPLACE INTO TABLE `t_recover` FIELDS TERMINATED BY '	' OPTIONALLY ENCLOSED BY '"' LINES STARTING BY 't_recover	' (`id`, `name`);
    

    7、抽取 Page 中的数据

    undrop-for-innodb]# ./c_parser -6f pages-ibdata1/FIL_PAGE_INDEX/0000000000000028.page -t t_recover.sql > dumps/default/t_recover 2> dumps/default/t_recover_load.sql
    

    8、加载数据

    source dumps/default/t_recover_load.sql
    

    但是,我测试是 5.6 ,数据被 purge 掉了

    undrop-for-innodb]# cat dumps/default/t_recover
    -- Page id: 3, Format: COMPACT, Records list: Valid, Expected records: (0 0)
    -- Page id: 3, Found records: 0, Lost records: NO, Leaf page: YES
    -- Page id: 3, Format: COMPACT, Records list: Valid, Expected records: (0 0)
    -- Page id: 3, Found records: 0, Lost records: NO, Leaf page: YES
    
    
    undrop-for-innodb]# cat dumps/default/t_recover_load.sql 
    SET FOREIGN_KEY_CHECKS=0;
    LOAD DATA LOCAL INFILE '/data1/jinlin3/undrop-for-innodb/dumps/default/t_recover' REPLACE INTO TABLE `t_recover` FIELDS TERMINATED BY '	' OPTIONALLY ENCLOSED BY '"' LINES STARTING BY 't_recover	' (`id`, `name`);
    

    若 truncate table 后,立刻 shutdown MySQL,文件是有内容,但不对啊!

    undrop-for-innodb]# cat dumps/default/t_truncate
    -- Page id: 3, Format: COMPACT, Records list: Valid, Expected records: (17 17)
    0000000B0000    00000000000025  t_truncate      -2147483648     ""
    0000000C0000    00000000000025  t_truncate      -2147483648     "@�"
    0000000D0000    00000000000025  t_truncate      -2147483648     "@"
    0000000E0000    00000000000025  t_truncate      -2147483648     ""
    0000000E0000    00010000000025  t_truncate      -2147483648     ""
    0000000F0000    00000000000025  t_truncate      -2147483648     ""
    000000100000    00000000000025  t_truncate      -2147483648     ""
    000000120000    00030000000025  t_truncate      -2147483648     "@ps"
    000000130000    00000000000025  t_truncate      -2147483648     ""
    000000140000    00000000000025  t_truncate      -2147483648     ""
    000000140000    00010000000025  t_truncate      -2147483648     ""
    000000150000    00000000000025  t_truncate      -2147483648     ""
    000000160000    00000000000025  t_truncate      -2147483648     ""
    000000170000    00000000000025  t_truncate      -2147483648     ""
    000000180000    00000000000025  t_truncate      -2147483648     "@ka"
    0000001A0000    00000000000025  t_truncate      -2147483648     ""
    0000001C0000    00000000000025  t_truncate      -2147483648     ""
    -- Page id: 3, Found records: 17, Lost records: NO, Leaf page: YES
    
    

    <u>这是一个悲伤的故事!</u>

    其他参考:

    undrop for innodb c_parser 源码分析

    Recover InnoDB dictionary

    MySQL 如何对InnoDB使用Undrop来恢复InnoDB数据

    希望小伙伴永远不需要用到此方法,备份才是王道!



    作者:saup007
    链接:https://www.jianshu.com/p/31818e1727be
    來源:简书
    简书著作权归作者所有,任何形式的转载都请联系作者获得授权并注明出处。
  • 相关阅读:
    leetcode Remove Linked List Elements
    leetcode Word Pattern
    leetcode Isomorphic Strings
    leetcode Valid Parentheses
    leetcode Remove Nth Node From End of List
    leetcode Contains Duplicate II
    leetcode Rectangle Area
    leetcode Length of Last Word
    leetcode Valid Sudoku
    leetcode Reverse Bits
  • 原文地址:https://www.cnblogs.com/DataArt/p/9873365.html
Copyright © 2011-2022 走看看