修改命令参数alter
1、不需要包含alter table关键字,可以包含多个修改操作,使用逗号分开,如"drop clolumn c1, add column c2 int" 2、不支持rename语句来对表进行重命名操作 3、不支持对索引进行重命名操作 4、如果删除外键,需要对外键名加下划线,如删除外键fk_uid, 修改语句为"DROP FOREIGN KEY _fk_uid" 5、重命名字段,不能使用drop add方式,会导致数据丢失,使用“change col1 col1_new type constraint",保持数据和约束一致。
版本检查参数
在使用PT-OSC对阿里云RDS上表进行操作时,如未指定--noversion-check参数,会报如下错误: Can't use an undefined value as an ARRAY reference at /usr/bin/pt-online-schema-change line 7547. 版本检查参数: --[no]version-check Check for the latest version of Percona Toolkit, MySQL, and other programs (default yes)
外键参数alter-foreign-keys-method
当对“被其他表外键关联的表”做修改时,RENAME操作会导致外键关联失败而不允许RENAME执行。 pt-osc提供--alter-foreign-keys-method选项:rebuild_constraints/drop_swap/auto/none 假设 t1 是要修改的表,t2 有外键依赖于 t1,_t1_new 是pt-osc工具产生的新临时表。 rebuild_constraints实现方式: 1、删除T2上的外键约束 2、对T1和_t1_new进行重命令 3、新增T2上的外键约束 drop_swap实现方式: 1、禁用T2的外键约束检查,FOREIGN_KEY_CHECKS=0, 2、DROP原始表t1,对_t1_new进行重命令 3、开启T2的外键约束检查 none实现方式: 1、禁用T2的外键约束检查,FOREIGN_KEY_CHECKS=0, 2、对表t1和_t1_new进行重命令 3、开启T2的外键约束检查 auto实现方式: 自动决定使用rebuild_constraints或drop_swap
none和drop swap区别在于none方式不删除原表
使用none参数生成的日志:
2019-10-10T07:58:42.430688-05:00 3430 Query RENAME TABLE `dadaabc_test`.`tb001` TO `dadaabc_test`.`_tb001_old`, `dadaabc_test`.`_tb001_new` TO `dadaabc_test`.`tb001` 2019-10-10T07:58:42.451818-05:00 3430 Query SET foreign_key_checks=0 2019-10-10T07:58:42.451884-05:00 3430 Query DROP TABLE IF EXISTS `dadaabc_test`.`_tb001_old` 2019-10-10T07:58:42.740162-05:00 3430 Query DROP TRIGGER IF EXISTS `dadaabc_test`.`pt_osc_dadaabc_test_tb001_del` 2019-10-10T07:58:42.740738-05:00 3430 Query DROP TRIGGER IF EXISTS `dadaabc_test`.`pt_osc_dadaabc_test_tb001_upd` 2019-10-10T07:58:42.741216-05:00 3430 Query DROP TRIGGER IF EXISTS `dadaabc_test`.`pt_osc_dadaabc_test_tb001_ins` 2019-10-10T07:58:42.741832-05:00 3430 Query SHOW TABLES FROM `dadaabc_test` LIKE '\_tb001\_new'
如果有表tb002依赖于tb001,那么就会导致修改tb001完成后tb002的建表语句变化为:
CREATE TABLE `tb002` ( `ID` INT(11) NOT NULL AUTO_INCREMENT, `C1` INT(11) DEFAULT NULL, `C2` INT(11) DEFAULT NULL, `C3` INT(11) DEFAULT NULL, PRIMARY KEY (`ID`), KEY `FK_C1` (`C1`), CONSTRAINT `_FK_C1` FOREIGN KEY (`C1`) REFERENCES `_tb001_old` (`ID`) ) ENGINE=INNODB AUTO_INCREMENT=6356899 DEFAULT CHARSET=utf8
正因如此,在使用alter-foreign-keys-method=none时,会有如下警告:
Using alter-foreign-keys-method = "none". This will typically cause foreign key violations! This method of handling foreign key constraints is only provided so that the database administrator can disable the tool’s built-in functionality if desired.
如果使用alter-foreign-keys-method=drop_swap时,不会对原表执行rename操作,也不会导致依赖该表的外键受影响,其执行日志为:
2019-10-10T07:48:06.695144-05:00 3428 Query ANALYZE TABLE `dadaabc_test`.`_tb001_new` /* pt-online-schema-change */ 2019-10-10T07:48:06.706498-05:00 3428 Query SET foreign_key_checks=0 2019-10-10T07:48:06.707180-05:00 3428 Query DROP TABLE IF EXISTS `dadaabc_test`.`tb001` 2019-10-10T07:48:06.731187-05:00 3428 Query RENAME TABLE `dadaabc_test`.`_tb001_new` TO `dadaabc_test`.`tb001` 2019-10-10T07:48:06.734388-05:00 3428 Query DROP TRIGGER IF EXISTS `dadaabc_test`.`pt_osc_dadaabc_test_tb001_del` 2019-10-10T07:48:06.736416-05:00 3428 Query DROP TRIGGER IF EXISTS `dadaabc_test`.`pt_osc_dadaabc_test_tb001_upd` 2019-10-10T07:48:06.736893-05:00 3428 Query DROP TRIGGER IF EXISTS `dadaabc_test`.`pt_osc_dadaabc_test_tb001_ins` 2019-10-10T07:48:06.737513-05:00 3428 Query SHOW TABLES FROM `dadaabc_test` LIKE '\_tb001\_new'
主机性能压力检查参数
在每次考完完成一个chunk数据后,会在主库上执行SHOW GLOBAL STATUS来获取当前的运行状态,判断是否继续拷贝数据和是否取消执行。 --critical-load=(type: Array; default: Threads_running=50) Examine SHOW GLOBAL STATUS after every chunk, and abort if the load is too high (default Threads_running=50) --max-load=(type: Array; default: Threads_running=25) Examine SHOW GLOBAL STATUS after every chunk, and pause if any status variables are higher than their thresholds (default Threads_running=25) 参数--max-load不仅可以执行Threads_running状态的阈值,还可以指定其他状态阈值,如果未指定,则会默认使用当前值的120%作为阈值,如假设未指定Threads_connected参数,且当前Threads_connected参数值为100,则当Threads_connected参数值超过120时,会暂停拷贝。 当状态值超过--critical-load阈值后,会暂停拷贝数据并休眠一个复制周期(--chunk-time值), 当状态值超过--critical-load阈值后,会取消pt-osc工具继续执行并清理中间数据。
从库复制延迟检查参数
--recursion-method Preferred recursion method for discovering replicas (default processlist,hosts) 选择通过何种方式获取从库信息,可选参数: processlist SHOW PROCESSLIST hosts SHOW SLAVE HOSTS dsn=DSN DSNs from a table none Do not find slaves --max-lag(type: time; default: 1s) Pause the data copy until all replicas' lag is less than this value (default 1s). Optional suffix s=seconds, m=minutes, h=hours, d=days; if no suffix, s is used. 默认为1s,每个chunks拷贝完成后,会查看check-slave-lag参数所指定的从库的延迟信息,如果超过max-log的阀值,则暂停复制数据,直到复制延迟小于max-log的阀值。 检查复制延迟信息依赖于SHOW SLAVE STATUS语句中返回的Seconds_Behind_Master列的值。 --check-interval(type: time; default: 1s) Sleep time between checks for --max-lag (default 1). Optional suffix s=seconds, m=minutes, h=hours, d=days; if no suffix, s is used. 当出现复制延迟暂停复制数据后,按照check-interval指定的时间进行周期检查复制延迟。 --check-slave-lag=s Pause the data copy until this replica's lag is less than --max-lag 需要检查复制延迟的从库IP 如果指定check-slave-lag参数,且从库无法正常连接或从库IO线程和SQL线程停止,会认为主从存在延迟,导致复制数据操作一直暂停。 如果未指定check-slave-lag参数,默认会检查从库的延迟,但复制延迟不会导致数据复制暂停。 --skip-check-slave-lag=d DSN to skip when checking slave lag 不需要检查复制延迟的从库IP --slave-user=s Sets the user to be used to connect to the slaves --slave-password=s Sets the password to be used to connect to the slaves 如果未指明从库账号信息,则默认使用主库相同的端口/用户/密码等信息。
批量复制数据参数
--chunk-index=s Prefer this index for chunking tables --chunk-index-columns=i Use only this many left-most columns of a --chunk-index --chunk-size=z Number of rows to select for each chunk copied (default 1000) --chunk-size-limit=f Do not copy chunks this much larger than the desired chunk size (default 4.0) --chunk-time=f Adjust the chunk size dynamically so each data-copy query takes this long to execute (default 0.5) 当chunk-size和chunk-time两者都未指定时,chunk-size默认值为1000,chunk-time默认值为0.5S,第一次按照chunk-size来进行数据复制,然后根据第一次复制的时间动态调整chumk-size的大小,以适应服务器的性能变化,如上一次复制1000行消耗0.1S,则下次动态调整chumk-size为5000。 如果明确指定chumk-size的值或将chunk-time指定为0,则每次都按照chunk-size复制数据。
帮助文档(3.0.3)
pt-online-schema-change alters a table's structure without blocking reads or writes. Specify the database and table in the DSN. Do not use this tool before reading its documentation and checking your backups carefully. For more details, please use the --help option, or try 'perldoc /usr/bin/pt-online-schema-change' for complete documentation. Usage: pt-online-schema-change [OPTIONS] DSN Options: --alter=s The schema modification, without the ALTER TABLE keywords --alter-foreign-keys-method=s How to modify foreign keys so they reference the new table --[no]analyze-before-swap Execute ANALYZE TABLE on the new table before swapping with the old one (default yes) --ask-pass Prompt for a password when connecting to MySQL --charset=s -A Default character set --[no]check-alter Parses the --alter specified and tries to warn of possible unintended behavior ( default yes) --check-interval=m Sleep time between checks for --max-lag ( default 1). Optional suffix s=seconds, m= minutes, h=hours, d=days; if no suffix, s is used. --[no]check-plan Check query execution plans for safety ( default yes) --[no]check-replication-filters Abort if any replication filter is set on any server (default yes) --check-slave-lag=s Pause the data copy until this replica's lag is less than --max-lag --chunk-index=s Prefer this index for chunking tables --chunk-index-columns=i Use only this many left-most columns of a -- chunk-index --chunk-size=z Number of rows to select for each chunk copied (default 1000) --chunk-size-limit=f Do not copy chunks this much larger than the desired chunk size (default 4.0) --chunk-time=f Adjust the chunk size dynamically so each data-copy query takes this long to execute ( default 0.5) --config=A Read this comma-separated list of config files; if specified, this must be the first option on the command line --critical-load=A Examine SHOW GLOBAL STATUS after every chunk, and abort if the load is too high ( default Threads_running=50) --data-dir=s Create the new table on a different partition using the DATA DIRECTORY feature --database=s -D Connect to this database --default-engine Remove ENGINE from the new table --defaults-file=s -F Only read mysql options from the given file --[no]drop-new-table Drop the new table if copying the original table fails (default yes) --[no]drop-old-table Drop the original table after renaming it ( default yes) --[no]drop-triggers Drop triggers on the old table. --no-drop- triggers forces --no-drop-old-table (default yes) --dry-run Create and alter the new table, but do not create triggers, copy data, or replace the original table --execute Indicate that you have read the documentation and want to alter the table --force This options bypasses confirmation in case of using alter-foreign-keys-method = none , which might break foreign key constraints --help Show help and exit --host=s -h Connect to host --max-flow-ctl=f Somewhat similar to --max-lag but for PXC clusters --max-lag=m Pause the data copy until all replicas' lag is less than this value (default 1s). Optional suffix s=seconds, m=minutes, h= hours, d=days; if no suffix, s is used. --max-load=A Examine SHOW GLOBAL STATUS after every chunk, and pause if any status variables are higher than their thresholds (default Threads_running=25) --new-table-name=s New table name before it is swapped. %T is replaced with the original table name ( default %T_new) --null-to-not-null Allows MODIFYing a column that allows NULL values to one that doesn't allow them --password=s -p Password to use when connecting --pause-file=s Execution will be paused while the file specified by this param exists --pid=s Create the given PID file --plugin=s Perl module file that defines a pt_online_schema_change_plugin class --port=i -P Port number to use for connection --print Print SQL statements to STDOUT --progress=a Print progress reports to STDERR while copying rows (default time,30) --quiet -q Do not print messages to STDOUT (disables -- progress) --recurse=i Number of levels to recurse in the hierarchy when discovering replicas --recursion-method=a Preferred recursion method for discovering replicas (default processlist,hosts) --remove-data-dir If the original table was created using the DATA DIRECTORY feature, remove it and create the new table in MySQL default directory without creating a new isl file (default no) --set-vars=A Set the MySQL variables in this comma- separated list of variable=value pairs --skip-check-slave-lag=d DSN to skip when checking slave lag --slave-password=s Sets the password to be used to connect to the slaves --slave-user=s Sets the user to be used to connect to the slaves --sleep=f How long to sleep (in seconds) after copying each chunk (default 0) --socket=s -S Socket file to use for connection --statistics Print statistics about internal counters --[no]swap-tables Swap the original table and the new, altered table (default yes) --tries=a How many times to try critical operations --[no]use-insert-ignore pt-online-schema-change by default use INSERT LOW_PRIORITY IGNORE statements to copy rows from the old table to the new one ( default yes) --user=s -u User for login if not current user --version Show version and exit --[no]version-check Check for the latest version of Percona Toolkit, MySQL, and other programs (default yes) Option types: s=string, i=integer, f=float, h/H/a/A=comma-separated list, d=DSN, z=size, m=time Rules: --dry-run and --execute are mutually exclusive. This tool accepts additional command-line arguments. Refer to the SYNOPSIS and usage information for details. DSN syntax is key=value[,key=value...] Allowable DSN keys: KEY COPY MEANING === ==== ============================================= A yes Default character set D yes Database for the old and new table F yes Only read default options from the given file P yes Port number to use for connection S yes Socket file to use for connection h yes Connect to host p yes Password to use when connecting t no Table to alter u yes User for login if not current user If the DSN is a bareword, the word is treated as the 'h' key. Options and values after processing arguments: --alter (No value) --alter-foreign-keys-method (No value) --analyze-before-swap TRUE --ask-pass FALSE --charset (No value) --check-alter TRUE --check-interval 1 --check-plan TRUE --check-replication-filters TRUE --check-slave-lag (No value) --chunk-index (No value) --chunk-index-columns (No value) --chunk-size 1000 --chunk-size-limit 4.0 --chunk-time 0.5 --config /etc/percona-toolkit/percona-toolkit.conf,/etc/percona-toolkit/pt-online-schema-change.conf,/root/.percona-toolkit.conf,/root/.pt-online-schema-change.conf --critical-load Threads_running=50 --data-dir (No value) --database (No value) --default-engine FALSE --defaults-file (No value) --drop-new-table TRUE --drop-old-table TRUE --drop-triggers TRUE --dry-run FALSE --execute FALSE --force FALSE --help TRUE --host (No value) --max-flow-ctl (No value) --max-lag 1 --max-load Threads_running=25 --new-table-name %T_new --null-to-not-null FALSE --password (No value) --pause-file (No value) --pid (No value) --plugin (No value) --port (No value) --print FALSE --progress time,30 --quiet FALSE --recurse (No value) --recursion-method processlist,hosts --remove-data-dir TRUE --set-vars --skip-check-slave-lag (No value) --slave-password (No value) --slave-user (No value) --sleep 0 --socket (No value) --statistics FALSE --swap-tables TRUE --tries (No value) --use-insert-ignore TRUE --user (No value) --version FALSE --version-check TRUE
参考资料
https://www.cnblogs.com/xiaoyanger/p/6043986.html