zoukankan      html  css  js  c++  java
  • Goldengate单向大事务复制性能测试

    Goldengate单向数据复制,为了图省事没有配pump,只配了source的extract和target的replicat;实际操作发现gg对大事务的支持还是比streams好一些,streams碰上大事务就只看到SPILL MESSAGES了:
    /*源端配置信息*/
    GGSCI (rh2.oracle.com) 1> view params mgr
    
    PORT 7809
    
    
    GGSCI (rh2.oracle.com) 2> view params load1
    
    
    --extract group--
    extract load1
    --connection to database--
    userid ggate, password ggate
    --hostname and port for trail--
    rmthost rh3.oracle.com, mgrport 7809
    --path and name for trail--
    rmttrail /s01/gg
    ddl include mapped objname sender.*
    table sender.*;
    
    GGSCI (rh2.oracle.com) 4> stats extract load1
    
    Sending STATS request to EXTRACT LOAD1 ...
    
    Start of Statistics at 2010-11-29 17:44:41.
    
    DDL replication statistics (for all trails):
    
    *** Total statistics since extract started     ***
            Operations                                   1.00
            Mapped operations                            1.00
            Unmapped operations                          0.00
            Other operations                             0.00
            Excluded operations                          0.00
    
    Output to /s01/gg:
    
    Extracting from GGATE.GGS_MARKER to GGATE.GGS_MARKER:
    
    *** Total statistics since 2010-11-29 16:22:25 ***
    
            No database operations have been performed.
    
    *** Daily statistics since 2010-11-29 16:22:25 ***
    
            No database operations have been performed.
    
    *** Hourly statistics since 2010-11-29 17:00:00 ***
    
            No database operations have been performed.
    
    *** Latest statistics since 2010-11-29 16:22:25 ***
    
            No database operations have been performed.
    
    Extracting from SENDER.ABC to SENDER.ABC:
    
    *** Total statistics since 2010-11-29 16:22:25 ***
            Total inserts                           2621440.00
            Total updates                                0.00
            Total deletes                                0.00
            Total discards                               0.00
            Total operations                        2621440.00
    
    *** Daily statistics since 2010-11-29 16:22:25 ***
            Total inserts                           2621440.00
            Total updates                                0.00
            Total deletes                                0.00
            Total discards                               0.00
            Total operations                        2621440.00
    
    *** Hourly statistics since 2010-11-29 17:00:00 ***
    
            No database operations have been performed.
    
    *** Latest statistics since 2010-11-29 16:22:25 ***
            Total inserts                           2621440.00
            Total updates                                0.00
            Total deletes                                0.00
            Total discards                               0.00
            Total operations                        2621440.00
    
    End of Statistics.
    
    GGSCI (rh2.oracle.com) 5> info all
    
    Program     Status      Group       Lag           Time Since Chkpt
    
    MANAGER     RUNNING                                           
    EXTRACT     RUNNING     LOAD1       00:00:00      00:00:08    
    
    /*目标端配置信息*/
    
    GSCI (rh3.oracle.com) 1> view params mgr
    
    PORT 7809
    USERID ggate, PASSWORD ggate
    PURGEOLDEXTRACTS /s01/gg
    
    
    GGSCI (rh3.oracle.com) 2> view params rep1
    
    --Replicat group --
    replicat rep1
    --source and target definitions
    ASSUMETARGETDEFS
    --target database login --
    userid ggate, password ggate
    --file for dicarded transaction --
    discardfile /s01/discard.txt, append, megabytes 10
    --ddl support
    DDL
    --Specify table mapping ---
    map sender.*, target receiver.*;
    
    GGSCI (rh3.oracle.com) 3> stats replicat rep1
    
    Sending STATS request to REPLICAT REP1 ...
    
    Start of Statistics at 2010-11-30 02:44:16.
    
    DDL replication statistics:
    
    *** Total statistics since replicat started     ***
            Operations                                   1.00
            Mapped operations                            1.00
            Unmapped operations                          0.00
            Other operations                             0.00
            Excluded operations                          0.00
            Errors                                       0.00
            Retried errors                               0.00
            Discarded errors                             0.00
            Ignored errors                               0.00
    
    Replicating from SENDER.ABC to RECEIVER.ABC:
    
    *** Total statistics since 2010-11-30 01:21:09 ***
            Total inserts                           2621440.00
            Total updates                                0.00
            Total deletes                                0.00
            Total discards                               0.00
            Total operations                        2621440.00
    
    *** Daily statistics since 2010-11-30 01:21:09 ***
            Total inserts                           2621440.00
            Total updates                                0.00
            Total deletes                                0.00
            Total discards                               0.00
            Total operations                        2621440.00
    
    *** Hourly statistics since 2010-11-30 02:00:00 ***
    
            No database operations have been performed.
    
    *** Latest statistics since 2010-11-30 01:21:09 ***
            Total inserts                           2621440.00
            Total updates                                0.00
            Total deletes                                0.00
            Total discards                               0.00
            Total operations                        2621440.00
    
    End of Statistics.
    
    GGSCI (rh3.oracle.com) 4> info all
    
    Program     Status      Group       Lag           Time Since Chkpt
    
    MANAGER     RUNNING                                           
    REPLICAT    RUNNING     REP1        00:00:00      00:00:01   
    
    接着尝试在源库端sender模式下产生一个大事务操作,观察目标段数据同步情况*/
    SQL> delete sender.abc;
    2621440 rows deleted.
    
    SQL> commit;
    Commit complete.
    /* commit操作完成前,EXTRACT进程并不会抽取日志 */
    
    Goldengate日志ggserror.log的相关记录:
    010-11-29 17:49:49  INFO    OGG-01026  Oracle GoldenGate Capture for Oracle, load1.prm:  Rolling over remote file /s01/gg000029.
    2010-11-29 17:49:50  INFO    OGG-01026  Oracle GoldenGate Capture for Oracle, load1.prm:  Rolling over remote file /s01/gg000030.
    2010-11-29 17:49:51  INFO    OGG-01026  Oracle GoldenGate Capture for Oracle, load1.prm:  Rolling over remote file /s01/gg000031.
    2010-11-29 17:49:53  INFO    OGG-01026  Oracle GoldenGate Capture for Oracle, load1.prm:  Rolling over remote file /s01/gg000032.
    2010-11-29 17:49:54  INFO    OGG-01026  Oracle GoldenGate Capture for Oracle, load1.prm:  Rolling over remote file /s01/gg000033.
    2010-11-29 17:49:55  INFO    OGG-01026  Oracle GoldenGate Capture for Oracle, load1.prm:  Rolling over remote file /s01/gg000034.
    2010-11-29 17:49:56  INFO    OGG-01026  Oracle GoldenGate Capture for Oracle, load1.prm:  Rolling over remote file /s01/gg000035.
    2010-11-29 17:49:57  INFO    OGG-01026  Oracle GoldenGate Capture for Oracle, load1.prm:  Rolling over remote file /s01/gg000036.
    2010-11-29 17:49:58  INFO    OGG-01026  Oracle GoldenGate Capture for Oracle, load1.prm:  Rolling over remote file /s01/gg000037.
    2010-11-29 17:49:59  INFO    OGG-01026  Oracle GoldenGate Capture for Oracle, load1.prm:  Rolling over remote file /s01/gg000038.
    2010-11-29 17:50:00  INFO    OGG-01026  Oracle GoldenGate Capture for Oracle, load1.prm:  Rolling over remote file /s01/gg000039.
    2010-11-29 17:50:01  INFO    OGG-01026  Oracle GoldenGate Capture for Oracle, load1.prm:  Rolling over remote file /s01/gg000040.
    2010-11-29 17:50:03  INFO    OGG-01026  Oracle GoldenGate Capture for Oracle, load1.prm:  Rolling over remote file /s01/gg000041.
    2010-11-29 17:50:04  INFO    OGG-01026  Oracle GoldenGate Capture for Oracle, load1.prm:  Rolling over remote file /s01/gg000042.
    2010-11-29 17:50:05  INFO    OGG-01026  Oracle GoldenGate Capture for Oracle, load1.prm:  Rolling over remote file /s01/gg000043.
    2010-11-29 17:50:06  INFO    OGG-01026  Oracle GoldenGate Capture for Oracle, load1.prm:  Rolling over remote file /s01/gg000044.
    2010-11-29 17:50:07  INFO    OGG-01026  Oracle GoldenGate Capture for Oracle, load1.prm:  Rolling over remote file /s01/gg000045.
    2010-11-29 17:50:08  INFO    OGG-01026  Oracle GoldenGate Capture for Oracle, load1.prm:  Rolling over remote file /s01/gg000046.
    2010-11-29 17:50:09  INFO    OGG-01026  Oracle GoldenGate Capture for Oracle, load1.prm:  Rolling over remote file /s01/gg000047.
    2010-11-29 17:50:10  INFO    OGG-01026  Oracle GoldenGate Capture for Oracle, load1.prm:  Rolling over remote file /s01/gg000048.
    2010-11-29 17:50:11  INFO    OGG-01026  Oracle GoldenGate Capture for Oracle, load1.prm:  Rolling over remote file /s01/gg000049.
    2010-11-29 17:50:13  INFO    OGG-01026  Oracle GoldenGate Capture for Oracle, load1.prm:  Rolling over remote file /s01/gg000050.
    2010-11-29 17:50:14  INFO    OGG-01026  Oracle GoldenGate Capture for Oracle, load1.prm:  Rolling over remote file /s01/gg000051.
    2010-11-29 17:50:15  INFO    OGG-01026  Oracle GoldenGate Capture for Oracle, load1.prm:  Rolling over remote file /s01/gg000052.
    2010-11-29 17:50:16  INFO    OGG-01026  Oracle GoldenGate Capture for Oracle, load1.prm:  Rolling over remote file /s01/gg000053.
    2010-11-29 17:50:17  INFO    OGG-01026  Oracle GoldenGate Capture for Oracle, load1.prm:  Rolling over remote file /s01/gg000054.
    
    /*产生了26个trail文件*/
    
    /*同时备库端开始陆续应用日志*/
    REP1.rpt是rep1 replicat复制进程的文本报告,出现相关内容:
    Switching to next trail file /s01/gg000029 at 2010-11-30 02:47:54 due to EOF, with current RBA 9999949
    Opened trail file /s01/gg000029 at 2010-11-30 02:47:54
    
    Switching to next trail file /s01/gg000030 at 2010-11-30 02:49:53 due to EOF, with current RBA 9999925
    Opened trail file /s01/gg000030 at 2010-11-30 02:49:53
    
    GGSCI (rh3.oracle.com) 6> stats replicat rep1
    
    Sending STATS request to REPLICAT REP1 ...
    
    Start of Statistics at 2010-11-30 02:52:20.
    
    DDL replication statistics:
    
    *** Total statistics since replicat started     ***
            Operations                                   1.00
            Mapped operations                            1.00
            Unmapped operations                          0.00
            Other operations                             0.00
            Excluded operations                          0.00
            Errors                                       0.00
            Retried errors                               0.00
            Discarded errors                             0.00
            Ignored errors                               0.00
    
    Replicating from SENDER.ABC to RECEIVER.ABC:
    
    *** Total statistics since 2010-11-30 01:21:09 ***
            Total inserts                           2621440.00
            Total updates                                0.00
            Total deletes                           208150.00
            Total discards                               0.00
            Total operations                        2829590.00
    
    *** Daily statistics since 2010-11-30 01:21:09 ***
            Total inserts                           2621440.00
            Total updates                                0.00
            Total deletes                           208150.00
            Total discards                               0.00
            Total operations                        2829590.00
    
    *** Hourly statistics since 2010-11-30 02:00:00 ***
            Total inserts                                0.00
            Total updates                                0.00
            Total deletes                           208150.00
            Total discards                               0.00
            Total operations                        208150.00
    
    *** Latest statistics since 2010-11-30 01:21:09 ***
            Total inserts                           2621440.00
            Total updates                                0.00
            Total deletes                           208150.00
            Total discards                               0.00
            Total operations                        2829590.00
    
    End of Statistics.
    
    
    GGSCI (rh3.oracle.com) 7> !
    stats replicat rep1
    
    Sending STATS request to REPLICAT REP1 ...
    
    Start of Statistics at 2010-11-30 02:52:26.
    
    DDL replication statistics:
    
    *** Total statistics since replicat started     ***
            Operations                                   1.00
            Mapped operations                            1.00
            Unmapped operations                          0.00
            Other operations                             0.00
            Excluded operations                          0.00
            Errors                                       0.00
            Retried errors                               0.00
            Discarded errors                             0.00
            Ignored errors                               0.00
    
    Replicating from SENDER.ABC to RECEIVER.ABC:
    
    *** Total statistics since 2010-11-30 01:21:09 ***
            Total inserts                           2621440.00
            Total updates                                0.00
            Total deletes                           210767.00
            Total discards                               0.00
            Total operations                        2832207.00
    
    *** Daily statistics since 2010-11-30 01:21:09 ***
            Total inserts                           2621440.00
            Total updates                                0.00
            Total deletes                           210767.00
            Total discards                               0.00
            Total operations                        2832207.00
    
    *** Hourly statistics since 2010-11-30 02:00:00 ***
            Total inserts                                0.00
            Total updates                                0.00
            Total deletes                           210767.00
            Total discards                               0.00
            Total operations                        210767.00
    
    *** Latest statistics since 2010-11-30 01:21:09 ***
            Total inserts                           2621440.00
            Total updates                                0.00
            Total deletes                           210767.00
            Total discards                               0.00
            Total operations                        2832207.00
    
    End of Statistics.
    
    
    /*由以上对replicat进程的统计信息可知其正在应用delete操作*/
    
    Switching to next trail file /s01/gg000031 at 2010-11-30 02:53:26 due to EOF, with current RBA 9999925
    Opened trail file /s01/gg000031 at 2010-11-30 02:53:26
    
    Switching to next trail file /s01/gg000032 at 2010-11-30 02:58:28 due to EOF, with current RBA 9999925
    Opened trail file /s01/gg000032 at 2010-11-30 02:58:28
    
    [maclean@rh3 s01]$ ls -lh gg00003[1-2]
    -rw-rw-rw- 1 maclean oinstall 9.6M Nov 30 02:47 gg000031
    -rw-rw-rw- 1 maclean oinstall 9.6M Nov 30 02:47 gg000032
    
    /*由上列trace信息推断,以这样一台双核cpu主频为2.2GHZ的pc机为例,gg应用一个大小为9.6M的trail文件耗时也要将近5分钟(当然这并不准确)。
    
    
    /*从进程argument可以看到replicate和extract进程的本质是调用了多个配置文件的$GGATE/extract和$GGATE/replicat*/
    [maclean@rh3 ~]$ ps -ef|grep repl
    maclean   7817  7476  3 01:18 ?        00:03:29 
    /home/maclean/gg/replicat PARAMFILE /home/maclean/gg/dirprm/rep1.prm 
    REPORTFILE /home/maclean/gg/dirrpt/REP1.rpt 
    PROCESSID REP1 USESUBDIRS
    
    [maclean@rh2 dirrpt]$ ps -ef|grep ex
    maclean    544 32432  1 16:21 ?        00:01:12 
    /home/maclean/gg/extract PARAMFILE /home/maclean/gg/dirprm/load1.prm 
    REPORTFILE /home/maclean/gg/dirrpt/LOAD1.rpt 
    PROCESSID LOAD1 USESUBDIRS
    
  • 相关阅读:
    “大型售票系统”和“真正的电商系统”在“库存”巨大的差异计算
    c#有关udp可靠传输(包传输数据包) 升级
    CreateCompatibleDC工作原理
    Android 内存管理 &Memory Leak & OOM 分析
    ubuntu文本界面乱码的中国解决方案
    hdu 4972 A simple dynamic programming problem(高效)
    java定义和实现接口
    HTTP 404
    pcre函数具体解释
    Android项目开发五-《星星生活志》1.使用MediaRecorder录制音频
  • 原文地址:https://www.cnblogs.com/macleanoracle/p/2967590.html
Copyright © 2011-2022 走看看