zoukankan      html  css  js  c++  java
  • Oracle GoldenGate常用参数

      OGG(Oracle GoldenGate)参数介绍

    所有的GoldenGate进程均有参数文件
        Manager
        Extract
        Replicat
        Utilities
    
    所有参数均有缺省配置
        实际应用只需对小部分参数进行配置
    
    所有参数文件均放在 ./dirprm目录下
        缺省通过进程名进行查找

    一、全局参数

    MGRSERVNAME         Specifies the name of the Manager process when it is installed as a Windows service.
    
    CHECKPOINTTABLE     Specifies a default checkpoint table.
    
    GGSCHEMA            Specifies the name of the schema that contains the database objects that support DDL synchronization for Oracle.
    
    DDLTABLE            Specifies a non-default name for the DDL history table that supports DDL synchronization for Oracle.
    
    MARKERTABLE         Specifies a non-default name for the DDL marker table that supports DDL synchronization for Oracle.
    
    OUTPUTFILEUMASK     Specifies a umask that can be used by Oracle GoldenGate processes to create trail files and discard files.
    
    SYSLOG              Filters the types of Oracle GoldenGate messages that are written to the system logs.
    
    UNLOCKEDTRAILFILES     Workaround for problems with trail files that remain locked by Collector if there is a network outage that affects the target system.

    二、Manager进程相关参数

    1、常用参数:

    PORT            制定GoldenGate的mgr进程使用哪个 TCP/IP端口侦听请求.
                    示例:PORT 7809
    
    DYNAMICPORTLIST 指定GoldenGate可以使用那些端口接受extract发送过来的数据.
                    示例:DYNAMICPORTLIST 7840-7850
               
    AUTOSTART       指定在mgr启动时自动启动那些进程.
                    示例:AUTOSTART ER *
                         AUTOSTART extract extsz
     
    AUTORESTART     指定在mgr可以定时重启那些进程。可以在网络中断等故障恢复后自动重起,避免人工干预.
                    示例:AUTORESTART ER *, WAITMINUTES 5, RETRIES 3  --每隔5分钟尝试启动一次,尝试3次
                   
    LAGREPORT       指定在ggserr.log中报告延迟的时间间隔.
                    示例:LAGREPORTHOURS 1       --每隔一小时检查一次传输延迟情况
                    
    LAGCRITICAL     指定认为超过此时间即为严重错误的延迟最大值,如果延迟达到此时间值则会在ggserr.log里面写入一条error信息
                    示例:LAGCRITICALMINUTES 45  --传输延时超过45分钟将写入警告日志
    
    LAGINFO         指定一个延迟时限,如果延迟达到此时间值则会在ggserr.log里面写入一条info信息
                    示例:LAGINFOMINUTES 30      --传输延时超过30分钟将写入错误日志                
    
    PURGEOLDEXTRACTS 定义自动删除过时的队列以节省硬盘空间。
    一般按照两个规则来删除:首先,要满足检查点要求,没有使用过的队列不能删除,保证无数据丢失;其次,可以保留一定的天数。只有当已经使用过且超过设定的天数后的队列会被自动删除。示例:purgeoldextracts
    /backup/goldengate/dirdat/*,usecheckpoints, minkeepdays 7

    2、完整示例:

    源端Manager配置参数示例:

    GGSCI> view params mgr
    
    PORT 7839                         --通信端口7839。 源端和目标端需要保持一致。
    DYNAMICPORTLIST  7840-7914        --动态端口列表的范围从7840到7914。当制定端口被占用或者出现通信故障,管理进程将会从列表中选择下一个端口尝试连接,避免通信端口的单点故障。
    AUTORESTART EXTRACT *, RETRIES 5, WAITMINUTES 7  --当提取进程中断后尝试自动重启,每隔7分钟尝试启动一次,尝试5次。
    PURGEOLDEXTRACTS ./dirdat/*, USECHECKPOINTS, MINKEEPDAYS 10  --定期清理dirdat路径下的本地队列(local trail)。保留期限10天,过期后自动删除。从而控制队列文件的目录不会增长过大。
    LAGREPORTHOURS 1                  --每隔一小时检查一次传输延迟情况
    LAGINFOMINUTES 30                 --传输延时超过30分钟将写入错误日志
    LAGCRITICALMINUTES 45             --传输延时超过45分钟将写入警告日志

    目标端Manager配置参数示例:

    GGSCI 4> view params mgr
    
    PORT 7839   --通信端口7839。 生产端和灾备端需要保持一致
    
    USERID goldengate, PASSWORD YGGJBFJGIH, ENCRYPTKEY default  
    DYNAMICPORTLIST 7840-7914  --动态端口列表的范围从7840到7914。当制定端口被占用或者出现通信故障,管理进程将会从列表中选择下一个端口尝试连接,避免通信端口的单点故障。
    AUTORESTART EXTRACT *,RETRIES 5,WAITMINUTES 3  --当提取进程中断后尝试自动重启,每隔7分钟尝试启动一次,尝试5次。
    PURGEOLDEXTRACTS ./dirdat/*,usecheckpoints, minkeepdays 5 --定期清理dirdat路径下的远程队列(remote trail)。保留期限5天,过期后自动删除。从而控制队列文件的目录不会增长过大。
    --PURGEDDLHISTORY MINKEEPDAYS 7, MAXKEEPDAYS 10     --删除DDL历史表,最小保存7天,最大保存10天。由于此项目没有开启DDL,所以该参数被注释。
    --PURGEMARKERHISTORY MINKEEPDAYS 7, MAXKEEPDAYS 10  --删除MARKER历史表,最小保存7天,最大保存10天。此项目没有使用该参数,所以该参数被注释。
    LAGREPORTHOURS 1         --每隔一小时检查一次传输延迟情况
    LAGINFOMINUTES 30        --传输延时超过30分钟将写入错误日志
    LAGCRITICALMINUTES 45   --传输延时超过45分钟将写入警告日志

    参考:真实环境的OracleGoldenGate配置参数详解 -- http://blog.itpub.net/29475508/viewspace-1297083/

    3、完整可选参数列表

    (1)通用参数:

    COMMENT | --  Allows insertion of comments in a parameter file.##允许在参数文件中插入注释信息
    SYSLOG     Filters the types of Oracle GoldenGate messages that are written to the system logs.

    (2)端口管理参数:

    DYNAMICPORTLIST             Specifies the ports that Manager can dynamically allocate.
                                ##指定Manager可以动态分配的端口列表(这些端口将用来接受extract发送过来的数据)
                                示例:DYNAMICPORTLIST 7840-7850
                                
                                
    DYNAMICPORTREASSIGNDELAY    Specifies a time to wait before assigning a port number that was previously assigned.
                                ##当要分配的端口被占用时,指定在再次分配前需要等待的时间
                                
    PORT                        Establishes the TCP/IP port number on which Manager listens for requests.
                                ##指定Manager监听请求所使用的TCP/IP端口号,示例:PORT 7809

    (3)进程管理参数:

    AUTORESTART         Specifies processes to be restarted by Manager after a failure.
                        ##指定Manager可(定时)重启哪些进程(可以在网络中断等故障恢复后自动重起,避免人工干预)
                        示例:AUTORESTART ER *, WAITMINUTES 5, RETRIES 3  --每隔5分钟尝试启动一次,尝试3次
                        
    AUTOSTART           Specifies processes to be started when Manager starts.
                        ##指定Manager进程启动时,自动启动哪些进程
                        示例:AUTOSTART ER *
                             AUTOSTART extract extsz
    
    BOOTDELAYMINUTES    Determines how long after system boot time Manager delays until performing main processing activities. This parameter supports Windows.
    
    UPREPORT            Determines how often process heartbeat messages are reported.                            
                        ##确定心跳检测信息多久报告一次

    (4)事件管理参数:

    DOWNCRITICAL        Reports processes that stopped gracefully or abnormally.
    
    DOWNREPORT          Controls the frequency for reporting stopped processes.
    
    LAGCRITICAL         Specifies a lag threshold that is considered critical and generates a warning to the error log.
                        指定认为超过此时间即为严重错误的延迟最大值,如果延迟达到此时间值则会在ggserr.log里面写入一条error信息
                        示例:LAGCRITICALMINUTES 45  --传输延时超过45分钟将写入警告日志
                        
    LAGINFO             Specifies a lag threshold at which an informational message is reported to the error log.
                        指定一个延迟时限,如果延迟达到此时间值则会在ggserr.log里面写入一条info信息
                        示例:LAGINFOMINUTES 30      --传输延时超过30分钟将写入错误日志
                        
    LAGREPORT           Sets an interval for reporting lag time to the error log
                        指定在ggserr.log中报告延迟的时间间隔.
                        示例:LAGREPORTHOURS 1       --每隔一小时检查一次传输延迟情况

    (5)数据库登陆:

    SOURCEDB            Specifies a data source name as part of the login information.
                        示例:SOURCEDB mydb
                              SOURCEDB mydb, USERID ggs, PASSWORD ggs123
                              
    USERID              Provides login information for Manager when it needs to access the database.
                        示例:USERID /
                              USERID ggs, PASSWORD ggs123
                              USERID ggs@ora1.ora, PASSWORD ggs123
                              USERID data1
                              USERID ggs, PASSWORD AACAAAAAAAAAAAIALCKDZIRHOJBHOJUH, ENCRYPTKEY superx128
                              USERID ggs, PASSWORD AACAAAAAAAAAAAIALCKDZIRHOJBHOJUH, ENCRYPTKEY default

    (6)维护相关参数:

    CHECKMINUTES                    Determines how often Manager cycles through maintenance activities.
    
    PURGEDDLHISTORY                 Purges rows from the Oracle DDL history table when they are no longer needed.
    
    PURGEMARKERHISTORY              Purges Oracle marker table rows that are no longer needed.
    
    PURGEOLDEXTRACTS                Purges trail data that is no longer needed.##自动删除过时的队列以节省硬盘空间。
                                    一般按照两个规则来删除:
                                        首先,要满足检查点要求,没有使用过的队列不能删除,保证无数据丢失;
                                        其次,可以保留一定的天数。只有当已经使用过且超过设定的天数后的队列会被自动删除。
                                    示例:purgeoldextracts /backup/goldengate/dirdat/*,usecheckpoints, minkeepdays 7
                                    
    PURGEOLDTASKS                     Purges Extract and Replicat tasks after a specified period of time.
    
    STARTUPVALIDATIONDELAY[CSECS]     Sets a delay time after which Manager checks that processes are still running after startup.

     参考:Oracle GoldenGate Windows and UNIX Reference Guide 11g Release 1(11.1.1) E17791-01

    Oracle Golden Gate 系列八 -- GG 参数文件 说明 http://blog.csdn.net/tianlesoftware/article/details/6970183

    二、extract进程相关参数

    1、常用参数

    抽取进程有以下方面参数:

    Checkpoint
    复制目标
        Local System
        Multiple Remote Systems
        One to many GoldenGate Trails
    数据源
        Which Tables
        Which Rows and Columns
        Which Operations
    数据影射和变换

    table参数:

    Syntax:
    TABLE <file name>
    [, WHERE (<where condition>)]
    [, FILTER (<expression>)]
    [, KEYCOLS (<key column specification>)]
    [, COLS (<column>) | [ , <column> ]) ]
    [, COLSEXCEPT (<column>) | [ , <column> ]) ]
    [, FETCHCOLS (<column>) | [ , <column> ]) ]
    [, FETCHCOLSEXCEPT (<column>) | [ , <column> ]) ]
    [, SQLEXEC <clause> ]
    [, FETCHBEFOREFILTER ]
    [, SQLPREDICATE “WHERE <clause>” ]
    [, COLMAP (<column mapping specification>)]
    [, <record type filter>, <record type filter>, …]
    ;

    注意:TABLE的结尾有一个分号。

    2、完整示例

    参数文件举例:日志抽取进程

    extract extsz
    userid goldengate, password *******
    --REPORT AT 01:59
    --reportrollover at 02:00
    --transmemory directory (/backup/goldengate/dirtmp,8G,4G),ram 2G,transram 500M
    tranlogoptions rawdeviceoffset 0 
    --warnlongtrans 12h, checkintervals 30m
    exttrail /backup/goldengate/dirdat/sz
    
    dynamicresolution
    numfiles 2000
    
    tableexclude ctais2.TMP_*;
    tableexclude ctais2.BAK_*;
    tableexclude ctais2.MLOG$_*;
    tableexclude ctais2.RUPD$_*;
    tableexclude ctais2.KJ_*;
    
    table ctais2.*;

    参数文件举例:传输进程

    extract dpesz
    setenv ( NLS_LANG = AMERICAN_AMERICA.ZHS16GBK )
    passthru
    REPORT AT 01:59
    reportrollover at 02:00
    transmemory directory (/backup/goldengate/dirtmp,8G,4G), ram 2G,transram        500M
    --threadoptions iolatencydelay 3000
    rmthost  99.16.1.12,mgrport 7809, compress     
    rmttrail /oradata/goldengate/dirdat/sz 
    dynamicresolution
    numfiles 3000
    table ctais2.*;

    示例3:

    extract extl                              --抽取进程名
    userid ddw,password ddw                   --连接本机DB的帐号密码
    rmthost 192.168.0.44, mgrport 7801        --目标数据库服务器地址和GG服务端口号
    rmttrail d:	oolsGGgg10gdirdat
    l      --远程队列的位置
    dynamicresolution                         --优化参数,动态分析表结构
    gettruncates                              --抓取truncate数据
    table ddw.aatest;                         --需要抽取的表,可以使用通配符

    3、全部可选参数

    (1)通用参数

    CHECKPARAMS         Verifies parameter file syntax. 
                        如果加入该参数,表示下次运行只是检查一下语法,并不实际运行进程。 注意:该命令只能检查一些简单语法,并不能保证参数文件是完全正确的。
    
    COMMENT | --        Denotes comments in a parameter file.  ##注释行,也可以用两个中划线--代替.--checkparams表示本行已经被注释掉
    
    ETOLDFORMAT         Generates trails in a format that is compatible with Replicat versions prior to Oracle GoldenGate version 6.0.
    
    GETENV              Retrieves variables that were set with the  SETENV parameter.
    SETENV              Specifies a value for a UNIX environment variable from within the GGSCI interface.
                        针对extract进程设定系统环境变量。
                        示例:SETENV (NLS_LANG="AMERICAN_AMERICA.UTF8")    --设置字符集环境变量为UTF8
    OBEY Processes parameter statements contained in a different parameter file. 可以将外部文件的内容包含到本参数文件中来。用于将一些可以重复利用的参数文件部分内容隔离出来,便于统一修改。Obey tables.txt RECOVERYOPTIONS Controls the recovery mode of the Extract process. TCPSOURCETIMER |
    NOTCPSOURCETIMER Adjusts timestamps of records transferred to other systems when those systems reflect different times. TRACETABLE |
    NOTRACETABLE Causes Extract to ignore database changes generated by Replicat during bidirectional synchronization. Supports Oracle

    (2)处理方法Processing method

    BEGIN             Specifies when to begin a processing run.
    
    DSOPTIONS         Specifies Extract processing options when a Teradata Access Module (TAM) is being used.
    
    END               Specifies when to end a processing run.
    
    EXTRACT           Defines an Extract group as an online process. ##定义抽取进程的名字
    
    GETAPPLOPS | 
    IGNOREAPPLOPS Controls whether or not operations from all processes except Replicat are written to a trail or file. GETREPLICATES |
    IGNOREREPLICATES Controls whether or not replicated operations are captured by an Extract on the same system.
    PASSTHRU | Controls whether tables will be processed by a data-pump Extract in pass-through mode or whether data definitions will be required NOPASSTHRU 禁止extract与数据库交互,适用于Data Pump传输进程(dpeXX)。
             说明:PASSTHRU直通模式用在两边表名、列名一致,可以直接映射的情况,不需要额外配置;
              NOPASSTHRU普通模式可以配置表名列名自定义映射,可以加FILTER、transformation等,需要配置一个数据定义文件(data-definitions file)
    RMTTASK                Creates a processing task on a remote system.
    
    SOURCEISTABLE          Extracts entire records from source tables.
    
    SPECIALRUN             Specifies a one-time processing task that does not checkpoint from run to run.
    
    VAM                    Indicates that a Teradata Access Module (TAM) is being used to provide transactional data to the Extract process.

    (3)数据库登陆Database login

    SOURCEDB           Specifies the data source as part of the login information.
    
    USERID             Specifies database connection information.
    
    [SOURCEDB,] USERID ,PASSWORD    指定所要登陆的数据库名称,用户名和密码。对于oracle无需指定sourcedb,直接指定用户名和密码即可。
                                    Userid goldengate, password goldengate

    (4)数据匹配和映射 Selecting and mapping data

    ASCIITOEBCDIC        Converts ASCII text to EBCDIC for DB2 on z/OS systems running UNIX System Services.
    
    COLMATCH             Establishes global column-mapping rules.
    
    COMPRESSDELETES |
    NOCOMPRESSDELETES Controls whether Oracle GoldenGate writes only the key or all columns to the trail for delete operations. COMPRESSUPDATES |
    NOCOMPRESSUPDATES Causes only primary key columns and changed columns to be logged for updates. DDL Enables and filters the capture of DDL operations. DDLSUBST Enables string substitution in DDL processing. FETCHOPTIONS Controls certain aspects of the way that Oracle GoldenGate fetches data. 示例:FETCHOPTIONS NOUSESNAPSHOT --不会从闪回日志中获取数据 FETCHOPTIONS FETCHPKUPDATECOLS --当使用了 HANDLECOLLISIONS 时,请使用该参数。复制进程出现丢失update记录(missing update)并且更新的是主键,update将转换成insert。由于插入的记录可能不是完整的行,若要保证完整需要加入此参数 GETDELETES | IGNOREDELETES Controls the extraction of delete operations.是否复制delete操作,缺省复制 GETINSERTS | IGNOREINSERTS Controls the extraction of insert operations.是否复制insert操作,缺省复制 GETTRUNCATES |IGNORETRUNCATES Controls the extraction of truncate statements. 是否复制truncate操作,缺省不复制. GETUPDATEAFTERS | IGNOREUPDATEAFTERS Controls the extraction of update after images.##是否在队列中写入后影像,缺省复制 GETUPDATEBEFORES |IGNOREUPDATEBEFORES Controls the extraction of update before images.是否在队列中写入前影像,缺省不复制 GETUPDATES | IGNOREUPDATES Controls the extraction of update operations 是否复制update操作,缺省复制 REPLACEBADCHAR Replaces invalid character values with another value. SEQUENCE Specifies sequences for synchronization. TABLE for Extract Specifies tables for extraction and controls column mapping and conversion. TABLEEXCLUDE Excludes tables from the extraction process. 定义所需要排除的表。如果在table里面定义了使用通配符,那么可以使用该参数定义排除掉其中的部分表。如: tableexclude ctais2.TMP_*; tableexclude ctais2.TEMPTAB; TARGETDEFS Specifies a file containing target table definitions for target databases that reside on the NonStop platform. TRIMSPACES |
    NOTRIMSPACES Controls whether trailing spaces are trimmed or not when mapping CHAR to VARCHAR columns. WILDCARDRESOLVE Defines rules for processing wildcard table specifications in a TABLE statement.

    (5)数据转发 Routing data

    EXTFILE         Specifies an extract file to which extracted data is written on the local system.
    
    EXTTRAIL        Specifies a trail to which extracted data is written on the local system.
                    指定写入到本地的哪个队列。
                    示例:EXTTRAIL ./dirdat/ya       --本地队列文件路径
    
    RMTFILE         Specifies an extract file to which extracted data is written on a remote system.
    
    RMTHOST         Specifies the target system and Manager port number.
                    指定目标系统及其GoldenGate Manager进程的端口号,也用于定义是否使用压缩进行传输。
                    示例:rmthost  99.16.1.12,mgrport 7809, compress
    
    RMTTRAIL        Specifies a trail to which extracted data is written on a remote system.
                    指定写入到目标端的哪个队列

    (6)数据格式化Formatting data

    FORMATASCII       Formats extracted data in external ASCII format.
    
    FORMATSQL         Formats extracted data into equivalent SQL statements.
    
    FORMATXML         Formats extracted data into equivalent XML syntax.
    
    NOHEADERS         Prevents record headers from being written to the trail.

    (7)自定义处理 Custom processing 

    CUSEREXIT           Invokes a user exit routine during processing.
    
    INCLUDE             Invokes a macro library.
    
    MACRO               Defines an Oracle GoldenGate macro.
    
    MACROCHAR           Defines a macro character other than the default of #.
    
    SQLEXEC             Executes a stored procedure or query during Extract processing.
                        在extract运行时首先运行一个sql语句。 示例:sqlexec "Alter session set constraints=deferred"
    
    VARWIDTHNCHAR | 
    NOVARWIDTHNCHAR Controls whether length information is written to the trail for NCHAR columns.

    (8)报告信息Reporting

    CMDTRACE            Displays macro expansion steps in the Extract report file.
    
    LIST | NOLIST       Displays or suppresses the listing of macros in the report file.
    
    REPORT              Schedules a statistical report.
                        定义自动定时报告。REPORT AT 01:59
    
    STATOPTIONS         Specifies information to include in statistical displays.
                        定义每次使用stat时统计数字是否需要重置
    
    REPORTCOUNT         Reports the number of records processed.报告已经处理的纪录条数统计数字。
                        示例:REPORTCOUNT EVERY 30 MINUTES, RATE   
                        --每隔30分钟报告一次从程序开始到现在的抽取进程或者复制进程的事物记录数,并汇报进程的统计信息
    
    TRACE/TRACE2        Shows Extract processing information to assist in revealing processing bottlenecks.
                        打开对GoldenGate进程的跟踪日志,一般用于调试。
    
    WARNLONGTRANS       Defines a long-running transaction and controls the frequency of checking for and reporting them
                        指定对于超过一定时间的长交易可以在ggserr.log里面写入警告信息。例如,每隔30分钟检查一次长交易,对于超过12个小时的进行告警:
                        示例:warnlongtrans 12h, checkintervals 30m
                             WARNLONGTRANS 2h, CHECKINTERVAL 3m   --每隔3分钟检查一下大事务,超过2小时还没结束的进行报告

    (9)错误处理Error handling

    DDLERROR            Controls error handling for DDL extraction.
    
    DISCARDFILE         Contains records that could not be processed.
                        定义discard文件位置,如果处理中有纪录出错会写入到此文件中。
                        示例1:discardfile /oradata/goldengate/repkj.dsc,append,megabytes 100m
                        示例2:DISCARDFILE ./dirrpt/extya.dsc, APPEND, MEGABYTES 1024  
                        --将执行失败的记录保存在discard file中,该文件位于./dirrpt/extya.dsc,大小为1024MB。 文件中已经包含记录的话,再后面继续追加,不删除之前的记录

    (10)协调Tuning

    ALLOCFILES           Controls the number of incremental memory structures allocated when the value of  NUMFILES is reached.
    
    BR                   Controls the Bounded Recovery feature of Extract.
    
    CACHEMGR             Controls the virtual memory cache manager. 
                         示例:CACHEMGR CACHESIZE 8000MB, CACHEDIRECTORY  /goldengate/temp, CACHEDIRECTORY /goldengate/temp2
                              在/goldengate/temp,/goldengate/temp2 目录设置虚拟内存
    
    CHECKPOINTSECS       Controls how often Extract writes a checkpoint.
    
    DBOPTIONS            Specifies database options.指定对于某种特定数据库所需要的特殊参数
                         示例:DBOPTIONS  ALLOWUNUSEDCOLUMN  
                         --当抽取进程遇到一个没有使用的字段时只生成一个警告,进程会继续执行而不会被异常终止(abend)
    
    DDLOPTIONS           Specifies DDL processing options.
    
    DYNAMICRESOLUTION | 
    NODYNAMICRESOLUTION  Suppresses the metadata lookup for a table until Extract encounters transactional data for it. Makes Extract start faster when there are numerous tables specified for synchronization.
                         示例:DYNAMICRESOLUTION  --有时候开启OGG进程的时候较慢,可能是因为需要同步的表太多,OGG在开启进程之前会将需要同步的表建立一个记录并且存入到磁盘中,这样就需要耗费大量的时间。使用该参数来解决此问题。
    
    EOFDELAY | EOFDELAYCSECS      Determines how long Extract delays before searching for more data to process.
    
    FLUSHSECS | FLUSHCSECS        Determines the amount of time that record data remains buffered before being written to the trail.
    
    FUNCTIONSTACKSIZE             Controls the size of the memory stack that is used for processing Oracle GoldenGate functions.
    
    GROUPTRANSOPS                 Controls the number of records that are sent to the trail in one batch.
    
    LOBMEMORY                     Controls the amount of memory and temporary disk space available for caching transactions that contain LOB s.
    
    MAXFETCHSTATEMENTS            Controls the maximum number of prepared queries that Extract can use to fetch data from the database.
    
    NUMFILES              Controls the initial allocation of memory dedicated to storing information about tables to be processed by Oracle GoldenGate.
                          定义本extract为最大多少张表预留空间,缺省为500,超过500张表需要设定一个比实际表数略大的值。
                          示例:numfiles 3000
    
    RMTHOSTOPTIONS        Specifies connection attributes other than host information for a TCP/IP connection used by a passive Extract group.
    
    THREADOPTIONS         Controls aspects of the way that Extract operates in an Oracle Real Application Cluster environment. Supports Oracle.
                          示例:THREADOPTIONS   MAXCOMMITPROPAGATIONDELAY 60000 IOLATENCY 60000
                          RAC集群中的所有节点必须同步系统时钟。GoldenGate通过比较本地系统的时间和事务提交的时间点来做出关键决策。可以通过NTP来不同系统时间。所有节点上的COMPATIBLE参数设置也必须相同。注:这个参数在Oracle11.2版本后就不再使用了。
    
    TRANLOGOPTIONS     Supplies log-processing options.
                       指定在解析数据库日志时所需要的特殊参数。例如,对于裸设备,可能需要加入下列参数:
                       示例1: tranlogoptions rawdeviceoffset 0 
                       示例2:TRANLOGOPTIONS  CONVERTUCS2CLOBS
                                    --指定在解析数据库日志时所需要的特殊参数
                                    --本参数只用在extract端 UTF字符类型,并且11.1.1前处理CLOB才需要:
                       示例3:--tranlogoptions asmuser sys@asm, asmpassword  AACAAAAAAAAAAGAIF, ENCRYPTKEY default
                            
                       示例4:TRANLOGOPTIONS altarchivelogdest primary instance sgpmdb1 /sgpmdb/arch altarchivelogdest  instance  sgpmdb2 /sgpmdb/arch                    指定RAC环境下每个节点的归档日志地址                        
                            
    TRANSMEMORY        Controls the amount of memory and temporary disk space available for caching uncommitted transaction data.
                       设定GoldenGate的抽取进程能够使用的内存大小。如下参数指定本进程最大只能占用2G内存,其中每个事务最大占用内存不能超过500M,如果超过则使用指定目录作为虚拟内存,该目录下的单个文件大小为4G,最多只能在该目录下占用8G空间作为缓存:
                       示例:transmemory directory (/backup/goldengate/dirtmp,8G,4G),ram 2G,transram 500M

    (11)维护Maintenance

    DISCARDROLLOVER         Controls how often to create a new discard file.
                            示例:DISCARDROLLOVER AT 3:00  --为了防止discard file被写满,每天3:00做一次文件过期设定
    
    PURGEOLDEXTRACTS        Purges obsolete trail files.同mgr进程,可以设置自动删除队列,建议在mgr设置
    
    REPORTROLLOVER          Specifies when to create new report files.
                            设定切换一个日志的时间和间隔。示例:reportrollover at 02:00
    
    ROLLOVER                Specifies the way that trail files are aged.

    (12)安全性Security

    DECRYPTTRAIL                         Decrypts data in a trail or extract file. 解密trail文件或extract文件数据
    
    ENCRYPTTRAIL | NOENCRYPTTRAIL        Controls encryption of data in a trail or extract file. 加密trail或extract文件的数据

     三、replicat相关参数

    1、常用参数

    replicat提供如下方面参数: 

    Checkpoint
    数据影射关系
        Optional row-level selection criteria
        Optional column mapping facilities
        Optional transformation services
        Optional Stored Procedure or SQL query execution
    错误处理机制
    其它可选参数

    参数举例:

    replicat repsz
    setenv ( NLS_LANG = AMERICAN_AMERICA.ZHS16GBK )
    userid goldengate, password *****
    sqlexec "Alter session set constraints=deferred"
    REPORT AT 01:59
    reportrollover at 02:00
    --handlecollisions 
    reperror default,discard
    discardfile /oradata/goldengate/dirrpt/repsz.dsc,append, megabytes 10
    --grouptransops 100
    --batchsql
    assumetargetdefs
    allownoopupdates
    dynamicresolution
    numfiles 3000
    --mapexclude CTAIS2.JC_GY_SWWSWH
    --map CTAIS2.SHOULIXINXI, target CTAIS2.SHOULIXINXI, keycols ( SHOULIBIANHAO );
    MAP ctais2.* ,TARGET ctais2.*;  

    Map参数:

    MAP <source table>, TARGET <target table>
        [, EXCEPTIONSONLY]
        [, COLMAP (<column mapping specification>)]
        [, KEYCOLS (<column list>)]
        [, WHERE (<where condition>)]
        [, FILTER (<expression>)]
    ;

    注意map语句必须以一个分号结束;

    2、完整参数示例:

    示例1:

    GGSCI 6> view params repsa
    
    REPLICAT repsa
    SETENV (NLS_LANG = "American_America.UTF8")  --设置字符集环境变量为UTF8
    SETENV (ORACLE_SID = "xxxx" )                --如果系统中存在多个数据库有时候会用参数SETENV设置ORACLE_HOME、ORACLE_SID等
    USERID goldengate, PASSWORD AACAAAAAAAAAAAJAPJYATHVIMGMAOFHIUCUEPFYGGJBFJGIH, ENCRYPTKEY default
    --SQLEXEC "ALTER SESSION SET CONSTRAINTS=DEFERRED"
    DBOPTIONS DEFERREFCONST    --约束延迟设置。在复制进程的事物被提交之前,延迟级联删除、级联更新时的校验和实施。
    GETTRUNCATES               --不捕获生产端truncate table的操作。
    REPORT AT 06:00            --每天早上6点报告 
    REPORTCOUNT EVERY 30 MINUTES, RATE    --每隔30分钟报告一次从程序开始到现在的抽取进程或者复制进程的事物记录数,并汇报进程的统计信息
    REPORTROLLOVER AT 02:00               --为了防止report file被写满,每天2:00做一次文件过期设定
    REPERROR DEFAULT, ABEND  --除了特殊指定的REPERROR语句,报告所有复制期间出现的错误,回滚非正常中断的事物和进程。
    --HANDLECOLLISIONS       --当灾备端已经存在数据的情况下,解决复制过程中出现的冲突。如果要重新做初始化,可以删除drop灾备端数据库后再rman恢复,这样做的话就不需要该参数了。
    ####ALLOWNOOPUPDATES     --当生产端有某些列但是目标表却没有,或者复制进程中配置了COLSEXCEPT 参数 在这些情况下,当生产端对那些列进行更新,目标表将不发生任何变化
    assumetargetdefs         --使用ASSUMETARGETDEFS参数时,用MAP语句中指定的生产库源表和灾备端目标表具有相同的列结构。它指示的Oracle GoldenGate不在生产端查找源表的结构定义。
    DISCARDFILE ./dirrpt/repsa.dsc, APPEND, MEGABYTES 1024M  --将执行失败的记录保存在discard file中,该文件位于./dirrpt/extya.dsc,大小为1024MB。 文件中已经包含记录的话,再后面继续追加,不删除之前的记录。
    DISCARDROLLOVER AT 02:00   --为了防止discard file被写满,每天2:00做一次文件过期设定
    —Schema名称涉及敏感字符,在此统一使用Schema来代替
    —table名称涉及敏感字符,在此统一使用table来代替
    map Schema1.table1, target Schema1.table1;
    map Schema1.table2, target Schema1.table2;
    map Schema2.table1, target Schema2.table1;
    map Schema2.table2, target Schema2.table2;

    示例2:

    replicat repl                               --复制进程名
    userid ddw,password ddw                     --目标数据库的帐号密码
    assumetargetdefs                            --两台数据库数据结构一致则使用此参数
    reperror default,discard                        --如果复制数据出错则忽略,
    discardfile D:
    epsz.dsc,append,megabytes 100   --错误信息写入XXXX文件,追加模式,最大100m
    gettruncates                                    --复制truncate操作                              
    map ddw.aatest, target ddw.aatest_1;            --映射关系,注意target前必须留一个空格
    --map ddw.aatest, target ddw.aatest_1, keycols (name), where (TYPE1 = "1")  
    --加两横杠在配置文件中进行注释;keycols指定唯一键;where筛选数据

    3、全部可选参数

    (1)通用参数

    CHECKPARAMS     Verifies parameter file syntax.检查参数语法然后停止。
    
    COMMENT | --    Denotes comments in a parameter file. 注释行。
    
    GETENV          Retrieves variables that were set with the  SETENV parameter.
    SETENV          Specifies a value for a UNIX environment variable from within the GGSCI interface.
                    示例:SETENV (NLS_LANG = "American_America.UTF8")  --设置字符集环境变量为UTF8
                    示例:SETENV (ORACLE_SID = "xxxx" )                --如果系统中存在多个数据库有时候会用参数SETENV设置ORACLE_HOME、ORACLE_SID等
    
    OBEY            Processes parameter statements contained in a different parameter file.将外部文件包含到参数文件中。
    
    TRACETABLE | NOTRACETABLE     Specifies a trace table to which Replicat adds a record whenever it updates the target database. Supports Oracle.

    (2)Processing method

    BEGIN            Specifies a starting point for Replicat processing. Required when  SPECIALRUN is specified.
    
    BULKLOAD         Loads data directly into the interface of the Oracle SQL*Loader utility.
    
    END              Specifies a stopping point for Replicat processing. Required when using  SPECIALRUN .
    
    GENLOADFILES     Generates run and control files that are compatible with a database load utility.
    
    REPLICAT         Specifies a Replicat group for online change synchronization.定义进程名称。示例:replicat repsz
    
    SPECIALRUN       Used for one-time processing that does not require checkpointing from run to run.

    (3)Database login

    TARGETDB         Specifies the data source as part of the login information.
    
    USERID          Specifies database connection information. 指定登陆目标数据库的用户名和密码。
                    示例:Userid goldengate, password 123456
                         USERID goldengate, PASSWORD AACAAAAAAAAAAAJAPJYATHVIMGMAOFHIUCUEPFYGGJBFJGIH, ENCRYPTKEY default

    (4)Selecting, converting, and mapping data

    ALLOWDUPTARGETMAP |
    NOALLOWDUPTARGETMAP     Allows the same source-target  MAP statement to appear more than once in the parameter file.
                         
    ASCIITOEBCDIC           Converts incoming ASCII text to EBCDIC for DB2 on z/OS systems running UNIX System Services.
                         
    ASSUMETARGETDEFS        Assumes that source and target tables have the same column structure.假定两端数据结构一致使用此参数。
                            示例: assumetargetdefs     --使用ASSUMETARGETDEFS参数时,用MAP语句中指定的生产库源表和灾备端目标表具有相同的列结构。它指示的Oracle GoldenGate不在生产端查找源表的结构定义。
                                            
    COLMATCH             Establishes global column-mapping rules.
                         
    DDL                  Enables and filters the capture of DDL operations.
    
    DDLSUBST             Enables string substitution in DDL processing.
    
    GETDELETES | IGNOREDELETES               Controls the replication of delete operations.是否复制delete操作,缺省为复制。
                        
    GETINSERTS | IGNOREINSERTS               Controls the replication of insert operations.是否复制insert操作,缺省为复制。
    
    GETUPDATEAFTERS | IGNOREUPDATEAFTERS     Controls the replication of update after images. 是否读取后影像,缺省为读取
    
    GETUPDATEBEFORES | IGNOREUPDATEBEFORES   Controls the replication of update before images. 是否读取前影像,缺省为不读取。
    
    GETUPDATES | IGNOREUPDATES               Controls the replication of update operations.是否复制update操作,缺省为复制。
    
    GETTRUNCATES | IGNORETRUNCATES           Includes or excludes the replication of  TRUNCATE statements.是否复制truncate操作,缺省为不复制。
                                             示例:GETTRUNCATES               --不捕获生产端truncate table的操作。
                                        
    INSERTALLRECORDS                  Inserts a new record into the target table for every change operation made to a record.
    
    INSERTDELETES | NOINSERTDELETES   Converts deletes to inserts.
    
    INSERTMISSINGUPDATES | 
    NOINSERTMISSINGUPDATES            Converts an update to an insert when the target row does not exist. 
    
    INSERTUPDATES | NOINSERTUPDATES   Converts updates to inserts.
    
    MAP for Replicat       Specifies a relationship between one or more source and target tables and controls column mapping and conversion.
    
    MAPEXCLUDE               Excludes tables from being processed by a wildcard specification supplied in  MAP statements.
                            用于使用在map中使用*匹配时排除掉指定的表,类似于于源端的tablexclude。
                           示例:mapexclude CTAIS2.JC_GY_SWWSWH
                                  MAP ctais2.* ,TARGET ctais2.*; 
    
    REPLACEBADCHAR      Replaces invalid character values with another value.
    
    REPLACEBADNUM       Replaces invalid numeric values with another value.
    
    SOURCEDEFS          Specifies a file that contains source data definitions created by the  DEFGEN utility.
                        假定两端数据结构不一致,使用此参数指定源端的数据结构定义文件。该文件需要由GoldenGate工具产生。
    
    SPACESTONULL | NOSPACESTONULL  Controls whether or not a target column containing only spaces is converted to  NULL .
    
    TABLE for Replicat            Specifies a table or tables for which event actions are to take place when a row satisfies the given filter criteria.
    
    TRIMSPACES | NOTRIMSPACES     Controls whether trailing spaces are trimmed or not when mapping  CHAR to  VARCHAR columns.
    
    UPDATEDELETES | NOUPDATEDELETES    Changes deletes to updates.
    
    USEDATEPREFIX              Prefixes data values for  DATE data types with a  DATE literal, as required by Teradata databases.
        
    USETIMEPREFIX              Prefixes data values for  TIME datatypes with a  TIME literal, as required by Teradata databases.
        
    USETIMESTAMPPREFIX         Prefixes data values for  TIMESTAMP datatypes with a TIMESTAMP literal, as required by Teradata databases. 

    (5)Routing data

    EXTFILE             Defines the name of an extract file on the local system that contains data to be replicated. Used for one-time processing. 
    
    EXTTRAIL            Defines a trail containing data to be replicated. Used for one-time processing.

    (6)Custom processing

    CUSEREXIT               Invokes a user exit routine during processing.
    
    DEFERAPPLYINTERVAL      Specifies a length of time for Replicat to wait before applying replicated operations to the target database.
    
    INCLUDE                 References a macro library in a parameter file.
    
    MACRO                   Defines an Oracle GoldenGate macro.
    
    SQLEXEC                 Executes a stored procedure or query during Replicat processing.
                            调用存储过程或者执行sql语句。可以将返回值作为依据进行过滤条件,或者用户改变session变量。
                            示例:SQLEXEC "ALTER SESSION SET CONSTRAINTS=DEFERRED"

    (7)Reporting

    CMDTRACE            Displays macro expansion steps in the report.
    
    LIST | NOLIST       Displays or suppresses the listing of macros in the report file.
    
    REPORT              Schedules a statistical report at a specified date or time.
                        示例:REPORT AT 06:00            --每天早上6点报告 
                        
    REPORTCOUNT         Reports the number of records processed.
                        示例:REPORTCOUNT EVERY 30 MINUTES, RATE    
                             --每隔30分钟报告一次从程序开始到现在的抽取进程或者复制进程的事物记录数,并汇报进程的统计信息
    
    SHOWSYNTAX          Causes Replicat to print its SQL statements to the report file.
    
    STATOPTIONS         Specifies information to include in statistical displays.
    
    TRACE/TRACE2        Shows Replicat processing information to assist in revealing bottlenecks.

    (8)Error handling

    CHECKSEQUENCEVALUE |
    NOCHECKSEQUENCEVALUE    Controls whether or not Replicat verifies that a target sequence value is higher than the one on the source and corrects any disparity that it finds.
    
    DDLERROR                Controls error handling for DDL replication.
    
    DISCARDFILE             Contains records that could not be processed.
                            定义出错数据的输出文件。当数据出现错误后,可以用于查找错误原因。
                            示例:discardfile /oradata/goldengate/dirrpt/repsz.dsc,append, megabytes 10
                                 DISCARDFILE ./dirrpt/repsa.dsc, APPEND, MEGABYTES 1024M  --将执行失败的记录保存在discard file中,该文件位于./dirrpt/extya.dsc,大小为1024MB。 文件中已经包含记录的话,再后面继续追加,不删除之前的记录。
    
    HANDLECOLLISIONS |      自动过滤重复时段的数据冲突,用于不能停机执行初始化。打开该参数后不会将数据错误报到discard文件中。
    NOHANDLECOLLISIONS      Handles errors for duplicate and missing records.
                            示例:--HANDLECOLLISIONS   
                                 --当灾备端已经存在数据的情况下,解决复制过程中出现的冲突。如果要重新做初始化,可以删除drop灾备端数据库后再rman恢复,这样做的话就不需要该参数了。
    
    
    HANDLETPKUPDATE         Prevents constraint errors associated with replicating transient primary key updates.
    
    OVERRIDEDUPS |
    NOOVERRIDEDUPS          Overlays a replicated insert record onto an existing target record whenever a duplicate-record error occurs.
    
    RESTARTCOLLISIONS |     Controls whether or not Replicat applies 
    NORESTARTCOLLISIONS     HANDLECOLLISIONS logic after Oracle GoldenGate has abended because of a conflict.
    
    REPERROR                Determines how Replicat responds to database errors.
                            定义出错以后replicat的响应,一般可以定义为两种:
                                Abend,即一旦出现错误即停止复制,此为缺省配置;
                                Discard,出现错误后继续复制,只把错误的数据放到discard文件中。
                            示例:REPERROR DEFAULT, DISCARD
                                 REPERROR DEFAULT, ABEND  --除了特殊指定的REPERROR语句,报告所有复制期间出现的错误,回滚非正常中断的事物和进程。
    
    
    REPFETCHEDCOLOPTIONS     Determines how Replicat responds to operations for which a fetch from the source database was required.
    
    SQLDUPERR                Specifies the database error number that indicates a duplicate record. Use with  OVERRIDEDUPS .
    
    WARNRATE                 Determines how often database errors are reported.

    (9)Tuning

    ALLOCFILES          Controls the amount of incremental memory that is allocated when the amount of memory specified with NUMFILES is reached.
    
    BATCHSQL            Increases the throughput of Replicat processing by arranging similar SQL statements into arrays and applying them at an accelerated rate.          针对批处理中针对某个表的大批量重复操作进行优化,提高批处理的处理速度。
    
    CHECKPOINTSECS      Controls how often Replicat writes a checkpoint when checkpoints are not being generated as the result of transaction commits.
    
    DBOPTIONS           Specifies database options.定义与数据库类型相关的特殊处理方式。
                        示例:DBOPTIONS DEFERREFCONST    --约束延迟设置。在复制进程的事物被提交之前,延迟级联删除、级联更新时的校验和实施。
    
    DDLOPTIONS          Specifies DDL processing options.
    
    DYNAMICRESOLUTION |     使replicat动态解析表的结构,加快启动速度。缺省为每次启动解析所有要复制表的结构。
    NODYNAMICRESOLUTION     Makes Replicat start faster when there is a large number of tables specified for synchronization.
    
    DYNSQL | NODYNSQL         Causes Replicat to use literal SQL statements rather than a compile-once, execute-many strategy.
    
    EOFDELAY | EOFDELAYCSECS  Determines how many seconds Replicat delays before looking for more data to process.
    
    FUNCTIONSTACKSIZE       Controls the size of the memory stack that is used for processing Oracle GoldenGate functions.
    
    GROUPTRANSOPS           Controls the number of records that are grouped into a Replicat transaction.
                            将小的交易合并成为一个大的交易进行提交,减少提交次数,降低系统IO消耗。示例:grouptransops 100
    
    INSERTAPPEND |
    NOINSERTAPPEND          Controls whether or not Replicat uses an  APPEND hint when applying  INSERT operations to Oracle target tables.
    
    MAXDISCARDRECS          Limits the number of discarded records reported to the discard file.
    
    MAXSQLSTATEMENTS        Controls the number of prepared SQL statements that can be used by Replicat.
    
    MAXTRANSOPS   Divides large source transactions into smaller ones on the target system.
                  将大交易拆分,每若干条纪录提交一次。示例:maxtransops 1000
    
    NUMFILES      Controls the initial allocation of memory that is dedicated to storing information about tables to be processed by Oracle GoldenGate.
                  定义进程中表的最大数据量,缺省为 500.
    
    RETRYDELAY                 Specifies the delay between attempts to retry a failed SQL operation.
    
    TRANSACTIONTIMEOUT         Specifies a time interval after which Replicat will commit its open target transaction and roll back any incomplete source transactions that it contains, saving them for when the entire source transaction is ready to be applied.
    
    TRANSMEMORY             Controls the amount of memory and temporary disk space available for caching uncommitted transaction data.
    
    WILDCARDRESOLVE         Alters the rules by which wildcard specifications in a  MAP statement are resolved.

    (10)Maintenance

    DISCARDROLLOVER         Specifies when to create new discard files.
                            示例:DISCARDROLLOVER AT 02:00   --为了防止discard file被写满,每天2:00做一次文件过期设定
    
    PURGEOLDEXTRACTS        Purges obsolete trail files. 定义自动删除队列,一般建议在mgr进程配置。
    
    REPORTROLLOVER          Specifies when to create new report files.
                            示例: REPORTROLLOVER AT 02:00   --为了防止report file被写满,每天2:00做一次文件过期设定

    (11)Security

    DECRYPTTRAIL             Decrypts data in a trail or extract file.

    补充:

    allownoopupdates  参数含义:允许执行无实际变化的update。例如,update a=a会纪录一条update,但是没有后影像,无法正确构筑where语句。
    ALLOWNOOPUPDATES  --当生产端有某些列但是目标表却没有,或者复制进程中配置了COLSEXCEPT 参数 在这些情况下,当生产端对那些列进行更新,目标表将不发生任何变化

    四、定义文件参数

    DEFSFILE             Identifies the name of the file to which  DEFGEN writes the definitions
    
    SOURCEDB             Specifies the data source as part of the login information.
    
    TABLE for DEFGEN     Identifies a table for which you want to capture a definition.
    
    USERID               Specifies database connection information.
  • 相关阅读:
    Vitrualbox虚拟机网络设置
    挂接命令(mount) 【转】
    ubuntu Error mounting /dev/sda6 at /media/xxx...
    mysql命令[转]
    fopen参数
    fprintf&prinft&sprintf
    UNIX 网络编程知识,函数积累
    vim 括号自动补全
    CDbConnection failed to open the DB connection
    [Android_蓝牙]Android4.0蓝牙使能的详细解析
  • 原文地址:https://www.cnblogs.com/chinas/p/7339465.html
Copyright © 2011-2022 走看看