GoldenGate Integrated Capture Mode

    通常我们配置的Goldengate是读取在线日志文件或者ALO（只读归档日志的）。还有另外一种抽取模式Integrated Capture Mode，它是依靠 logmining server读取数据库改变的信息

    Integrated capture supports more data types as well as compressed data and as it is fully integrated with the database there is no additional setup steps required when we are configuring GoldenGate with things like RAC, ASM and TDE 

配置环境：

Source：
OS:Redhat 6.3
DB:11.2.0.4
IP:192.168.56.60

Target:
OS:Redhat 6.3
DB:11.2.0.4
IP:192.168.56.61

Prepare(Source And Target DB):

useradd ogg -g oinstall

Add Environment Varible
vi /home/oracle/.bash_profile
export LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib:/ogg

Configure Database
alter database archivelog;
alter database force logging;
alter database add supplemental log data (primary key ,unique index) columns;

select SUPPLEMENTAL_LOG_DATA_MIN, SUPPLEMENTAL_LOG_DATA_PK,
SUPPLEMENTAL_LOG_DATA_UI from v$database;


Create OGG User:
sqlplus / as sysdba
create user ogg identified by ogg;
grant dba to ogg;


GGSCI (zbdba1) 1> edit params ./GLOBAL
GGSCHEMA ogg
CHECKPOINTTABLE ogg.checkpoint

SQL> show parameter goldengate

NAME                                 TYPE        VALUE
------------------------------------ ----------- ------------------------------
enable_goldengate_replication        boolean     TRUE


Installing the DDL objects:
1、Run the @marker_setup.sql script. This script installs support for the Oracle GoldenGate
marker system, which is required for DDL support. You will be prompted for the name
of the Oracle GoldenGate schema.
drop trigger ggs_ddl_trigger_before;
grant create table,create view,create sequence to ogg;
2、Run the @ddl_setup.sql script   Script that installs the Oracle GoldenGate DDL extraction and
replication objects. (Oracle installations)
3、Run the @role_setup.sql script. This script drops and creates the role needed for DDL
synchronization. It grants DML permissions on the Oracle GoldenGate DDL objects
4、Grant the role to all Oracle GoldenGate Extract users. You may need to make multiple
grants if the processes have different user names.
Run the @ddl_enable.sql script to enable the DDL trigger.
5、To improve the performance of the DDL trigger, make the ddl_pin script part of the database
startup. It must be invoked with the Oracle GoldenGate DDL user name, as in:
@?/rdbms/admin/dbmspool.sql
SQL> @ddl_pin ogg
To improve the performance of the DDL trigger 
This script pins the PL/SQL package that is used by the trigger into memory. If executing
this script from SQL*Plus, connect as SYSDBA from the Oracle GoldenGate home directory.
This script relies on the Oracle dmbs_shared_pool system package, so install that package
before using ddl_pin.

Source DB:

MANAGER
GGSCI (zbdba1) 1>  create subdirs

GGSCI (zbdba1) 22> view param mgr

Source:
port 7839
DYNAMICPORTLIST 7840-7914
USERID ogg,PASSWORD ogg
AUTORESTART EXTRACT *,RETRIES 5,WAITMINUTES 3
PURGEOLDEXTRACTS ./DIRDAT/*,MINKEEPDAYS 3
PURGEDDLHISTORY MINKEEPDAYS 7,MAXKEEPDAYS 10,FREQUENCYMINUTES 30
PURGEMARKERHISTORY MINKEEPDAYS 7,MAXKEEPDAYS 10
LAGREPORTHOURS 1
LAGINFOMINUTES 30
LAGCRITICALMINUTES 45

Target:

port 7839
DYNAMICPORTLIST 7840-7914
USERID ogg,PASSWORD ogg
AUTORESTART REPLICAT *,RETRIES 5,WAITMINUTES 3
PURGEOLDEXTRACTS ./DIRDAT/*,MINKEEPDAYS 3
PURGEDDLHISTORY MINKEEPDAYS 7,MAXKEEPDAYS 10,FREQUENCYMINUTES 30
PURGEMARKERHISTORY MINKEEPDAYS 7,MAXKEEPDAYS 10
LAGREPORTHOURS 1
LAGINFOMINUTES 30
LAGCRITICALMINUTES 45

EXTRACT

GGSCI (zbdba1) 1> dblogin userid ogg,password ogg
GGSCI (zbdba1) 1> add trandata zbdba.*

GGSCI (zbdba1) 65> REGISTER EXTRACT ext1 DATABASE



2015-03-03 21:57:13  WARNING OGG-01758  This EXTRACT EXT1 is already registered with the database.


GGSCI (zbdba1) 2> ADD EXTRACT ext1 INTEGRATED TRANLOG, BEGIN NOW


EXTRACT added.


GGSCI (zbdba1) 3>
GGSCI (zbdba1) 3>
GGSCI (zbdba1) 3> add exttrail /ogg/dirdat/sa  extract ext1
EXTTRAIL added.


GGSCI (zbdba1) 4> add rmttrail ./dirdat/sa,extract ext1
RMTTRAIL added.

GGSCI (zbdba1) 22> view param ext1

EXTRACT EXT1
setenv ( NLS_LANG = AMERICAN_AMERICA.ZHS16GBK )
USERID ogg,PASSWORD ogg
DDL &
INCLUDE MAPPED OBJTYPE 'TABLE' &
INCLUDE MAPPED OBJTYPE 'IDNEX' &
INCLUDE MAPPED OBJTYPE 'SEQUENCE' &
INCLUDE MAPPED OBJTYPE 'VIEW' &
INCLUDE MAPPED OBJTYPE 'PROCEDURE' &
INCLUDE MAPPED OBJTYPE 'FUNCTION' &
INCLUDE MAPPED OBJTYPE 'PACKAGE' &
INCLUDE MAPPED OBJTYPE 'MATERIALIZED VIEW' &
EXCLUDE OPTYPE COMMENT
TRANLOGOPTIONS INTEGRATEDPARAMS (MAX_SGA_SIZE 100)
DDLOPTIONS ADDTRANDATA NOCROSSRENAME REPORT
REPORTCOUNT EVERY 1 MINUTES,RATE
DISCARDFILE ./dirrpt/EXTSA.DSC,APPEND,MEGABYTES 1024
THREADOPTIONS MAXCOMMITPROPAGATIONDELAY 60000 IOLATENCY 60000
DBOPTIONS ALLOWUNUSEDCOLUMN
WARNLONGTRANS 2H,CHECKINTERVAL 3M
exttrail ./dirdat/sa
FETCHOPTIONS NOUSESNAPSHOT,FETCHPKUPDATECOLS,MISSINGROW REPORT
table zbdba.*;

在设置TRANLOGOPTIONS INTEGRATEDPARAMS (MAX_SGA_SIZE 100) 参数时，启动可能报错
修改streams_pool_size的大小：

SQL> show parameter streams

NAME                                 TYPE        VALUE
------------------------------------ ----------- ------------------------------
streams_pool_size                    big integer 152M

开启ext1：

GGSCI (zbdba1) 13> start ext1

Sending START request to MANAGER ...
EXTRACT EXT1 starting

查看日志：

2015-03-03 22:00:34  INFO    OGG-00992  Oracle GoldenGate Capture for Oracle, ext1.prm:  EXTRACT EXT1 starting.
2015-03-03 22:00:34  INFO    OGG-03035  Oracle GoldenGate Capture for Oracle, ext1.prm:  Operating system character set identified as UTF-8. Locale: en_US, LC_ALL:.
2015-03-03 22:00:34  INFO    OGG-01635  Oracle GoldenGate Capture for Oracle, ext1.prm:  BOUNDED RECOVERY: reset to initial or altered checkpoint.
2015-03-03 22:00:34  INFO    OGG-01815  Oracle GoldenGate Capture for Oracle, ext1.prm:  Virtual Memory Facilities for: BR
    anon alloc: mmap(MAP_ANON)  anon free: munmap
    file alloc: mmap(MAP_SHARED)  file free: munmap
    target directories:
    /ogg/BR/EXT1.
2015-03-03 22:00:34  INFO    OGG-01815  Oracle GoldenGate Capture for Oracle, ext1.prm:  Virtual Memory Facilities for: COM
    anon alloc: mmap(MAP_ANON)  anon free: munmap
    file alloc: mmap(MAP_SHARED)  file free: munmap
    target directories:
    /ogg/dirtmp.
2015-03-03 22:00:35  WARNING OGG-01423  Oracle GoldenGate Capture for Oracle, ext1.prm:  No valid default archive log destination directory found for thread 1.
2015-03-03 22:00:47  INFO    OGG-02036  Oracle GoldenGate Capture for Oracle, ext1.prm:  <span style="color:#ff0000;">Integrated capture successfully attached to logmining server OGG$EXT1.</span>
2015-03-03 22:00:48  INFO    OGG-00993  Oracle GoldenGate Capture for Oracle, ext1.prm:  EXTRACT EXT1 started.
2015-03-03 22:00:48  INFO    OGG-01056  Oracle GoldenGate Capture for Oracle, ext1.prm:  Recovery initialization completed for target file ./dirdat/sa000000, at RBA 81396296, CSN 1296687.
2015-03-03 22:00:48  INFO    OGG-01478  Oracle GoldenGate Capture for Oracle, ext1.prm:  Output file ./dirdat/sa is using format RELEASE 11.2.
2015-03-03 22:00:48  WARNING OGG-01438  Oracle GoldenGate Capture for Oracle, ext1.prm:  Checkpoint marked as from graceful shutdown, but records found after checkpoint in trail ./dirdat/sa.  Expected EOF Seqno 0, RBA 0.  Found Seqno 0, RBA 81396296.
2015-03-03 22:00:48  INFO    OGG-01026  Oracle GoldenGate Capture for Oracle, ext1.prm:  Rolling over remote file ./dirdat/sa000000.

datapump

add extract ext2,exttrailsource /ogg/dirdat/sa
ADD EXTTRAIL ./dirdat/sa, EXTRACT EXT2
add rmttrail  /ogg/dirdat/sa, extract ext2



GGSCI (zbdba1) 2>  view param ext2

EXTRACT EXT2
passthru
RMTHOST 192.168.56.61,MGRPORT 7839,COMPRESS
RMTTRAIL ./dirdat/sa
table zbdba.*;

Target DB:

GGSCI (zbdba2) 1>  create subdirs

GGSCI (zbdba2) 42> view param mgr

port 7839
DYNAMICPORTLIST 7840-7914
USERID OGG,PASSWORD ogg
AUTORESTART REPLICAT *, RETRIES 5, WAITMINUTES 3
PURGEOLDEXTRACTS ./dirdat/*,MINKEEPDAYS 3
PURGEDDLHISTORY MINKEEPDAYS 7,MAXKEEPDAYS 10,FREQUENCYMINUTES 30
PURGEMARKERHISTORY MINKEEPDAYS 7,MAXKEEPDAYS 10
LAGREPORTHOURS 1
LAGINFOMINUTES 30
LAGCRITICALMINUTES 45 


Create Replicat 
GGSCI (zbdba2) 1>  view param ./GLOBALS
GGSCHEMA ogg
CHECKPOINTTABLE ogg.checkpoint


dblogin userid ogg,password ogg
 add checkpointtable  ogg.checkpoint

Successfully created checkpoint table OGG.CHECKPOINT.

GGSCI (zbdba2) 2>  add replicat rep1,exttrail /ogg/dirdat/sa, checkpointtable ogg.checkpoint
REPLICAT added.


GGSCI (zbdba2) 19> edit param rep1

REPLICAT rep1
ASSUMETARGETDEFS
USERID ogg, PASSWORD ogg
discardfile ./dirdat/rep1_discard.txt,append,megabytes 5
DDL INCLUDE MAPPED
DDLOPTIONS REPORT
BATCHSQL
DBOPTIONS DEFERREFCONST
DBOPTIONS LOBWRITESIZE 102400
HANDLECOLLISIONS
DDLERROR DEFAULT DISCARD RETRYOP MAXRETRIES 5 RETRYDELAY 20
--grouptransops 1
--maxtransops 1
APPLYNOOPUPDATES
MAP zbdba.*, TARGET zbdba.*;


start manager


GGSCI (zbdba2) 23> info all

Program     Status      Group       Lag at Chkpt  Time Since Chkpt

MANAGER     RUNNING                                          
REPLICAT    STOPPED     REP1        00:00:00      00:00:03  

Test Data(Source and Target DB):

create tablepsace zbdab datafile '/opt/oracle/oradata/zbdba.dbf' size 100m;
create zbdba identified by oracle default tablespace zbdba;
grant dba to zbbda;


On Source DB:

User data pump:
expdp zbdba/oracle DIRECTORY=dumpdir
           DUMPFILE=zbdba.dmp LOGFILE=zbdba.log
           SCHEMAS=zbdba JOB_NAME=exp_zbdba_schema


scp data to target DB


On Target DB:
impdp zbdba/oracle DIRECTORY=dumpdir
           DUMPFILE=zbdba.dmp SCHEMAS=zbdba
           JOB_NAME=imp_zbdba_schema