Oracle goldengate部署(ogg-for-kafka)
源端数仓系统环境如下:
组件名称 | 版本信息 |
---|---|
操作系统环境 | HP-UX evansdb2 B.11.31 U ia64 |
数据库版本 | 11.2.0.4.180116 |
OGG 版本 | 12.3.0.1.190115 |
KAFKA 目标端环境如下:
组件名称 | 版本信息 |
---|---|
操作系统环境 | BigCloud Enterprise Linux For Euler release 21.10 (LTS-SP2) |
目标端 OGG 版本 | (12.3.2.1) for Big Data |
kafka 版本 | 2.2.0 |
---|---|
java 版本 | 1.8.0_221 |
2、添加表的补充日志
#确定表名称
SQL> select owner,table_name from dba_tables where owner='OS' and table_n ame like 'OM_BUSI';
#添加表附加日志
add trandata OS.OM_BUSI
#检查表的附加日志是否添加成功
select owner,table_name,count(*) from dba_log_groups where table_name in ('OM_BUSI') group by owner,table_name;
3、创建 EK_INSC1 抽取进程
GGSCI (evansdb2) 5> dblogin UserIDAlias user_ogg Successfully logged into database.
GGSCI (evansdb2 as ogg@evansdb2) 7> register extract ek_insc1 database
2024-08-13 16:02:52 INFO OGG-02003 Extract EK_INSC1 successfully
registered with database at SCN 14458220262885.
GGSCI (evansdb2 as ogg@evansdb2) 8> add extract ek_insc1 integrated tranlog,begin now
EXTRACT (Integrated) added.
view param ek_insc1
Extract ek_insc1
UserIDAlias user_ogg ExtTrail ./dirdat/ca
LogAllSupCols
UpdateRecordFormat compact
WarnLongTrans 2h, CheckInterval 300s GetTruncates
DiscardFile ./dirrpt/ek_insc1.dsc, Append, MegaBytes 128
TranLogOptions IntegratedParams( max_sga_size 64 Parallelism 2 )
table OS.OM_BUSI;
GGSCI (evansdb2 as ogg@evansdb2) 10> add exttrail ./dirdat/ca,extract
ek_insc1,megabytes 1024 EXTTRAIL added.
4、创建PK_INSC1投递进程
GGSCI(evansdb2 as ogg@evansdb2)70> add extract pk_insc1,tranlog,exttrailsource ./dirdat/ca
EXTRACT added.
GGSCI (evansdb2 as ogg@evansdb2) 71> add rmttrail ./dirdat/ca,extract
pk_insc1,megabytes 1024
RMTTRAIL added.
edit param pk_insc1
Extract pk_insc1
Passthru
RmtHost 192.168.20.50, MgrPort 7809
RmtTrail ./dirdat/ca
table OS.OM_BUSI;
1、安装 OGG For Bigdata 软件
2、创建目录及环境变量
#ogg 软件安装在 ogg 用户下
#配置 kafka 环境变量
[kafka@ ~]$ cat .bashrc 增加下面环境变量
export GG_HOME=/oggapp
export JRE_HOME=$JAVA_HOME/jre
export CLASSPATH=.:$JAVA_HOME/lib:$JRE_HOME/lib:$CLASSPATH export PATH=$PATH:$GG_HOME:$JAVA_HOME/lib:$JRE_HOME/lib
export LD_LIBRARY_PATH=$JAVA_HOME/jre/lib/amd64:$JAVA_HOME/jre/lib/amd64/s erver:$LD_LIBRARY_PATH
3解压缩安装软件
$mkdir -p /data12/ogg/ogg_home1
$ tar –xvf OGG_BigData_Linux_x64_12.3.2.1.0.tar –C /data12/ogg/ogg_home1
--运行 ggsic 报错
[ogg@nmcc ogg_home1]$ ldd ggsci
linux-vdso.so.1 (0x00007ffdb62c3000)
librt.so.1 => /lib64/librt.so.1 (0x00007f8150fce000) libdl.so.2 => /lib64/libdl.so.2 (0x00007f8150fc9000)
libgglog.so => /data12/ogg/ogg_home1/./libgglog.so (0x00007f8150 b7b000)
libggutil.so =