canal-adapter 1.1.7构建镜像
canal-adapter 可以实现mysql表数据同步到ES的适配器,官方没有提供docker镜像,可自行构建镜像简单几步即可实现。
准备安装包
官网地址: https://github.com/alibaba/canal
https://github.com/alibaba/canal/releases/tag/canal-1.1.7
https://github.com/alibaba/canal/releases/download/canal-1.1.7/canal.adapter-1.1.7.tar.gz
编写Dockerfile
FROM openjdk:8-jre-alpine
WORKDIR /opt/canal/canal-adapter
COPY bin /opt/canal/canal-adapter/bin
COPY conf /opt/canal/canal-adapter/conf
COPY lib /opt/canal/canal-adapter/lib
COPY logs /opt/canal/canal-adapter/logs
COPY plugin /opt/canal/canal-adapter/plugin
ENV LANG=C.UTF-8
ENV TZ=Asia/Shanghai
CMD ["sh", "-c", "sh /opt/canal/canal-adapter/bin/startup.sh && tail -F /opt/canal/canal-adapter/logs/adapter.log"]
LABEL version="1.1.7"
修改启动脚本startup.sh
#!/bin/bash
current_path=`pwd`
case "`uname`" in
Linux)
bin_abs_path=$(readlink -f $(dirname $0))
;;
*)
bin_abs_path=`cd $(dirname $0); pwd`
;;
esac
base=${bin_abs_path}/..
export LANG=en_US.UTF-8
export BASE=$base
if [ -f $base/bin/adapter.pid ] ; then
echo "found adapter.pid , Please run stop.sh first ,then startup.sh" 2>&2
exit 1
fi
if [ ! -d $base/logs ] ; then
mkdir -p $base/logs
fi
## set java path
if [ -z "$JAVA" ] ; then
JAVA=$(which java)
fi
ALIBABA_JAVA="/usr/alibaba/java/bin/java"
TAOBAO_JAVA="/opt/taobao/java/bin/java"
if [ -z "$JAVA" ]; then
if [ -f $ALIBABA_JAVA ] ; then
JAVA=$ALIBABA_JAVA
elif [ -f $TAOBAO_JAVA ] ; then
JAVA=$TAOBAO_JAVA
else
echo "Cannot find a Java JDK. Please set either set JAVA or put java (>=1.5) in your PATH." 2>&2
exit 1
fi
fi
case "$#"
in
0 )
;;
2 )
if [ "$1" = "debug" ]; then
DEBUG_PORT=$2
DEBUG_SUSPEND="n"
JAVA_DEBUG_OPT="-Xdebug -Xnoagent -Djava.compiler=NONE -Xrunjdwp:transport=dt_socket,address=$DEBUG_PORT,server=y,suspend=$DEBUG_SUSPEND"
fi
;;
* )
echo "THE PARAMETERS MUST BE TWO OR LESS.PLEASE CHECK AGAIN."
exit;;
esac
str=`file -L $JAVA | grep 64-bit`
if [ -n "$str" ]; then
JAVA_OPTS="-server -Xms2048m -Xmx3072m -Xmn1024m -XX:SurvivorRatio=2 -Xss256k -XX:+DisableExplicitGC -XX:+HeapDumpOnOutOfMemoryError"
else
JAVA_OPTS="-server -Xms1024m -Xmx1024m -XX:NewSize=256m -XX:MaxNewSize=256m -XX:MaxPermSize=128m "
fi
JAVA_OPTS=" $JAVA_OPTS -Djava.awt.headless=true -Djava.net.preferIPv4Stack=true -Dfile.encoding=UTF-8"
ADAPTER_OPTS="-DappName=canal-adapter"
for i in $base/lib/*;
do CLASSPATH=$i:"$CLASSPATH";
done
CLASSPATH="$base/conf:$CLASSPATH";
echo "cd to $bin_abs_path for workaround relative path"
cd $bin_abs_path
echo CLASSPATH :$CLASSPATH
exec $JAVA $JAVA_OPTS $JAVA_DEBUG_OPT $ADAPTER_OPTS -classpath .:$CLASSPATH com.alibaba.otter.canal.adapter.launcher.CanalAdapterApplication
打包镜像
docker build -t canal-adapter:1.1.7 .
报错“found adapter.pid , Please run stop.sh first ,then startup.sh”
1、如果服务启动,需要先执行stop.sh,在执行startup.sh
2、如果是刚构建成功启动startup.sh,可以删除adapter.pid文件在执行startup.sh
验证canal-adapter成功同步ES数据
插入一条数据到mysql表,在es查询数据,并在canal-adapter日志查看信息