1. 当打开调度任务,执行任务时,发现执行失败。然后去查看执行任务的日志:报了下面的错。
[ERROR] 2023-12-01 14:50:30.320 TaskLogLogger-class org.apache.dolphinscheduler.plugin.task.sql.SqlTask:[209] - execute sql error: errCode = 2, detailMessage = driver connect Error: 01000 [unixODBC][Driver Manager]Can't open lib '/usr/local/lib/libmyodbc8w.so' : file not found (0)
[ERROR] 2023-12-01 14:50:30.326 TaskLogLogger-class org.apache.dolphinscheduler.plugin.task.sql.SqlTask:[160] - sql task error: java.sql.SQLException: errCode = 2, detailMessage = driver connect Error: 01000 [unixODBC][Driver Manager]Can't open lib '/usr/local/lib/libmyodbc8w.so' : file not found (0)
2.然后去查看doris的日志,发现有一个新加的be节点报了这个错。
于是比较新节点和旧节点的配置,发现都是一样的。
3.然后去查阅资料发现,确是odbc的驱动安装有问题。
于是将其他be节点的驱动的/usr/local/lib/的目录,copy到新节点下。
然后通过查看odbc配置的实例,这个实例可以提供给doris创建外表来使用。
CREATE EXTERNAL TABLE `external_table` (
`id` bigint(20) NOT NULL COMMENT "主键id",
`company_code` varchar(50) NULL COMMENT "公司编码",
`company_name` varchar(200) NULL COMMENT "公司名称",
`account_year` varchar(200) NULL COMMENT "会计年度",
`account_period` varchar(200) NULL COMMENT "会计期间",
`accounts_payable_term_end_balance` decimal(20, 9) NULL COMMENT "应付账款期末余额",
`source_data_create_time` datetime NULL COMMENT "来源数据创建时间(用于同步数据作为查询条件)",
`is_sync_cloud` int(11) NULL DEFAULT "0" COMMENT "是否同步云仓:{0:否,1:是}",
`is_sync_center` int(11) NULL DEFAULT "0" COMMENT "是否同步数据中心::{0:否,1:是}",
`creator` varchar(50) NULL COMMENT "创建人",
`create_time` datetime NULL COMMENT "创建时间",
`updater` varchar(50) NULL COMMENT "更新人",
`update_time` datetime NULL COMMENT "更新时间",
`remark` varchar(2000) NULL DEFAULT "" COMMENT "备注"
) ENGINE=ODBC
COMMENT "苍穹-财务-应付"
PROPERTIES (
"odbc_catalog_resource" = "bywl_transfer_resource",
"database" = "bywl_cloud_data_transfer",
"table" = "external_table"
);
vi /etc/odbc.ini 可以查看odbc配置的实例:
DSN=datasource-mysql
description = Data source MySQL
Driver = MySQL ODBC 8.0 Driver
Server = mysql_ip
Host = mysql_ip
Database = bywl_cloud_data_transfer
Port = 3306
User = root
Password = 123456
dsn时数据源名称,可以用于查看数据源是否来联通
isql -v datasource-mysql
vi /etc/odbcinst.ini
[PostgreSQL]
Description = ODBC for PostgreSQL
Driver = /usr/lib/psqlodbc.so
Setup = /usr/lib/libodbcpsqlS.so
FileUsage = 1
# Driver from the mysql-connector-odbc package
# Setup from the unixODBC package
[MySQL ODBC 8.0 Unicode Driver]
Description = ODBC for MySQL
Driver = /usr/lib64/libmyodbc8w.so
FileUsage = 1
# Driver from the oracle-connector-odbc package
# Setup from the unixODBC package
[Oracle 19 ODBC driver]
Description=Oracle ODBC driver for Oracle 19
Driver=/usr/lib/libsqora.so.19.1
[MySQL ODBC 8.0 Driver]
Description = ODBC for MySQL
Driver= /usr/local/lib/libmyodbc8w.so
FileUsage = 1
从这里看到MySQL ODBC 8.0 Driver 的配置,以及lib路径:
/usr/local/lib/libmyodbc8w.so
前面我已经把这个路径的包已经copy到新节点了。但是去执行任务依然报相同的错。
4.然后想着驱动还是有可能有问题。
于是查阅资料了解到,可以用ldd命令查看odbc库是否安装成功。
ldd /usr/local/lib/libmyodbc8w.so
linux-vdso.so.1 => (0x00007ffc00d59000)
libodbcinst.so.2 => not found
libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f5fd3d26000)
librt.so.1 => /lib64/librt.so.1 (0x00007f5fd3b1e000)
libcrypto.so.1.1 => /usr/local/lib/private/libcrypto.so.1.1 (0x00007f5fd366c000)
libssl.so.1.1 => /usr/local/lib/private/libssl.so.1.1 (0x00007f5fd33dc000)
libdl.so.2 => /lib64/libdl.so.2 (0x00007f5fd31d8000)
libresolv.so.2 => /lib64/libresolv.so.2 (0x00007f5fd2fbe000)
libm.so.6 => /lib64/libm.so.6 (0x00007f5fd2cbc000)
libstdc++.so.6 => /lib64/libstdc++.so.6 (0x00007f5fd29b4000)
libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007f5fd279e000)
libc.so.6 => /lib64/libc.so.6 (0x00007f5fd23d0000)
/lib64/ld-linux-x86-64.so.2 (0x00007f5fd4aeb000)
libltdl.so.7 => /lib64/libltdl.so.7 (0x00007f5fd21c6000)
发现确实有个库包不存在。
然后去可用的be节点,使用ldd查看,发现其他be节点没有这个问题,然后找到相应的路径复制古来。再用ldd检查,最终没有问题:
linux-vdso.so.1 => (0x00007ffc00d59000)
libodbcinst.so.2 => /lib64/libodbcinst.so.2 (0x00007f5fd3f42000)
libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f5fd3d26000)
librt.so.1 => /lib64/librt.so.1 (0x00007f5fd3b1e000)
libcrypto.so.1.1 => /usr/local/lib/private/libcrypto.so.1.1 (0x00007f5fd366c000)
libssl.so.1.1 => /usr/local/lib/private/libssl.so.1.1 (0x00007f5fd33dc000)
libdl.so.2 => /lib64/libdl.so.2 (0x00007f5fd31d8000)
libresolv.so.2 => /lib64/libresolv.so.2 (0x00007f5fd2fbe000)
libm.so.6 => /lib64/libm.so.6 (0x00007f5fd2cbc000)
libstdc++.so.6 => /lib64/libstdc++.so.6 (0x00007f5fd29b4000)
libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007f5fd279e000)
libc.so.6 => /lib64/libc.so.6 (0x00007f5fd23d0000)
/lib64/ld-linux-x86-64.so.2 (0x00007f5fd4aeb000)
libltdl.so.7 => /lib64/libltdl.so.7 (0x00007f5fd21c6000)
4.最后一部验证是否可以通过
isql -v mysql datasource-mysql
查看是否可以联通