为了查看查询日志方便,安装插件auditloader,在doris中fe节点的doris/fe/audit_loader目录下有压缩包auditloader.zip,查看官网按照本想按照官网步骤进行安装,但是官网写的不清不楚,尝试在测试安装以失败告终。
各种尝试后确定了它的安装步骤:
1、解压auditloader.zip
unzip auditloader.zip
这时会产生四个文件:
2、这里需要对文件plugin.conf编辑
frontend_host_port、user、password三个按照自己的端口,doris用户名/密码编辑,我这里端口自己管理从8030改成了8131:
#这里ip是不用改的就是端口需要和自己的端口一样
frontend_host_port=127.0.0.1:8131
# Database of the audit table
#这里设的数据库是将要把查询日志放到数据库
database=doris_audit_db__
# Audit table name, to save the audit data.
#这里设置的是将要放的表格
table=doris_audit_tbl__
# Doris user. This user must have LOAD_PRIV to the audit table.
#这里是doris对上边设置的日志数据库有插入权限的用户
user=root
# Doris user's password
#对日志数据库有插入权限的用户的密码
password=XXXXXXXX
编辑后保存。
3、将四个文件重新压缩为auditloader.zip
zip -r -q -m auditloader.zip auditloader.jar plugin.properties plugin.conf
这时四个文件就会重新压缩为一个auditloader.zip文件,分发到所有fe节点上的doris/fe/audit_loader/的目录上。
4、在doris创建对应日志数据库和日志数据表
create database doris_audit_db__;
create table doris_audit_db__.doris_audit_tbl__
(
query_id varchar(48) comment "Unique query id",
`time` datetime not null comment "Query start time",
client_ip varchar(32) comment "Client IP",
user varchar(64) comment "User name",
db varchar(96) comment "Database of this query",
state varchar(8) comment "Query result state. EOF, ERR, OK",
error_code int comment "Error code of failing query.",
error_message string comment "Error message of failing query.",
query_time bigint comment "Query execution time in millisecond",
scan_bytes bigint comment "Total scan bytes of this query",
scan_rows bigint comment "Total scan rows of this query",
return_rows bigint comment "Returned rows of this query",
stmt_id int comment "An incremental id of statement",
is_query tinyint comment "Is this statemt a query. 1 or 0",
frontend_ip varchar(32) comment "Frontend ip of executing this statement",
cpu_time_ms bigint comment "Total scan cpu time in millisecond of this query",
sql_hash varchar(48) comment "Hash value for this query",
sql_digest varchar(48) comment "Sql digest of a slow query",
peak_memory_bytes bigint comment "Peak memory bytes used on all backends of this query",
stmt string comment "The original statement, trimed if longer than 2G "
) engine=OLAP
duplicate key(query_id, `time`, client_ip)
partition by range(`time`) ()
distributed by hash(query_id) buckets 1
properties(
"dynamic_partition.time_unit" = "DAY",
"dynamic_partition.start" = "-30",
"dynamic_partition.end" = "3",
"dynamic_partition.prefix" = "p",
"dynamic_partition.buckets" = "1",
"dynamic_partition.enable" = "true",
"replication_num" = "3"
);
在官网上没有说创建doris_audit_db__.doris_audit_tbl__表,这里我不是很理解是为何要创建那两个不在文件中配置的表,而且创建了后也没有日志信息进去,很是困惑。
5、通过以下语句安装 Audit Loader 插件,直接在doris执行:
INSTALL PLUGIN FROM "/opt/soft/doris/fe/audit_loader/auditloader.zip";
查看是否已经安装状态成功:
这里状态是INSTALLED就是安装基本完成。
到fe的文件下查看log日志
grep 'audit' fe.log
可以看到日志成功。
就可以在doris查询日志查询表了:
正式完成。
ps:如果中间安装失败可以卸载重新配置
UNINSTALL PLUGIN AuditLoader;