Doris常见问题

常见问题

1. 内存超限。

Process has no memory available, cancel top memory usage load: load memory tracker consumption 12.34 GB, backend 10.10.20.14 process memory used 26.06 GB exceed limit 24.97 GB or sys mem available 3.43 GB less than low water mark 1.60 GB. Execute again after enough memory, details see be.INFO.

错误信息分为三部分：

1、Memory limit exceeded:：当前正在执行query 3c88608cf35c461d-95fe88969aa6fc30的内存申请过程中发现内存超限。

2、process memory used 2.68 GB exceed limit 2.47 GB or sys mem available 50.95 GB less than low water mark 3.20 GB, failed alloc size 2.00 MB：超限的原因是 BE 进程使用的内存 2.68GB 超过了 2.47GB 的limit，limit的值来自 be.conf 中的 mem_limit * system MemTotal，默认等于操作系统总内存的80%，当前操作系统剩余可用内存 50.95 GB 仍高于最低水位 3.2GB，本次尝试申请 2MB 的内存。

3、executing msg:>, backend 172.24.47.117 process memory used 2.68 GB, limit 2.47 GB：本次内存申请的位置是ExecNode:VAGGREGATION_NODE (id=7)>，当前BE节点的IP是 172.1.1.1，以及再次打印BE节点的内存统计。

1. Doris设置为1时，使用catalog访问Oracle表，会报错。

Doris 会默认将大写换成小写。此时使用catalog访问Oracle时，会出错，因为导入的是小写

1. dataX数据同步时，字符串中经常会有 '/',',','%'，会报错，提示写入失败，

使用 querySql配置，使用字符串函数替换。Oracle测试，基本不会影响同步的速度，根据字段值同步速度影响会有差异

错误信息：

2023-07-18 16:08:45.882 [Thread-1] INFO DorisStreamLoadObserver - Start to join batch data: rows[35007] bytes[18118626] label[datax_doris_writer_20e45ede-a7e8-4e19-9297-2a496ecbc9d5].

2023-07-18 16:08:45.892 [Thread-1] INFO DorisStreamLoadObserver - Executing stream load to: 'http://10.10.20.70:8030/api/db_test/DM_CUST_CONF/_stream_load', size: '18153634'

2023-07-18 16:08:46.498 [Thread-1] WARN DorisStreamLoadObserver - Request failed with code:500

2023-07-18 16:08:46.499 [Thread-1] INFO DorisStreamLoadObserver - StreamLoad response :null

2023-07-18 16:08:46.501 [Thread-1] WARN DorisWriterManager - Failed to flush batch data to Doris, retry times = 0

java.io.IOException: Unable to flush data to Doris: unknown result status.

at com.alibaba.datax.plugin.writer.doriswriter.DorisStreamLoadObserver.streamLoad(DorisStreamLoadObserver.java:66) ~[doriswriter-0.0.1-SNAPSHOT.jar:na]

at com.alibaba.datax.plugin.writer.doriswriter.DorisWriterManager.asyncFlush(DorisWriterManager.java:163) [doriswriter-0.0.1-SNAPSHOT.jar:na]

at com.alibaba.datax.plugin.writer.doriswriter.DorisWriterManager.access$000(DorisWriterManager.java:19) [doriswriter-0.0.1-SNAPSHOT.jar:na]

at com.alibaba.datax.plugin.writer.doriswriter.DorisWriterManager$1.run(DorisWriterManager.java:134) [doriswriter-0.0.1-SNAPSHOT.jar:na]