解决Flume执行命令常遇错误

一.解决Flume执行命令时遇到的ChannelException和InterruptedException异常
 1.问题描述:
 在Flume的日志中，我们发现了如下异常信息：

2023-10-12 23:49:45,863 EFRCR scurce.ExecSource: Failed while running command: tail -F /opt/server/projectl.py
org.apache.flume.ChannelException: java.lang.InterruptedException  at.org.apache.flume.channel.BasicTransactionSemantics.commit(BasicTransactionSemantics.java:154)      
  at.org.apache.flume.channel.ChannelProcessor.processEventBatch(ChannelProcessor.java:19)
  at.org.apache.flume.source.ExecSource$ExecRunnable.flushEventBatch(ExecSource.java:385)
  at org.apache.flume.source.ExecSource$ExecRunnable.run(ExecSource.java:352)
  at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
  at java.util.concurrent.FutureTask.run(FutureTask.java:266)
  at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
  at java.lang.Thread.run(Thread.java:750)

2.问题原因:
 这个异常的出现通常是由于以下原因引起的：
 (1).资源不足：系统资源（例如内存或CPU）不足，导致Flume执行过程中被迫中断。

3.解决方法:
针对上述问题，我们可以采取以下措施来解决Flume遇到的ChannelException和InterruptedException异常：
  (1).增加系统资源：如果系统资源不足，可以考虑增加系统的内存或CPU。确保Flume在运行时有   足够的资源支持。把上述图片1000改为2000

二.解决Flume执行命令时遇到的时间戳的问题

1.问题描述:
 在Flume的日志中，我们发现了如下异常信息：

java.lang.NullPointerException: Expected timestamp in the Flume event headers, but it was null
        at com.google.common.base.Preconditions.checkNotNull(Preconditions.java:204)
        at org.apache.flume.formatter.output.BucketPath.replaceShorthand(BucketPath.java:200)
        at org.apache.flume.formatter.output.BucketPath.escapeString(BucketPath.java:396)
        at org.apache.flume.sink.hdfs.HDFSEventSink.process(HDFSEventSink.java:388)
        at org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:68)
        at org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:147)
        at java.lang.Thread.run(Thread.java:744)
2023-10-12 15:31:41,836 (SinkRunner-PollingRunner-DefaultSinkProcessor) [ERROR - org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:160)] Unable to deliver event. Exception follows.
org.apache.flume.EventDeliveryException: java.lang.NullPointerException: Expected timestamp in the Flume event headers, but it was null
        at org.apache.flume.sink.hdfs.HDFSEventSink.process(HDFSEventSink.java:471)
        at org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:68)
        at org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:147)
        at java.lang.Thread.run(Thread.java:744)
Caused by: java.lang.NullPointerException: Expected timestamp in the Flume event headers, but it was null
        at com.google.common.base.Preconditions.checkNotNull(Preconditions.java:204)
        at org.apache.flume.formatter.output.BucketPath.replaceShorthand(BucketPath.java:200)
        at org.apache.flume.formatter.output.BucketPath.escapeString(BucketPath.java:396)
        at org.apache.flume.sink.hdfs.HDFSEventSink.process(HDFSEventSink.java:388)
        ... 3 more

2.问题原因:
 这个异常的出现通常是由于以下原因引起的：
 (1).使用Flume的HDFSEventSink时，Flume事件中缺少预期的时间戳字段。

3.解决方法:

(1).加入下面代码：

a1.sinks.k1.hdfs.useLocalTimeStamp = true
a1.sinks.k1.hdfs.fileType = DataStream

三.解决Flume执行命令时遇到的guava-11.0.2.jar的版本不兼容

1.问题描述:
 在Flume的日志中，我们发现了如下异常信息：

2.问题原因:
 这个异常的出现通常是由于以下原因引起的：

 (1).flume安装目录中lib文件夹下的guava-11.0.2.jar的版本不兼容。

3.解决方法:

(1).删除guava包

[user@hadoop102 lib]$  rm /opt/module/flume/lib/guava-11.0.2.jar