Eclipse查看flume源码

最新推荐文章于 2023-04-16 20:55:46 发布
最新推荐文章于 2023-04-16 20:55:46 发布
阅读量1.6k 收藏 1
点赞数
分类专栏： Flume 文章标签： Flume eclipse
版权声明：本文为博主原创文章，遵循 CC 4.0 BY-SA 版权协议，转载请附上原文出处链接和本声明。
本文链接：https://blog.csdn.net/u012689336/article/details/52700621
版权

首先电脑要安装Maven

本文使用的是：apache-maven-3.0.5-bin.tar.gz

解压，配置环境变量。


Maven安装好后检查，出现下面信息表示安装Maven成功。


C:\Users\Administrator>mvn
[INFO] Scanning for projects...
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 0.080s
[INFO] Finished at: Thu Sep 29 11:44:05 CST 2016
[INFO] Final Memory: 4M/182M
[INFO] ------------------------------------------------------------------------
[ERROR] No goals have been specified for this build. You must specify a valid li
fecycle phase or a goal in the format <plugin-prefix>:<goal> or <plugin-group-id
>:<plugin-artifact-id>[:<plugin-version>]:<goal>. Available lifecycle phases are
: validate, initialize, generate-sources, process-sources, generate-resources, p
rocess-resources, compile, process-classes, generate-test-sources, process-test-
sources, generate-test-resources, process-test-resources, test-compile, process-
test-classes, test, prepare-package, package, pre-integration-test, integration-
test, post-integration-test, verify, install, deploy, pre-site, site, post-site,
 site-deploy, pre-clean, clean, post-clean. -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e swit
ch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please rea
d the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/NoGoalSpecifie
dException



下载源码：

apache-flume-1.5.0-src.tar.gz


开始编译：

E:\software\apache-flume-1.5.0-src>mvn install -DskipTests


出现的问题：

[ERROR] Failed to execute goal on project flume-ng-morphline-solr-sink: Could not resolve dependencies for project org.apache.flume.flume-ng-sinks:flume-ng-morphline-solr-sink:jar:1.5.0: Failed to collect dependencies for [org.apache.flume:flume-ng-core:jar:1.5.0 (compile), org.slf4j:slf4j-api:jar:1.6.1 (compile), org.kitesdk:kite-morphlines-all:pom:0.12.0 (compile?), org.slf4j:jcl-over-slf4j:jar:1.6.1 (provided), org.apache.solr:solr-test-framework:jar:4.3.0 (test), org.kitesdk:kite-morphlines-solr-core:jar:tests:0.12.0 (test), junit:junit:jar:4.10 (test)]: Failed to read artifact descriptor for ua_parser:ua-parser:jar:1.3.0: Could not transfer artifact ua_parser:ua-parser:pom:1.3.0 from/to maven-twttr (http://maven.twttr.com): Connection to http://maven.twttr.com refused: Connection timed out: connect -> [Help 1]


解决办法：

进入apache-flume-1.5.0-src\flume-ng-sinks\flume-ng-morphline-solr-sink\目录下pom.xml文件：

将<<repositories>>....</repositories>里面的内容注释掉。

然后加上：

<repository>
 <id>p2.jfrog.org</id>
 <url>http://p2.jfrog.org/libs-releases</url>
</repository>




[ERROR] Failed to execute goal on project flume-ng-morphline-solr-sink: Could not resolve dependencies for project org.apache.flume.flume-ng-sinks:flume-ng-morphline-solr-sink:jar:1.5.0: Failed to collect dependencies for [org.apache.flume:flume-ng-core:jar:1.5.0 (compile), org.slf4j:slf4j-api:jar:1.6.1 (compile), org.kitesdk:kite-morphlines-all:pom:0.12.0 (compile?), org.slf4j:jcl-over-slf4j:jar:1.6.1 (provided), org.apache.solr:solr-test-framework:jar:4.3.0 (test), org.kitesdk:kite-morphlines-solr-core:jar:tests:0.12.0 (test), junit:junit:jar:4.10 (test)]: Failed to read artifact descriptor for org.apache.hadoop:hadoop-common:jar:1.0.1: Could not transfer artifact org.apache.hadoop:hadoop-common:pom:1.0.1 from/to maven-twttr (http://maven.twttr.com): Failed to transfer file: http://maven.twttr.com/org/apache/hadoop/hadoop-common/1.0.1/hadoop-common-1.0.1.pom. Return
 code is: 504 , ReasonPhrase:Gateway Timeout. -> [Help 1]


解决办法：

修改apache-flume-1.5.0-src目录下的pom.xml文件，将其中

<kite.version>0.12.0</kite.version>，修改为<kite.version>0.15.0</kite.version>。


如果还有错误的话，修改apache-flume-1.5.0-src\flume-ng-sinks\flume-ng-morphline-solr-sink\目录下pom.xml文件：

添加：

<repository>
<id>maven-restlet</id>
<name>Public online Restlet repository</name>
<url>http://maven.restlet.org</url>
</repository>




出现下面信息，表示编译成功。

[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Apache Flume ...................................... SUCCESS [1.157s]
[INFO] Flume NG SDK ...................................... SUCCESS [2.330s]
[INFO] Flume NG Configuration ............................ SUCCESS [0.340s]
[INFO] Flume NG Core ..................................... SUCCESS [2.700s]
[INFO] Flume NG Sinks .................................... SUCCESS [0.110s]
[INFO] Flume NG HDFS Sink ................................ SUCCESS [0.480s]
[INFO] Flume NG IRC Sink ................................. SUCCESS [0.210s]
[INFO] Flume NG Channels ................................. SUCCESS [0.070s]
[INFO] Flume NG JDBC channel ............................. SUCCESS [0.270s]
[INFO] Flume NG file-based channel ....................... SUCCESS [0.490s]
[INFO] Flume NG Spillable Memory channel ................. SUCCESS [0.260s]
[INFO] Flume NG Node ..................................... SUCCESS [0.340s]
[INFO] Flume NG Embedded Agent ........................... SUCCESS [0.530s]
[INFO] Flume NG HBase Sink ............................... SUCCESS [0.530s]
[INFO] Flume NG ElasticSearch Sink ....................... SUCCESS [0.380s]
[INFO] Flume NG Morphline Solr Sink ...................... SUCCESS [4.030s]
[INFO] Flume Sources ..................................... SUCCESS [0.070s]
[INFO] Flume Scribe Source ............................... SUCCESS [0.852s]
[INFO] Flume JMS Source .................................. SUCCESS [1.460s]
[INFO] Flume Twitter Source .............................. SUCCESS [0.950s]
[INFO] Flume legacy Sources .............................. SUCCESS [0.050s]
[INFO] Flume legacy Avro source .......................... SUCCESS [1.100s]
[INFO] Flume legacy Thrift Source ........................ SUCCESS [1.190s]
[INFO] Flume NG Clients .................................. SUCCESS [0.060s]
[INFO] Flume NG Log4j Appender ........................... SUCCESS [3.684s]
[INFO] Flume NG Tools .................................... SUCCESS [2.792s]
[INFO] Flume NG distribution ............................. SUCCESS [5.960s]
[INFO] Flume NG Integration Tests ........................ SUCCESS [1.070s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 33.946s
[INFO] Finished at: Thu Sep 29 12:38:14 CST 2016
[INFO] Final Memory: 75M/1618M
[INFO] ------------------------------------------------------------------------



然后执行下面这个：

E:\software\apache-flume-1.5.0-src>mvn eclipse:eclipse -DdownloadSources


出现下面信息表示成功。

[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Apache Flume ...................................... SUCCESS [0.785s]
[INFO] Flume NG SDK ...................................... SUCCESS [1:09.714s]
[INFO] Flume NG Configuration ............................ SUCCESS [16.472s]
[INFO] Flume NG Core ..................................... SUCCESS [1:13.887s]
[INFO] Flume NG Sinks .................................... SUCCESS [0.020s]
[INFO] Flume NG HDFS Sink ................................ SUCCESS [2:14.869s]
[INFO] Flume NG IRC Sink ................................. SUCCESS [1.180s]
[INFO] Flume NG Channels ................................. SUCCESS [0.040s]
[INFO] Flume NG JDBC channel ............................. SUCCESS [8.272s]
[INFO] Flume NG file-based channel ....................... SUCCESS [4.770s]
[INFO] Flume NG Spillable Memory channel ................. SUCCESS [0.210s]
[INFO] Flume NG Node ..................................... SUCCESS [2.543s]
[INFO] Flume NG Embedded Agent ........................... SUCCESS [0.290s]
[INFO] Flume NG HBase Sink ............................... SUCCESS [2:38.160s]
[INFO] Flume NG ElasticSearch Sink ....................... SUCCESS [1:59.248s]
[INFO] Flume NG Morphline Solr Sink ...................... SUCCESS [16:15.959s]
[INFO] Flume Sources ..................................... SUCCESS [0.030s]
[INFO] Flume Scribe Source ............................... SUCCESS [0.160s]
[INFO] Flume JMS Source .................................. SUCCESS [7:08.242s]
[INFO] Flume Twitter Source .............................. SUCCESS [31.244s]
[INFO] Flume legacy Sources .............................. SUCCESS [0.030s]
[INFO] Flume legacy Avro source .......................... SUCCESS [0.230s]
[INFO] Flume legacy Thrift Source ........................ SUCCESS [0.140s]
[INFO] Flume NG Clients .................................. SUCCESS [0.020s]
[INFO] Flume NG Log4j Appender ........................... SUCCESS [0.200s]
[INFO] Flume NG Tools .................................... SUCCESS [0.310s]
[INFO] Flume NG distribution ............................. SUCCESS [0.260s]
[INFO] Flume NG Integration Tests ........................ SUCCESS [1.280s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 33:49.445s
[INFO] Finished at: Thu Sep 29 13:14:57 CST 2016
[INFO] Final Memory: 44M/443M
[INFO] ------------------------------------------------------------------------



上面编译之后，在eclipse下，Import–>Existing Projects into Workspace，然后选择flume编译源码的路径即可，结果如下图所示：






Polaris-zlf
关注
  • 0
    点赞
  • 踩
  • 1
    收藏
    觉得还不错? 一键收藏
  • 0
    评论
专栏目录
Apache Flume 源码解析
loki的博客
01-04 957
第一步：源码下载 进入官网：http://flume.apache.org/index.html 进入下载页面： 点击下载：https://mirrors.bfsu.edu.cn/apache/flume/1.9.0/apache-flume-1.9.0-src.tar.gz 第二步：加压项目,导入IDE 解压到目录： 导入到IDE： 第三步：查看启动脚本，找到程序入口 查看目录：bin/flume-ng 找到启动类：org.apache.flume.no.
Eclipse中查看源代码
m0_67306446的博客
05-11 3039
找到jdk的安装位置，可以看到一个src.zip文件，这就是jdk的源代码，我们先把路径复制下来： 点击Attach Source…按钮，进行配置： [外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-PmxXUVnd-1652250352283)(https://im 《一线大厂Java面试题解析+后端开发学习笔记+最新架构讲解视频+实战项目源码讲义》无偿开源 威信搜索公众号【编程进阶路】 g-blog.csdnimg.cn/a7ec9a6ae77b4dcbbe5b6c1db9
16_Flume之源码分析
weixin_40727028的博客
04-16 359
随后，这个轮询线程会通过 Guava 的 EventBus 机制通知 Application 类配置发生了更新，从而触发 Application#handleConfigurationEvent 方法，重新加载所有的组件。组件的start方法是在 MonitorRunnable 的run方法中调用的，MonitorRunnable 是一个线程，run方法中会获取当前组件的状态(默认STOP)以及预期的状态(START)，如果不相等则通过预期的状态执行相应的逻辑（ETCD直呼内行~~~）
flume执行流程及源码解读
weixin_40809627的博客
12-30 868
1、flume 指定配置文件启动 bin/flume-ng agent -c conf -f conf/LogDetailExpose4KafkaFlume.conf -n aApacheFlume --classpath /home/dmpflume-jar-with-dependencies.jar -Dflume.root.logger=INFO,console #注意：因为需要指定自己实现...
Flume 1.9.0 源码解析 : 一篇文章弄清Flume 程序启动流程
张伯毅的专栏
03-16 4603
一 .环境准备. 1.配置文件: example.conf # example.conf: A single-node Flume configuration # Name the components on this agent a1.sources = r1 a1.sinks = k1 a1.channels = c1 # Describe/configure t...
【大数据】Flume源码分析之启动过程
像风走过八千里
06-20 870
要分析flume的源码，首先要从Flume的启动开始，然后再分析Flume的各个组件服务Source、Channel、Sink的源码，接下来我们就来看下Flume是如何启动的吧 Flume的启动入口 org.apache.flume.node.Application 1、校验命令行 Options options = new Options(); Option option = new Option("n", "name", true, "the name of this agent".
flume 1.8所有源代码 编译通过版 附 maven3.5.2 安装包
08-23
在Eclipse中，你可以浏览源代码，查看Flume的工作流程，包括数据源（Sources）、处理器（Channels）和数据接收器（Sinks）的实现。对于开发者来说，理解这些组件如何协同工作以实现高效的数据流传输至关重要。同时，...
flume_jars.zip
03-18
使用IDE如IntelliJ IDEA或Eclipse进行Flume项目开发时，必须正确导入这些JAR包，以确保代码能够正常编译和运行。 Flume的核心功能是将数据从源（sources）收集，通过通道（channels）暂存，并最终传输到接收器...
JAVA上百实例源码以及开源项目源代码
09-17
　Java绘制图片火焰效果，源代码相关注释：前景和背景Image对象、Applet和绘制火焰的效果的Image对象、Applet和绘制火焰的效果的Graphics对象、火焰效果的线程、Applet的高度,图片到图片装载器、绘制火焰效果的X坐标...
flume1.7.0源码
05-19
flume源码
编译flume源码遇到的一些问题
Defender_bakery的博客
05-10 996
在编译flume源码时遇到了一些jar包找不到导致编译失败的问题 ** 1.flume-ng-morphline-solr-sink:Failed to execute goal on project flume-ng-morphline-solr-sink ** [ERROR] Failed to execute goal on project flume-ng-morphline-solr-sink: Could not resolve dependencies for project org.a.
另一种解决Failed to execute goal on project xxx: Could not resolve dependencies for project com的方法
热门推荐
two_brother_的博客
02-02 2万+
解决Failed to execute goal on project 或 Could not find artifact com.atguigu:store-commons:jar 或 Unresolved dependency: 'com.atguigu:store-commons:jar:1.0.0
flume源码分析
亚瑟的守护的专栏
10-08 2564
1、LifecycleAware public interface LifecycleAware {    public void start();    public void stop();    public LifecycleState getLifecycleState();  } 所有核心组件，全都实现这个接口。通过start，stop启动关闭，通过
flume 源码解读一
u012842247的专栏
11-24 613
通过查看flume-ng.sh 脚本可以知道 flume 首先启动是哪个类？ //位于 flume-ng-node 包中 org.apache.flume.node.Application 1、 flume 有三大模块： source 、 channel 、 sink 。在三个又有 选择器、拦截器等，来自官网的图例。 2、如何启动一个agent代理，拿官网的例子使用，我们的...
数据采集框架 Flume
weixin_43589563的博客
10-07 1193
Flume官网 （1）Flume官网地址：http://flume.apache.org/ （2）文档查看地址：http://flume.apache.org/FlumeUserGuide.html （3）下载地址：http://archive.apache.org/dist/flume/ Flume概述 (1)Flume是cdh公司提供的 (2)海量日志采集、聚合和传输的框架 (3)高可用的，高可靠的，分布式 高可用：flume挂了以后还有别的flume可以代替来工作 高可靠：数据传输可靠，不丢失 分布
02. flume-ng 核心源码解析【小明同学】
u011046509的博客
04-06 593
文章目录flume-ng 核心代码解析org.apache.flume.node.Application#main 解析PollingPropertiesFileConfigurationProvider 构造函数Application 构造函数eventBus.register(application)然后我们继续看下 application.start() 方法，不用想，我们的核心入口就在这个 start 方法里了。接下来我们就看下 org.apache.flume.lifecycle.Lifecycl
flume 1.9 源码阅读(一)
Cynric的专栏
10-29 800
Flume源码阅读Flume NG和 Flume OG 相比Flume 源码目录结构Flume 结构数据流模型 参考: https://www.jianshu.com/p/0187459831af http://flume.apache.org/FlumeUserGuide.html Flume NG Flume 是一个分布式的、可靠的、高效的日志采集、聚合、转移系统。他可以从多个不同的Sourc...
Flume源码分析—利用Eclipse对Flume源码进行远程调试分析环境搭建（一）
muriyi的博客
04-14 4614
一、引言 最近在学习了解大数据分析相关的工作，对于其中用到的收集部分用到了flume，特意花了点时间了解了一下flume的工作原理及工作机制。个人了解一个新的系统首先从概略上了解一下其基本原理后，接着从源代码入手了解其部分关键实现部分，最后会尝试去修改部分内容，从而加深对其的理解。有关flume原理部分相关资料网上很多，这里介绍一下源码分析过程。 二、环境介绍 1、apache-flume-1
查看Flume代理名称
最新发布
06-10
要查看 Flume 代理的名称，您可以使用以下命令： ``` ps -ef | grep flume ``` 这个命令会列出正在运行的 Flume 代理的信息，包括代理的名称、进程 ID 等。在这个列表中，您可以找到您想要查看的代理的名称。 ...

“相关推荐”对你有帮助么？

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
评论
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值