【爬虫】Nutch1.15 & Solr 8.2.0 配置

最新推荐文章于 2021-01-19 02:55:07 发布

飞奔的六六

最新推荐文章于 2021-01-19 02:55:07 发布

阅读量665

点赞数

分类专栏： env 文章标签：爬虫配置 nutch

本文链接：https://blog.csdn.net/Art1st_D/article/details/101431727

版权

本文档介绍了如何在CentOS 7.6环境下配置Nutch 1.15和Solr 8.2.0。Nutch的1.x版本比2.x稳定并拥有更多功能，而2.x提供了灵活的存储选项。在Nutch的配置中，包括JAVA_HOME设置、爬虫注入、生成、抓取、解析等步骤，而Solr配置涉及核心创建和版本兼容性问题。在遇到Nutch和Solr配置问题时，可以通过修改nutch-site.xml进行调整。

摘要由CSDN通过智能技术生成

文章目录

Nutch

Nutch

1.x vs 2.x

Nutch development has been focused mainly on 1.x for the last few years. 2.x was designed with Apache Gora, so you can have a wide variety of storage backends, which is very useful! But it hasn’t been stable for quite a few years, although it has improved very much! Unfortunately, many new features that are about crawling have been added to 1.x, not 2.x because it would take so much time for us to maintain two branches. Nutch 1.x has many bug fixes and many new features and plugins. Nutch 2.x has awesome flexible storage, but lacks many features that have been added to 1.x. On the bright side, a good number of cool features in 1.x are plugins which are easier to port ?

环境

CentOS 7.6
JDK 1.8
Nutch 1.15
Solr 8.2.0

安装过程

JAVA_HOME

export JAVA_HOME=/usr/lib/jvm/java-1.8.0-openjdk
export CLASSPATH=.: $JAVA_HOME/jre/lib/rt.jar:$ JAVA_HOME/lib/dt.jar: $JAVA_HOME/lib/tools.jar export PATH=$