mysql nutch_eclipse环境下 Nutch+Mysql 二次开发环境搭建

6.配置conf/nutch-site.xml

http.agent.name

YourNutchSpider

http.accept.language

ja-jp, en-us,en-gb,en,zh-cn,zh-tw;q=0.7,*;q=0.3

Value of the “Accept-Language” request header field.

This allows selecting non-English language as default one to retrieve.

It is a useful setting for search engines build for certain national group.

parser.character.encoding.default

utf-8

The character encoding to fall back to when no other information

is available

plugin.folders

src/plugin

Directories where nutch plugins are located. Each

element may be a relative or absolute path. If absolute, it is used

as is. If relative, it is searched for on the classpath.