【Nutch】Linux下应用nutch-1.0Web前端实现HDFS数据检索

启动hadoop集群

配置/usr/program/apache-tomcat-6.0.18/webapps/nutch-1.0/WEB-INF/classes下的配置文件

hadoop-site.xml:

<property>
        <name>fs.default.name</name>
        <value>hdfs://gc03vm12:9000</value>
        <description> </description>
</property>

nutch-site.xml:

<property>
  <name>http.agent.name</name>
  <value>nutch-1.0</value>
  <description>HTTP 'User-Agent' request header.</description>
</property>

<property>
  <name>http.robots.agents</name>
  <value>nutch-1.0,*</value>
  <description>The agent strings we'll look for in robots.txt files,
  comma-separated, in decreasing order of precedence. You should
  put the value of http.agent.name as the first agent name, and keep the
  default * at the end of the list. E.g.: BlurflDev,Blurfl,*
  </description>
</property>

<property>
  <name>searcher.dir</name>
  <value>data</value>            爬虫数据的存放目录，hdfs的相对路径
  <description> index dir </description>
</property>

data目录所处位置如下：
[nutch@gc03vm12 nutch-1.0]$ bin/hadoop dfs -ls
Found 5 items
drwxr-xr-x   - nutch supergroup          0 2010-05-19 01:54 /user/nutch/data

重新启动tomcat

[nutch@gc03vm12 apache-tomcat-6.0.18]$ bin/shutdown.sh
[nutch@gc03vm12 apache-tomcat-6.0.18]$ bin/startup.sh

搜索