htrace

links

New in Cloudera Labs: Apache HTrace (incubating)

Installing htraced

我们hadoop集群使用的是CDH的源，所以直接 yum install Htrace-htraced 即可。或者到 http://archive.cloudera.com/cloudera-labs/htrace/ 地址自己下载安装包安装。
例如：
http://archive.cloudera.com/cloudera-labs/htrace/redhat/6/x86_64/htrace/1/RPMS/x86_64/
其他集群需要参考原博客安装。
安装完成后可以执行 htracedTool version 命令进行校验。

Configuring htraced

配置文件：/etc/htraced/conf/htraced-conf.xml

<configuration>
  <property>
    <name>htraced.log.level</name>
    <value>TRACE</value>
  </property>
  <property>
    <name>htraced.log.path</name>
    <value>/tmp/htraced.log</value>
  </property>
  <property>
    <name>web.address</name>
    <value>namenode:9096</value>
  </property>
  <property>
    <name>data.store.directories</name>
    <value>/data/htraced</value>
  </property>
  <property>
    <name>data.store.clear</name>
    <value>true</value>
  </property>
  <property>
    <name>data.store.span.buffer.size</name>
    <value>100</value>
  </property>
  <property>
    <name>metrics.heartbeat.period.ms</name>
    <value>60000</value>
  </property>
</configuration>

其中data.store.directories 文件夹需要提前配置好。

mkdir /data/htraced
chown htrace:htrace /data/htraced
chmod 755 /data/htraced

Hadoop 集群启用htrace:/etc/hadoop/conf/core-site.xml

  <!-- for htrace -->
  <property>
    <name>hadoop.htrace.span.receiver.classes</name>
    <value>org.apache.htrace.impl.HTracedSpanReceiver</value>
  </property>
  <property>
    <name>hadoop.htrace.htraced.receiver.address</name>
    <value>namenode:9075</value>
  </property>
  <property>
    <name>hadoop.htrace.htraced.error.log.period.ms</name>
    <value>10000</value>
  </property>
  <property>
    <name>fs.client.htrace.sampler.classes</name>
    <value>ProbabilitySampler</value>
  </property>
  <property>
    <name>hadoop.htrace.sampler.fraction</name>
    <value>0.001</value>
  </property>

The htraced Web UI

通过 htraced 命令启动接收htrace服务，且会启动一个WEB UI，地址在 http://host:9096

补充调试命令

Receiver 调试

$ su hdfs -c "hadoop trace -list -host namenode:8020"
ID  CLASS                                      
1   org.apache.htrace.impl.HTracedSpanReceiver

# 添加一个receiver

$ su hdfs -c "hadoop trace -add -class org.apache.htrace.core.LocalFileSpanReceiver -Chadoop.htrace.local.file.span.receiver.path=/tmp/htrace.out -host namenode:8020"

$ su hdfs -c "hadoop trace -list -host namenode:8020"
ID  CLASS                                        
1   org.apache.htrace.impl.HTracedSpanReceiver   
2   org.apache.htrace.core.LocalFileSpanReceiver

下面这个测试失败，没有在本地生产span接收文件：

su - hdfs -c "hdfs dfs -Dfs.shell.htrace.span.receiver.classes=LocalFileSpanReceiver -Dfs.shell.htrace.sampler.classes=AlwaysSampler -ls /"