hadoop调用ruby代码

shell调用：

%x[ /opt/mapr/hadoop/hadoop-0.20.2/bin/hadoop \
jar /opt/mapr/hadoop/hadoop-0.20.2/contrib/streaming/hadoop-*streaming*.jar \
-D mapred.line.input.format.linespermap=1 \
-D mapred.reduce.tasks=0 \
-D mapred.map.tasks.speculative.execution=false \
-D mapred.job.name='dispatch.rb unique name' \
-mapper '/usr/local/bin/ruby dispatch.rb --map --log_interval=10000 --log_seconds=30' \
-reducer '' \
-input '#{hdfs_nodify_out}/dispatch.cfg' \
-output '#{hdfs_nodify_out}/dummy' \
-file '#{param['unix_root']}/dispatch.rb' \
-inputformat org.apache.hadoop.mapred.lib.NLineInputFormat ]

pig中调用：

DEFINE nodify `/usr/local/bin/ruby nodify.rb '$PARAMYML' '$REDISYML' --map` SHIP('$NODIFYSCRIPT','$REDISYML','$PARAMYML');