hive python udf_python写hive的UDF问题

我用python写UDF函数时总报错：python：importsysdefmain():forlineinsys.stdin:line=line.strip()fields=line.split('\t')iffields[2]isNoneorfields[2]=""fields[2]="o"print'%s\t%...

我用python写UDF函数时总报错：

python：

import sys

def main():

for line in sys.stdin:

line = line.strip()

fields = line.split('\t')

if fields[2] is None or fields[2]=""

fields[2] = "o"

print '%s\t%s\t%s' % (fields[0],fields[1],fields[2])

if __name__ == "__main__":

main()

hql语句：

select transform(device_id,device_type,gender) using 'python gender.py' as(device_id,device_type,gender) from dm_device_gender where year=2017 and month=02 and day=23 limit 20;

报错：

Error: java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing row {"device_id":"pom7vptz-wnu7-tvxj-xeqm-7anh1ga5vo9x","device_type":"gaid","package_names":"com.k12.equivalenceTiles","source":"m:dsp","gender":"f","ratio":"0.5002738483938325","tag":"calc","year":"2017","month":"02","day":"23"}

at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.map(ExecMapper.java:195)

at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)

at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:455)

请问大神这是什么原因？

python改成这样就没事：

import sys

def main():

for line in sys.stdin:

line = line.strip()

fields = line.split('\t')

if fields[2] is None or fields[2]=""

fields[2] = "o"

try:

print '%s\t%s\t%s' % (fields[0],fields[1],fields[2])

except ValueError:

continue

if __name__ == "__main__":

main()

但是好像缺少数据。

展开