hive默认是以tab为分隔符,分隔各个输出字段,如
hive> select get_json_object(json.value,'$.hour'),get_json_object(json.value,"$.channel") from json limit 10;
Total MapReduce jobs = 1
Launching Job 1 out of 1
Number of reduce tasks is set to 0 since there's no reduce operator
Starting Job = job_201304271626_0032, Tracking URL = http://linjianke:50030/jobdetails.jsp?jobid=job_201304271626_0032
Kill Command = /usr/lib/hadoop/bin/hadoop job -Dmapred.job.tracker=192.168.10.44:8021 -kill job_201304271626_0032
2013-05-15 10:07:11,102 Stage-1 map = 0%, reduce = 0%
2013-05-15 10:07:15,116 Stage-1 map = 100%, reduce = 0%
2013-05-15 10:07:17,125 Stage-1 map = 100%, reduce = 100%
Ended Job = job_201304271626_0032
OK
2013-04-07 16 f32d98f8-8ac5-11e2-8a47-bc305bf4bcb8
2013-04-07 22 f32d98f8-8ac5-11e2-8a47-bc305bf4bcb8
2013-04-08 00 f32d98f8-8ac5-11e2-8a47-bc305bf4bcb8
2013-04-08 01 f32d98f8-8ac5-11e2-8a47-bc305bf4bcb8
2013-04-09 01 f32d98f8-8ac5-11e2-8a47-bc305bf4bcb8
2013-04-09 07 f32d98f8-8ac5-11e2-8a47-bc305bf4bcb8
2013-04-09 16 f32d98f8-8ac5-11e2-8a47-bc305bf4bcb8
2013-04-09 17 f32d98f8-8ac5-11e2-8a47-bc305bf4bcb8
2013-04-09 21 f32d98f8-8ac5-11e2-8a47-bc305bf4bcb8
2013-04-10 01 f32d98f8-8ac5-11e2-8a47-bc305bf4bcb8
Time taken: 9.531 seconds
因为业务需要,蒋输出转为json格式,可以用hive提供的transform函数
hive> select transform(get_json_object(json.value,'$.hour'),get_json_object(json.value,"$.channel")) using '/usr/bin/python transform.py aa bb' as (result string) from json limit 10;
输出为
{"aa":"2013-04-07 16","bb":"f32d98f8-8ac5-11e2-8a47-bc305bf4bcb8"}
{"aa":"2013-04-07 22","bb":"f32d98f8-8ac5-11e2-8a47-bc305bf4bcb8"}
{"aa":"2013-04-08 00","bb":"f32d98f8-8ac5-11e2-8a47-bc305bf4bcb8"}
{"aa":"2013-04-08 01","bb":"f32d98f8-8ac5-11e2-8a47-bc305bf4bcb8"}
{"aa":"2013-04-09 01","bb":"f32d98f8-8ac5-11e2-8a47-bc305bf4bcb8"}
{"aa":"2013-04-09 07","bb":"f32d98f8-8ac5-11e2-8a47-bc305bf4bcb8"}
{"aa":"2013-04-09 16","bb":"f32d98f8-8ac5-11e2-8a47-bc305bf4bcb8"}
{"aa":"2013-04-09 17","bb":"f32d98f8-8ac5-11e2-8a47-bc305bf4bcb8"}
{"aa":"2013-04-09 21","bb":"f32d98f8-8ac5-11e2-8a47-bc305bf4bcb8"}
{"aa":"2013-04-10 01","bb":"f32d98f8-8ac5-11e2-8a47-bc305bf4bcb8"}
python脚本:对python还不大熟悉,写的可能比较搓
#!/usr/bin/python
#coding:utf8
import sys
for line in sys.stdin:
if len(line) == 0:
#print 'end'
break
if line.count('\n') == len(line):
#print 'continue'
continue
line=line.strip('\n')
arr=line.split('\t')
if len(arr) != len(sys.argv) - 1:
print 'error'
break
for i in range(0,len(arr)):
arr[i]='"%s":"%s"' % (sys.argv[i+1],arr[i])
content = ','.join(arr)
print '{%s}' % content