关闭

Hives UDF 编写入门

标签: mapreducecommandjobsfiletablekill
380人阅读 评论(0) 收藏 举报
分类:

大写转小写


package com.afan;
import org.apache.hadoop.hive.ql.exec.UDF;
import org.apache.hadoop.io.Text;

public class UDFLower extends UDF{
    public Text evaluate(final Text s){
        if (null == s){
            return null;
        }
        return new Text(s.toString().toLowerCase());
    }
}

1加载udf jar包
afan@ubuntu:/usr/local/hadoop/hive$ bin/hive
Hive history file=/tmp/afan/hive_job_log_afan_201105150623_175667077.txt
hive> add jar udf_hive.jar;
Added udf_hive.jar to class path
Added resource: udf_hive.jar


2 创建udf函数
hive> create temporary function my_lower as 'com.afan.UDFLower';
OK
Time taken: 0.253 seconds


3 创建测试数据
hive> create table dual (info string);
OK
Time taken: 0.178 seconds
hive> load data local inpath 'data.txt' into table dual;
Copying data from file:/usr/local/hadoop/hive/data.txt
Copying file: file:/usr/local/hadoop/hive/data.txt
Loading data to table default.dual
OK
Time taken: 0.377 seconds
hive> select info from dual;
Total MapReduce jobs = 1
Launching Job 1 out of 1
Number of reduce tasks is set to 0 since there's no reduce operator
Starting Job = job_201105150525_0003, Tracking URL = http://localhost:50030/jobdetails.jsp?jobid=job_201105150525_0003
Kill Command = /usr/local/hadoop/bin/../bin/hadoop job  -Dmapred.job.tracker=localhost:9001 -kill job_201105150525_0003
2011-05-15 06:46:05,459 Stage-1 map = 0%,  reduce = 0%
2011-05-15 06:46:10,905 Stage-1 map = 100%,  reduce = 0%
2011-05-15 06:46:13,963 Stage-1 map = 100%,  reduce = 100%
Ended Job = job_201105150525_0003
OK
WHO
AM
I
HELLO
worLd

Time taken: 14.874 seconds

4使用udf函数
hive> select my_lower(info) from dual;
Total MapReduce jobs = 1
Launching Job 1 out of 1
Number of reduce tasks is set to 0 since there's no reduce operator
Starting Job = job_201105150525_0002, Tracking URL = http://localhost:50030/jobdetails.jsp?jobid=job_201105150525_0002
Kill Command = /usr/local/hadoop/bin/../bin/hadoop job  -Dmapred.job.tracker=localhost:9001 -kill job_201105150525_0002
2011-05-15 06:43:26,100 Stage-1 map = 0%,  reduce = 0%
2011-05-15 06:43:34,364 Stage-1 map = 100%,  reduce = 0%
2011-05-15 06:43:37,484 Stage-1 map = 100%,  reduce = 100%
Ended Job = job_201105150525_0002
OK
who
am
i
hello
world

Time taken: 20.834 seconds

0
0

查看评论
* 以上用户言论只代表其个人观点,不代表CSDN网站的观点或立场
    个人资料
    • 访问:207977次
    • 积分:2568
    • 等级:
    • 排名:第14766名
    • 原创:50篇
    • 转载:47篇
    • 译文:0篇
    • 评论:18条
    最新评论