TOKENIZE 分割函数的使用:
1. 假设有如下两个文件:test_tokenize.pig, log
log 内容如下:
---------------------------------------
1 abc,zhoujie
2 mmm,liudehua
3 nnn*zhourunfa
0 nnn()zhouxingchi
5 xxx(zhoujielun)zhouhaimei(zhouhuimin
4 abc
5 nnn
2 yyy
---------------------------------------
test_tokenize.pig内容如下:
------------------------------------------
DEFINE SUB InvokerForString('java.lang.String.substring', 'String');
A = LOAD 'log' AS (fhour:int , fallyesid:chararray );
B = FOREACH A GENERATE TOKENIZE(fallyesid);
--DUMP A;
DUMP B;
-------------------------------------------
2. 执行命令:pig -x local test_tokenize.pig,输出如下:
--------------------------------------------------
({(abc),(zhoujie)})
({(mmm),(liudehua)})
({(nnn),(zhourunfa)})
({(nnn),(zhouxingchi)})
({(xxx),(zhoujielun),(zhouhaimei),(zhouhuimin)})
({(abc)})
({(nnn)})
({(yyy)})
--------------------------------------------------