hive导入 nginx 或 apache 日志

1、创建表

CREATE TABLE nginxlog (
 ipaddress STRING,
 identity STRING,
 user STRING,
 time STRING,
 request STRING,
 protocol STRING,
 status STRING,
 size STRING,
 referer STRING,
 agent STRING) 
ROW FORMAT SERDE 'org.apache.hadoop.hive.contrib.serde2.RegexSerDe' 
WITH SERDEPROPERTIES (
 "input.regex" = "([^ ]*) ([^ ]*) ([^ ]*) (-|\\[[^\\]]*\\]) ([^ \"]*|\"[^\"]*) ([^ ]*\") (-|[0-9]*) (-|[0-9]*)(?: ([^ \"]*|\".*\") ([^ \"]*|\".*\"))?","output.format.string" = "%1$s %2$s %3$s %4$s %5$s %6$s %7$s %8$s %9$s %10$s")
STORED AS TEXTFILE; 

2、加载日志数据

load data local inpath '/home/hadoop/anjl/access.log' overwrite into table nginxlog

3、查询

select * from nginxlog;


日志例子:

125.75.93.233 - - [14/Apr/2014:17:18:21 +0800] "GET /test/123.htm?a=1 HTTP/1.1" 200 14498 "-" -"Apache-HttpClient/UNAVAILABLE (java 1.4)" "-"
218.201.111.114 - - [14/Apr/2014:17:18:21 +0800] "GET /test/123.htm?a=1 HTTP/1.1" 200 17593 "-" -"Apache-HttpClient/UNAVAILABLE (java 1.4)" "-"
119.180.35.121 - - [14/Apr/2014:17:18:22 +0800] "GET /test/123.htm?a=1 HTTP/1.1" 200 19492 "-" -"Apache-HttpClient/UNAVAILABLE (java 1.4)" "-"




评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值