h大数据常用命令

查看tez任务:yarn application -list

查看hadoop任务：hadoop job -list

 

 

1.6.hive的一些常用命令

查询显示列名：set hive.cli.print.header=true;

a.查看存储在数据仓库中的数据文件：
hadoop fs -text /user/hive/warehouse/tb_yn_all_city_code/000000_0 |head

b.将数据仓库中的数据合并后下载到本地
hadoop fs -getmerge /user/hive/warehouse/tb_sub_imsi_usernum_final/ /home/zhaojing/shisuo/tb_sub_imsi_usernum_final.txt

c.hdfs上面查看文件并按文件大小排序并写入文件
hdfs dfs -du -s /user/hive/warehouse/* | sort -nr  > disk.txt 

d.hdfs上面查看文件取前十排序
hdfs dfs -du -s /user/hive/warehouse/* | sort -nr | head

 hive保存数据为scv : hive -e "set hive.cli.print.header=true; select * from tmp_user_info where some_query_conditions" | sed 's/[\t]/,/g'  > userinfo.csv

hive保存数据： set fileencoding=gbk;

hive向本地写数据

insert overwrite local directory '/web/soft/apache-hive-2.1.1-bin/result_csv/tmp_xxx' row format delimited fields terminated by ','
select distinct itv_account as usercode,parent_code as parentcode,status,team_id,stb_id as stbid,boss_usercode as bossusercode  from table_name;

本地数据加载到hive

alter table xxx drop partition(date_no= {Date_No});
alter table xxx add partition (date_no= {Date_No});
load data local inpath '/AMT/ftp/stbinfo/stbinfo_{Date_No}.txt' overwrite into table xxx partition (date_no= {Date_No});

查看集群运行状况：hadoop dfsadmin -report