Widrow-CSDN博客

转载 21款酷炫的数据可视化工具

http://mp.weixin.qq.com/s?__biz=MzAxNzcyMTk5NA==&mid=400593778&idx=5&sn=89fed707b7d7cba107dd2d08b1cd6eb8&scene=1&srcid=103047S0ZapN4k33pwZI7wFq&uin=OTgyMzY1MTQy&key=04dce534b3b035ef2630a550596950ec892

2016-04-11 09:03:56 926

转载 hive写入Elasticsearch参数设置

1. 在hive中增加number of Hadoop input splitsReduce the input split size from the default value. The mappers will get increased.SET mapreduce.input.fileinputformat.split.maxsize;参考：http://stack

2016-03-08 11:11:04 2842

原创 Hadoop Native Libraries

https://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/NativeLibraries.htmlWARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin

2016-03-03 17:04:42 459

原创 hadoop hive python 笔记

在ubuntu中安装pyhs2，用于使用python访问hive 使用pip安装pyhs2之前，需要先安装python-dev 、 libsasl2-dev和gcc ：apt-get install python-dev libsasl2-dev gcc 在CentOS/RHEL中使用yum（见pyhs2的github主页wiki）：yum install gc

2016-03-02 14:43:27 525

原创 python笔记

风格规范：http://zh-google-styleguide.readthedocs.org/en/latest/google-python-styleguide/python_style_rules/

2015-12-31 10:12:06 282

原创使用webhdfs

修改fs.defaultFS后应该：http://stackoverflow.com/questions/19431074/hive-not-fully-honoring-fs-default-name-fs-defaultfs-value-in-core-site-xmldfs.namenode.http-address默认端口为50070

2015-12-27 17:58:33 654

原创 hive笔记

HiveServer2 is a rewrite of HiveServer that addresses these problems, starting with Hive 0.11.0. Use of HiveServer2 is recommended.HiveServer was removed from Hive releases starting in Hive 1.0.

2015-12-21 10:40:15 688

原创 linux学习笔记

man: 显示命令manualiostat -y 省略第一条结果查看操作系统：内核：cat /proc/versionuname -alinux版本：cat /etc/issue查看磁盘：cat /sys/block/sda/queue/rotational为0则是SSD硬盘或者 lsscsi工具

2015-12-20 15:01:13 348

原创 Elasticsearch配置

环境变量：export ES_HEAP_SIZE=2gFile Descriptors：cat /proc/sys/fs/file-max/etc/security/limits.conf* soft nofile 4096* hard nofile 4096/etc/pam.d/common-session*ADD:session

2015-12-13 17:12:04 247

原创 151201笔记

Elasticsearch用filter_path指定返回数据，避免打印太多。不知是不是常规做法。shellwhile循环#!/bin/bashCOUNTER=0while [ $COUNTER -lt 180 ]do COUNTER=$[ $COUNTER + 1 ] echo $COUNTERdonesh脚本异常（转自h

2015-12-01 14:02:02 250

转载学习笔记

linux下如何查看权限http://zhidao.baidu.com/link?url=4z7f7pX5Js_A6MT07pc9fkVv2bcHqqmVJafhyOE_ixpYRokw_Ujd6TX8YZpRc0Tc9rQFviZiM68nEyfbrYMJ8_修改linux文件权限http://www.cnblogs.com/avril/archive/2010/03/2

2015-10-19 19:56:29 260