qq_15009447的博客

建表插入create table ods_sail2(id bigInt,num int,sail_pirce decimal(6,2),discount DECIMAL(2,1),employee_id BIGINT,sail_date date)row format delimited fields terminated by ','stored as orcload data local inpath 'file:///root/testData/ods_sail.txt' i

create table userbehavior_partitioned2(user_id string,item_id string,category_id string,behavior_type string)partitioned by (time string)insert into userbehavior_partitioned2 partition(time)select user_id,item_id,category_id,behavior_type,from_u

Starting to launch local task to process map join; maximum memory = 518979584Exception in thread “main” java.lang.OutOfMemoryError: Java heap spacemapjoin的时候 OOM了。MapJoin简单逻辑：读取小文件，缓存到distributeCache 再从cache里边读，和大表进行关联。简单来说就是这边把表读到内存中，内存塞不下了爆了OOM.

代码create external table ${db}.mg_train(user_id String,event_id String,invited String,time_stamp String,interested String)stored by 'com.mongodb.hadoop.hive.MongoStorageHandler'with serdeproperties('mongo.columns.mapping'='{"user_id":"user","e

hive 映射hbase

hdfs 内存使用情况linux查看yarn状态命令yarn node -list allwep port 8088 查看yarn50070 dataNode不是很健康的节点 需要删除文件腾出空间 不然mapreduce不能运行查看当前目录下的文件占的内存大小du -h -x --max-depth=1//查看内存使用df -h如遇到hive不能起的情况（元数据相关的原因）尝试起mysql 和 如下命令hive --service metastore &删除库