2018年01月_houzhizhen

12月 11月 10月 09月 08月 07月 06月 05月 04月 03月 02月 01月

原创 hadoop 2.7.5 YARN ResouceManager Submit Application Process

1. Client calls submitApplication method of ClientRMService using RPC.2. ClientRMService. submitApplication calls RMAppManager.submitApplicationpublic SubmitApplicationResponse submitApplicat...

2018-01-29 17:00:02 1047

原创 NodeManager Health Check 经验总结

一、默认情况默认情况下，每个NdeManager自己的资源是在yarn-site.xml中配置的。如下面的配置时32个cpu cores，100G内存。现在集群中服务器cpu的线程数有32,40,48等几类。 property> name>yarn.nodemanager.resource.cpu-vcoresname> value>32value>

2018-01-28 15:20:39 1559

转载 Container killed by the ApplicationMaster, Exit code is 143

之前发现在map任务里面经常看到Container killed by the ApplicationMaster，挺奇怪，不过任务最终是成功的，就没怎么管。不过最近测试集群跑的任务报143错误，还是重新看一下这个问题。分析版本：hadoop cdh5.4错误日志2015-12-30 17:31:09,994 INFO [AsyncDispatcher event handler]

2018-01-26 09:20:07 15153 2

转载读懂 diff 命令

iff是Unix系统的一个很重要的工具程序。它用来比较两个文本文件的差异，是代码版本管理的基石之一。你在命令行下，输入：1 $diff< 变动前的文件 > ;diff就会告诉你，这两个文件有何差异。它的显示结果不太好懂，下面我就来说明，如何读懂diff。一、diff的三种格式

2018-01-12 15:21:53 469

原创 Upgrade YARN from Hadooop 2.6 to Hadoop 2.7

When resourcemanage is restarted, it may print the following error messages.2018-01-11 16:02:23,991 WARN org.apache.hadoop.yarn.server.resourcemanager.RMAuditLogger: USER=hadoop OPERATION=Applic

2018-01-12 14:47:27 440

原创 hadoop 2.7.5, hadoop 2.8.2, hadoop 2.9.0 proto 文件大小对比

[houzhizhen@localhost hadoop]$ sh proto_ls.sh -rw-r--r--. 1 houzhizhen root 2741 1月 4 16:09 ./hadoop-2.7.5-src/hadoop-hdfs-project/hadoop-hdfs/src/main/proto/acl.proto-rw-r--r--. 1 houzhizhen ro

2018-01-05 16:49:22 1268

Hive 性能测试程序，https://github.com/hortonworks/hive-testbench 编译

2023-02-24

Transactional Operations in Apache Hive:Present and Future

Hive 事务设计

2022-03-01

The Log-Structured Merge-Tree (LSM-Tree).pdf

2021-06-22

Distributed cycle detection in large-scale sparse graphs.pdf

分布式大规模环检测的计算方法。In this paper we present a distributed algorithm for detecting cycles in large-scale directed graphs, along with its correctness proof and analysis. The algorithm is then extended to find strong components in directed graphs. We indicate an application to detecting cycles in number theoretic functions such as the proper divisor function. Our prototype implementation of the cycle detection algorithm, when applied to the proper divisor function, detects all sociable groups of numbers (cycles in the proper divisor function) up to 107.

2020-08-11

空空如也

TA创建的收藏夹 TA关注的收藏夹

TA关注的人

houzhizhen的专栏

原创 hadoop 2.7.5 YARN ResouceManager Submit Application Process

原创 NodeManager Health Check 经验总结

转载 Container killed by the ApplicationMaster, Exit code is 143

转载读懂 diff 命令

原创 Upgrade YARN from Hadooop 2.6 to Hadoop 2.7

原创 hadoop 2.7.5, hadoop 2.8.2, hadoop 2.9.0 proto 文件大小对比

Hive 性能测试程序，https://github.com/hortonworks/hive-testbench 编译

Tez 的设计者在 Hadoop 大会上的分享

Transactional Operations in Apache Hive:Present and Future

The Log-Structured Merge-Tree (LSM-Tree).pdf

Distributed cycle detection in large-scale sparse graphs.pdf

空空如也

原创 hadoop 2.7.5 YARN ResouceManager Submit Application Process

原创 NodeManager Health Check 经验总结

转载 Container killed by the ApplicationMaster, Exit code is 143

转载 读懂 diff 命令

原创 Upgrade YARN from Hadooop 2.6 to Hadoop 2.7

原创 hadoop 2.7.5, hadoop 2.8.2, hadoop 2.9.0 proto 文件大小对比

Hive 性能测试程序，https://github.com/hortonworks/hive-testbench 编译

Tez 的设计者在 Hadoop 大会上的分享

Transactional Operations in Apache Hive: Present and Future

The Log-Structured Merge-Tree (LSM-Tree).pdf

Distributed cycle detection in large-scale sparse graphs.pdf

空空如也

转载读懂 diff 命令

Transactional Operations in Apache Hive:Present and Future