Hadoop 常见问题 常见错误

1. 启用Heap Dump

In each script, there is an export for *_OPTS, which contains the options passed to the JVM. For example, in the hadoop-env.sh script, the line that begins with export HADOOP_NAMENODE_OPTS= contains the options for the NameNode service.

Map and reduce processes are slightly different, as these are a child process of the MapReduce service. Each map or reduce process runs in a child container, and there are two entries that contain the JVM options for these. Both contained in mapred-site.xml:

• mapreduce.admin.map.child.java.opts
• mapreduce.admin.reduce.child.java.opts

-XX:+HeapDumpOnOutOfMemoryError enables heap dumps when an OutOfMemoryError occurs; The default location for the dump file is the current working directory. You can control where the file is stored using the following option -XX:HeapDumpPath=/path.

E.g. Hadoop-env.sh :  export Hadoop_NAMENODE_OPTS="-XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/tmp...". When enabling heap dumps for the map or reduce child process, you will instead look for the fields labled mapreduce.admin.map.child.java.opts and mapreduce.admin.reduce.child.java.opts.

转载于:https://www.cnblogs.com/chinesedream/p/4983798.html