HBase

Apache HBase Configuration

HBase Configuration File Descriptions

backup-masters

Not present by default. A plain-text file which lists hosts on which the Master should start a backup Master process, one host per line.

hadoop-metrics2-hbase.properties

Used to connect HBase Hadoop’s Metrics2 framework. See the Hadoop Wiki entry for more information on Metrics2. Contains only commented-out examples by default.

hbase-env.cmd and hbase-env.sh

Script for Windows and Linux / Unix environments to set up the working environment for HBase, including the location of Java, Java options, and other environment variables. The file contains many commented-out examples to provide guidance.

hbase-policy.xml

The default policy configuration file used by RPC servers to make authorization decisions on client requests. Only used if HBase security is enabled.

hbase-site.xml

The main HBase configuration file. This file specifies configuration options which override HBase’s default configuration. You can view (but do not edit) the default configuration file at docs/hbase-default.xml. You can also view the entire effective configuration for your cluster (defaults and overrides) in the HBase Configuration tab of the HBase Web UI.

log4j.properties

Configuration file for HBase logging via log4j

regionservers

A plain-text file containing a list of hosts which should run a RegionServer in your HBase cluster. By default this file contains the single entry localhost. It should contain a list of hostnames or IP addresses, one per line, and should only contain localhost if each node in your cluster will run a RegionServer on its localhost interface

dfs.datanode.max.transfer.threads

setting the dfs.datanode.max.transfer.threads value to at least the following:

<property>

  <name>dfs.datanode.max.transfer.threads</name>

  <value>4096</value>

</property>

Namespace

This abstraction lays the groundwork for upcoming multi-tenancy related features:

   • Quota Management (HBASE-8410) - Restrict the amount of resources (i.e. regions, tables) a namespace can consume.

• Namespace Security Administration (HBASE-9206) - Provide another level of security administration for tenants.
• Region server groups (HBASE-6721) - A namespace/table can be pinned onto a subset of RegionServers thus guaranteeing a coarse level of isolation

Apache HBase Performance Tuning

Memory   Swapping 

Network

GC

Avoiding Full GCs with MemStore-Local Allocation Buffers

Todd Lipcon describes two cases of stop-the-world garbage collections common in HBase, especially during loading; CMS failure modes and old generation heap fragmentation brought.

-XX:CMSInitiatingOccupancyFraction=60/70

hbase.hregion.memstore.mslab.enabled true

HDFS Configurations

 

 dfs.block.size, dfs.blocksize

   dfs.umaskmode, fs.permissions.umask-mode

   dfs.datanode.handler.count

   dfs.datanode.max.xcievers,dfs.datanode.max.transfer.threads

   dfs.thrift.threads.max

   dfs.thrift.threads.min

   dfs.namenode.handler.count

   dfs.namenode.service.handler.count

 hbase.regionserver.handler.count

 hbase.ipc.server.max.callqueue.size.

  hfile.block.cache.size

zookeeper.session.timeout

dfs.datanode.failed.volumes.tolerated

Hbase问题：

hbase regionserver  region   hfile的数量关系以及决定因素