imapa 内存使用设置

Impala Guide

• Cloudera.com
• Training
• Support
• Documentation
• Dev Center

• |Contact Us
• Downloads

This is the documentation for Impala 2.1.x, included as part of CDH 5.3.x.
Latest Version of this Page | All Cloudera Docs

Modifying Impala Startup Options

The configuration options for the Impala-related daemons let you choose which hosts and ports to use for the services that run on a single host, specify directories for logging, control resource usage and security, and specify other aspects of the Impala software.

• Configuring Impala Startup Options through Cloudera Manager
• Configuring Impala Startup Options through the Command Line
• Checking the Values of Impala Configuration Options
• Startup Options for impalad Daemon
• Startup Options for statestored Daemon
• Startup Options for catalogd Daemon

Configuring Impala Startup Options through Cloudera Manager

If you manage your cluster through Cloudera Manager, configure the settings for all the Impala-related daemons by navigating to this page: Services > Impala > Configuration > View and Edit. See the Cloudera Manager documentation for instructions about how to configure Impala through Cloudera Manager.

If the Cloudera Manager interface does not yet have a form field for a newly added option, or if you need to use special options for debugging and troubleshooting, the Advanced option page for each daemon includes one or more fields where you can enter option names directly. In Cloudera Manager 4, these fields are labelled Safety Valve; in Cloudera Manager 5, they are called Advanced Configuration Snippet. There is also a free-form field for query options, on the top-level Impala Daemon options page.

Configuring Impala Startup Options through the Command Line

When you run Impala in a non-Cloudera Manager environment, the Impala server, statestore, and catalog services start up using values provided in a defaults file, /etc/default/impala.

This file includes information about many resources used by Impala. Most of the defaults included in this file should be effective in most cases. For example, typically you would not change the definition of the CLASSPATH variable, but you would always set the address used by the statestore server. Some of the content you might modify includes:

IMPALA_STATE_STORE_HOST=127.0.0.1
IMPALA_STATE_STORE_PORT=24000
IMPALA_BACKEND_PORT=22000
IMPALA_LOG_DIR=/var/log/impala
IMPALA_CATALOG_SERVICE_HOST=...
IMPALA_STATE_STORE_HOST=...

export IMPALA_STATE_STORE_ARGS=${IMPALA_STATE_STORE_ARGS:- \
    -log_dir=${IMPALA_LOG_DIR} -state_store_port=${IMPALA_STATE_STORE_PORT}}
IMPALA_SERVER_ARGS=" \
-log_dir=${IMPALA_LOG_DIR} \
-catalog_service_host=${IMPALA_CATALOG_SERVICE_HOST} \
-state_store_port=${IMPALA_STATE_STORE_PORT} \
-use_statestore \
-state_store_host=${IMPALA_STATE_STORE_HOST} \
-be_port=${IMPALA_BACKEND_PORT}"
export ENABLE_CORE_DUMPS=${ENABLE_COREDUMPS:-false}

To use alternate values, edit the defaults file, then restart all the Impala-related services so that the changes take effect. Restart the Impala server using the following commands:

$ sudo service impala-server restart
Stopping Impala Server:                                    [  OK  ]
Starting Impala Server:                                    [  OK  ]

Restart the Impala statestore using the following commands:

$ sudo service impala-state-store restart
Stopping Impala State Store Server:                        [  OK  ]
Starting Impala State Store Server:                        [  OK  ]

Restart the Impala catalog service using the following commands:

$ sudo service impala-catalog restart
Stopping Impala Catalog Server:                            [  OK  ]
Starting Impala Catalog Server:                            [  OK  ]

Some common settings to change include:

• Statestore address. Cloudera recommends the statestore be on a separate host not running the impalad daemon. In that recommended configuration, the impalad daemon cannot refer to the statestore server using the loopback address. If the statestore is hosted on a machine with an IP address of 192.168.0.27, change:

  IMPALA_STATE_STORE_HOST=127.0.0.1

  to:

  IMPALA_STATE_STORE_HOST=192.168.0.27

• Catalog server address (including both the hostname and the port number). Update the value of the IMPALA_CATALOG_SERVICE_HOST variable. Cloudera recommends the catalog server be on the same host as the statestore. In that recommended configuration, the impalad daemon cannot refer to the catalog server using the loopback address. If the catalog service is hosted on a machine with an IP address of 192.168.0.27, add the following line:

  IMPALA_CATALOG_SERVICE_HOST=192.168.0.27:26000

  The /etc/default/impala defaults file currently does not define an IMPALA_CATALOG_ARGS environment variable, but if you add one it will be recognized by the service startup/shutdown script. Add a definition for this variable to /etc/default/impala and add the option -catalog_service_host=hostname. If the port is different than the default 26000, also add the option -catalog_service_port=port.

• Memory limits. You can limit the amount of memory available to Impala. For example, to allow Impala to use no more than 70% of system memory, change:

  export IMPALA_SERVER_ARGS=${IMPALA_SERVER_ARGS:- \
      -log_dir=${IMPALA_LOG_DIR} \
      -state_store_port=${IMPALA_STATE_STORE_PORT} \
      -use_statestore -state_store_host=${IMPALA_STATE_STORE_HOST} \
      -be_port=${IMPALA_BACKEND_PORT}}

  to:

  export IMPALA_SERVER_ARGS=${IMPALA_SERVER_ARGS:- \
      -log_dir=${IMPALA_LOG_DIR} -state_store_port=${IMPALA_STATE_STORE_PORT} \
      -use_statestore -state_store_host=${IMPALA_STATE_STORE_HOST} \
      -be_port=${IMPALA_BACKEND_PORT} -mem_limit=70%}

  You can specify the memory limit using absolute notation such as 500m or 2G, or as a percentage of physical memory such as 60%.

地址：http://www.cloudera.com/documentation/archive/impala/2-x/2-1-x/topics/impala_config_options.html#config_options_cm_unique_1