Ambari日常运维操作命令收集

Ambari REST API查询相关

• 获取集群配置信息：http://hzadg-mammut-platform1.server.163.org:8080/api/v1/clusters/cluster1?fields=Clusters/desired_configs

• 特定配置信息(cluster-env)：http://hzadg-mammut-platform1.server.163.org:8080/api/v1/clusters/cluster1/configurations?type=cluster-env&tag=version1522220015974073

• 添加一个新的配置(yarn-env)：使用场景也可能是丢失了yarn-env配置

curl -u admin:admin -H 'X-Requested-By:admin' -X PUT 'http://hzadg-mammut-platform1.server.163.org:8080/api/v1/clusters/cluster1' -d '{
  "Clusters": {
    "desired_config": {
      "type": "yarn-env",
      "tag": "version2",
      "properties": {
        "yarn_heapsize": "1024", 
            "yarn_pid_dir_prefix": "/var/run/hadoop-yarn", 
            "yarn_user_nproc_limit": "65536", 
            "yarn_log_dir_prefix": "/var/log/hadoop-yarn", 
            "yarn_user_nofile_limit": "32768", 
            "service_archive_rela_path": "{\"DEFAULT\":\"hadoop/hadoop-2.7.3-1.2.0.tar.gz\", \"METRICS_HADOOP_SINK\":\"ambari-metrics/ambari-metrics-hadoop-sink-2.5.1.0.0.tar.gz\"}", 
            "service_version": "2.7.3", 
            "is_supported_yarn_ranger": "false", 
            "nodemanager_heapsize": "1024", 
            "server_role_directory_map": "{\"YARN_CLIENT\":\"yarn_client\",\"RESOURCEMANAGER\":\"yarn_resourcemanager\",\"NODEMANAGER\":\"yarn_nodemanager\",\"APP_TIMELINE_SERVER\":\"yarn_timelineserver\", \"METRICS_HADOOP_SINK\":\"ambari_metrics_hadoop_sink\"}", 
            "content": "\nexport HADOOP_YARN_HOME={{component_home}}\nexport YARN_LOG_DIR={{yarn_log_dir}}\nexport YARN_PID_DIR={{yarn_pid_dir}}\nexport HADOOP_LIBEXEC_DIR={{component_libexec_dir}}\nexport JAVA_HOME={{java_home}}\nexport JAVA_LIBRARY_PATH=\"${JAVA_LIBRARY_PATH}:{{hadoop_java_io_tmpdir}}\"\n\n# We need to add the EWMA appender for the yarn daemons only;\n# however, YARN_ROOT_LOGGER is shared by the yarn client and the\n# daemons. This is restrict the EWMA appender to daemons only.\nINVOKER=\"${0##*/}\"\nif [ \"$INVOKER\" == \"yarn-daemon.sh\" ]; then\n  export YARN_ROOT_LOGGER=${YARN_ROOT_LOGGER:-INFO,EWMA,RFA}\nfi\n\n# User for YARN daemons\nexport HADOOP_YARN_USER=${HADOOP_YARN_USER:-yarn}\n\n# resolve links - $0 may be a softlink\nexport YARN_CONF_DIR=\"${YARN_CONF_DIR:-$HADOOP_YARN_HOME/conf}\"\n\n# some Java parameters\nexport JAVA_HOME={{java_home}}\nif [ \"$JAVA_HOME\" != \"\" ]; then\n  #echo \"run java in $JAVA_HOME\"\n  JAVA_HOME=$JAVA_HOME\nfi\n\nif [ \"$JAVA_HOME\" = \"\" ]; then\n  echo \"Error: JAVA_HOME is not set.\"\n  exit 1\nfi\n\nJAVA=$JAVA_HOME/bin/java\nJAVA_HEAP_MAX=-Xmx1000m\n\n# For setting YARN specific HEAP sizes please use this\n# Parameter and set appropriately\nYARN_HEAPSIZE={{yarn_heapsize}}\n\n# check envvars which might override default args\nif [ \"$YARN_HEAPSIZE\" != \"\" ]; then\n  JAVA_HEAP_MAX=\"-Xmx\"\"$YARN_HEAPSIZE\"\"m\"\nfi\n\n# Resource Manager specific parameters\n\n# Specify the max Heapsize for the ResourceManager using a numerical value\n# in the scale of MB. For example, to specify an jvm option of -Xmx1000m, set\n# the value to 1000.\n# This value will be overridden by an Xmx setting specified in either YARN_OPTS\n# and/or YARN_RESOURCEMANAGER_OPTS.\n# If not specified, the default value will be picked from either YARN_HEAPMAX\n# or JAVA_HEAP_MAX with YARN_HEAPMAX as the preferred option of the two.\nexport YARN_RESOURCEMANAGER_HEAPSIZE={{resourcemanager_heapsize}}\n\n# Specify the JVM options to be used when starting the ResourceManager.\n# These options will be appended to the options specified as YARN_OPTS\n# and therefore may override any similar flags set in YARN_OPTS\n#export YARN_RESOURCEMANAGER_OPTS=\n\n# Node Manager specific parameters\n\n# Specify the max Heapsize for the NodeManager using a numerical value\n# in the scale of MB. For example, to specify an jvm option of -Xmx1000m, set\n# the value to 1000.\n# This value will be overridden by an Xmx setting specified in either YARN_OPTS\n# and/or YARN_NODEMANAGER_OPTS.\n# If not specified, the default value will be picked from either YARN_HEAPMAX\n# or JAVA_HEAP_MAX with YARN_HEAPMAX as the preferred option of the two.\nexport YARN_NODEMANAGER_HEAPSIZE={{nodemanager_heapsize}}\n\n# Specify the max Heapsize for the timeline server using a numerical value\n# in the scale of MB. For example, to specify an jvm option of -Xmx1000m, set\n# the value to 1024.\n# This value will be overridden by an Xmx setting specified in either YARN_OPTS\n# and/or YARN_TIMELINESERVER_OPTS.\n# If not specified, the default value will be picked from either YARN_HEAPMAX\n# or JAVA_HEAP_MAX with YARN_HEAPMAX as the preferred option of the two.\nexport YARN_TIMELINESERVER_HEAPSIZE={{apptimelineserver_heapsize}}\n\n# Specify the JVM options to be used when starting the NodeManager.\n# These options will be appended to the options specified as YARN_OPTS\n# and therefore may override any similar flags set in YARN_OPTS\n#export YARN_NODEMANAGER_OPTS=\n\n# so that filenames w/ spaces are handled correctly in loops below\nIFS=\n\n\n# default log directory and file\nif [ \"$YARN_LOG_DIR\" = \"\" ]; then\n  YARN_LOG_DIR=\"$HADOOP_YARN_HOME/logs\"\nfi\nif [ \"$YARN_LOGFILE\" = \"\" ]; then\n  YARN_LOGFILE='yarn.log'\nfi\n\n# default policy file for service-level authorization\nif [ \"$YARN_POLICYFILE\" = \"\" ]; then\n  YARN_POLICYFILE=\"hadoop-policy.xml\"\nfi\n\n# restore ordinary behaviour\nunset IFS\n\n\nYARN_OPTS=\"$YARN_OPTS -Dhadoop.log.dir=$YARN_LOG_DIR\"\nYARN_OPTS=\"$YARN_OPTS -Dyarn.log.dir=$YARN_LOG_DIR\"\nYARN_OPTS=\"$YARN_OPTS -Dhadoop.log.file=$YARN_LOGFILE\"\nYARN_OPTS=\"$YARN_OPTS -Dyarn.log.file=$YARN_LOGFILE\"\nYARN_OPTS=\"$YARN_OPTS -Dyarn.home.dir=$YARN_COMMON_HOME\"\nYARN_OPTS=\"$YARN_OPTS -Dyarn.id.str=$YARN_IDENT_STRING\"\nYARN_OPTS=\"$YARN_OPTS -Dhadoop.root.logger=${YARN_ROOT_LOGGER:-INFO,console}\"\nYARN_OPTS=\"$YARN_OPTS -Dyarn.root.logger=${YARN_ROOT_LOGGER:-INFO,console}\"\nexport YARN_NODEMANAGER_OPTS=\"$YARN_NODEMANAGER_OPTS -Dnm.audit.logger=INFO,NMAUDIT\"\nexport YARN_RESOURCEMANAGER_OPTS=\"$YARN_RESOURCEMANAGER_OPTS -Drm.audit.logger=INFO,RMAUDIT\"\nif [ \"x$JAVA_LIBRARY_PATH\" != \"x\" ]; then\n  YARN_OPTS=\"$YARN_OPTS -Djava.library.path=$JAVA_LIBRARY_PATH\"\nfi\nYARN_OPTS=\"$YARN_OPTS -Dyarn.policy.file=$YARN_POLICYFILE\"\nYARN_OPTS=\"$YARN_OPTS -Djava.io.tmpdir={{hadoop_java_io_tmpdir}}\"", 
            "service_archive_root_dir": "{\"DEFAULT\":\"hadoop-2.7.3\", \"METRICS_HADOOP_SINK\":\"ambari-metrics-hadoop-sink-2.5.1.0.0\"}", 
            "service_check.queue.name": "default", 
            "min_user_id": "1000", 
            "yarn_cgroups_enabled": "false", 
            "yarn_user": "yarn", 
            "resourcemanager_heapsize": "1024", 
            "apptimelineserver_heapsize": "1024", 
            "yarn_data_dir": "/usr/ndp/data/yarn"
      }
    }
  }
}'

添加完配置后，可以将其设置为线上使用配置版本

curl -u admin:admin -H "X-Requested-By: ambari" -X PUT -d '[{"Clusters":{
  "desired_config":[{
    "type":"yarn-env",
    "tag":"version2",
    "service_config_version_note":"New config version"}]}}]' "http://hzadg-mammut-platform1.server.163.org:8080/api/v1/clusters/cluster1"

基于API升级集群操作：

curl -H "X-Requested-By: ambari" -X POST -u admin:admin  http://hzadg-mammut-platform1.server.163.org:8080/api/v1/clusters/cluster1/upgrades -d @upgrade.json

> {"Upgrade":{"repository_version":"3.3.1.0.1","upgrade_type":"NON_ROLLING", "direction":"UPGRADE", "skip_prerequisite_checks":"true", "from_version":"3.2.1", "to_version":"3.3.1", "cluster_name":"cluster1"}}

AmbariServer操作相关

备份AmbariServer配置:

cd /var/lib/ambari-server/resources/scripts && python upgradeHelper.py –user=admin –password=admin –clustername=cluster2 –hostname=hzadg-mammut-platform5.server.163.org backup-configs

开启重启时包替换功能：

/var/lib/ambari-server/resources/scripts/configs.py –user admin –password admin –action set –host hzadg-mammut-platform1.server.163.org –cluster cluster1 –config-type cluster-env –key reinstall_component_package –value true

Ambari数据库相关操作

repo版本表：

select * from repo_version;

当前集群版本表：

update cluster_version set state=’CURRENT’ where id=1;