Ambari REST API查询相关
- 获取集群配置信息:http://hzadg-mammut-platform1.server.163.org:8080/api/v1/clusters/cluster1?fields=Clusters/desired_configs
特定配置信息(cluster-env):http://hzadg-mammut-platform1.server.163.org:8080/api/v1/clusters/cluster1/configurations?type=cluster-env&tag=version1522220015974073
添加一个新的配置(yarn-env):使用场景也可能是丢失了yarn-env配置
curl -u admin:admin -H 'X-Requested-By:admin' -X PUT 'http://hzadg-mammut-platform1.server.163.org:8080/api/v1/clusters/cluster1' -d '{
"Clusters": {
"desired_config": {
"type": "yarn-env",
"tag": "version2",
"properties": {
"yarn_heapsize": "1024",
"yarn_pid_dir_prefix": "/var/run/hadoop-yarn",
"yarn_user_nproc_limit": "65536",
"yarn_log_dir_prefix": "/var/log/hadoop-yarn",
"yarn_user_nofile_limit": "32768",
"service_archive_rela_path": "{\"DEFAULT\":\"hadoop/hadoop-2.7.3-1.2.0.tar.gz\", \"METRICS_HADOOP_SINK\":\"ambari-metrics/ambari-metrics-hadoop-sink-2.5.1.0.0.tar.gz\"}",
"service_version": "2.7.3",
"is_supported_yarn_ranger": "false",
"nodemanager_heapsize": "1024",
"server_role_directory_map": "{\"YARN_CLIENT\":\"yarn_client\",\"RESOURCEMANAGER\":\"yarn_resourcemanager\",\"NODEMANAGER\":\"yarn_nodemanager\",\"APP_TIMELINE_SERVER\":\"yarn_timelineserver\", \"METRICS_HADOOP_SINK\":\"ambari_metrics_hadoop_sink\"}",
"content": "\nexport HADOOP_YARN_HOME={{component_home}}\nexport YARN_LOG_DIR={{yarn_log_dir}}\nexport YARN_PID_DIR={{yarn_pid_dir}}\nexport HADOOP_LIBEXEC_DIR={{component_libexec_dir}}\nexport JAVA_HOME={{java_home}}\nexport JAVA_LIBRARY_PATH=\"${JAVA_LIBRARY_PATH}:{{hadoop_java_io_tmpdir}}\"\n\n# We need to add the EWMA appender for the yarn daemons only;\n# however, YARN_ROOT_LOGGER is shared by the yarn client and the\n# daemons. This is restrict the EWMA appender to daemons only.\nINVOKER=\"${0##*/}\"\nif [ \"$INVOKER\" == \"yarn-daemon.sh\" ]; then\n export YARN_ROOT_LOGGER=${YARN_ROOT_LOGGER:-INFO,EWMA,RFA}\nfi\n\n# User for YARN daemons\nexport HADOOP_YARN_USER=${HADOOP_YARN_USER:-yarn}\n\n# resolve links - $0 may be a softlink\nexport YARN_CONF_DIR=\"${YARN_CONF_DIR:-$HADOOP_YARN_HOME/conf}\"\n\n# some Java parameters\nexport JAVA_HOME={{java_home}}\nif [ \"$JAVA_HOME\" != \"\" ]; then\n #echo \"run java in $JAVA_HOME\"\n JAVA_HOME=$JAVA_HOME\nfi\n\nif [ \"$JAVA_HOME\" = \"\" ]; then\n echo \"Error: JAVA_HOME is not set.\"\n exit 1\nfi\n\nJAVA=$JAVA_HOME/bin/java\nJAVA_HEAP_MAX=-Xmx1000m\n\n# For setting YARN specific HEAP sizes please use this\n# Parameter and set appropriately\nYARN_HEAPSIZE={{yarn_heapsize}}\n\n# check envvars which might override default args\nif [ \"$YARN_HEAPSIZE\" != \"\" ]; then\n JAVA_HEAP_MAX=\"-Xmx\"\"$YARN_HEAPSIZE\"\"m\"\nfi\n\n# Resource Manager specific parameters\n\n# Specify the max Heapsize for the ResourceManager using a numerical value\n# in the scale of MB. For example, to specify an jvm option of -Xmx1000m, set\n# the value to 1000.\n# This value will be overridden by an Xmx setting specified in either YARN_OPTS\n# and/or YARN_RESOURCEMANAGER_OPTS.\n# If not specified, the default value will be picked from either YARN_HEAPMAX\n# or JAVA_HEAP_MAX with YARN_HEAPMAX as the preferred option of the two.\nexport YARN_RESOURCEMANAGER_HEAPSIZE={{resourcemanager_heapsize}}\n\n# Specify the JVM options to be used when starting the ResourceManager.\n# These options will be appended to the options specified as YARN_OPTS\n# and therefore may override any similar flags set in YARN_OPTS\n#export YARN_RESOURCEMANAGER_OPTS=\n\n# Node Manager specific parameters\n\n# Specify the max Heapsize for the NodeManager using a numerical value\n# in the scale of MB. For example, to specify an jvm option of -Xmx1000m, set\n# the value to 1000.\n# This value will be overridden by an Xmx setting specified in either YARN_OPTS\n# and/or YARN_NODEMANAGER_OPTS.\n# If not specified, the default value will be picked from either YARN_HEAPMAX\n# or JAVA_HEAP_MAX with YARN_HEAPMAX as the preferred option of the two.\nexport YARN_NODEMANAGER_HEAPSIZE={{nodemanager_heapsize}}\n\n# Specify the max Heapsize for the timeline server using a numerical value\n# in the scale of MB. For example, to specify an jvm option of -Xmx1000m, set\n# the value to 1024.\n# This value will be overridden by an Xmx setting specified in either YARN_OPTS\n# and/or YARN_TIMELINESERVER_OPTS.\n# If not specified, the default value will be picked from either YARN_HEAPMAX\n# or JAVA_HEAP_MAX with YARN_HEAPMAX as the preferred option of the two.\nexport YARN_TIMELINESERVER_HEAPSIZE={{apptimelineserver_heapsize}}\n\n# Specify the JVM options to be used when starting the NodeManager.\n# These options will be appended to the options specified as YARN_OPTS\n# and therefore may override any similar flags set in YARN_OPTS\n#export YARN_NODEMANAGER_OPTS=\n\n# so that filenames w/ spaces are handled correctly in loops below\nIFS=\n\n\n# default log directory and file\nif [ \"$YARN_LOG_DIR\" = \"\" ]; then\n YARN_LOG_DIR=\"$HADOOP_YARN_HOME/logs\"\nfi\nif [ \"$YARN_LOGFILE\" = \"\" ]; then\n YARN_LOGFILE='yarn.log'\nfi\n\n# default policy file for service-level authorization\nif [ \"$YARN_POLICYFILE\" = \"\" ]; then\n YARN_POLICYFILE=\"hadoop-policy.xml\"\nfi\n\n# restore ordinary behaviour\nunset IFS\n\n\nYARN_OPTS=\"$YARN_OPTS -Dhadoop.log.dir=$YARN_LOG_DIR\"\nYARN_OPTS=\"$YARN_OPTS -Dyarn.log.dir=$YARN_LOG_DIR\"\nYARN_OPTS=\"$YARN_OPTS -Dhadoop.log.file=$YARN_LOGFILE\"\nYARN_OPTS=\"$YARN_OPTS -Dyarn.log.file=$YARN_LOGFILE\"\nYARN_OPTS=\"$YARN_OPTS -Dyarn.home.dir=$YARN_COMMON_HOME\"\nYARN_OPTS=\"$YARN_OPTS -Dyarn.id.str=$YARN_IDENT_STRING\"\nYARN_OPTS=\"$YARN_OPTS -Dhadoop.root.logger=${YARN_ROOT_LOGGER:-INFO,console}\"\nYARN_OPTS=\"$YARN_OPTS -Dyarn.root.logger=${YARN_ROOT_LOGGER:-INFO,console}\"\nexport YARN_NODEMANAGER_OPTS=\"$YARN_NODEMANAGER_OPTS -Dnm.audit.logger=INFO,NMAUDIT\"\nexport YARN_RESOURCEMANAGER_OPTS=\"$YARN_RESOURCEMANAGER_OPTS -Drm.audit.logger=INFO,RMAUDIT\"\nif [ \"x$JAVA_LIBRARY_PATH\" != \"x\" ]; then\n YARN_OPTS=\"$YARN_OPTS -Djava.library.path=$JAVA_LIBRARY_PATH\"\nfi\nYARN_OPTS=\"$YARN_OPTS -Dyarn.policy.file=$YARN_POLICYFILE\"\nYARN_OPTS=\"$YARN_OPTS -Djava.io.tmpdir={{hadoop_java_io_tmpdir}}\"",
"service_archive_root_dir": "{\"DEFAULT\":\"hadoop-2.7.3\", \"METRICS_HADOOP_SINK\":\"ambari-metrics-hadoop-sink-2.5.1.0.0\"}",
"service_check.queue.name": "default",
"min_user_id": "1000",
"yarn_cgroups_enabled": "false",
"yarn_user": "yarn",
"resourcemanager_heapsize": "1024",
"apptimelineserver_heapsize": "1024",
"yarn_data_dir": "/usr/ndp/data/yarn"
}
}
}
}'
添加完配置后,可以将其设置为线上使用配置版本
curl -u admin:admin -H "X-Requested-By: ambari" -X PUT -d '[{"Clusters":{
"desired_config":[{
"type":"yarn-env",
"tag":"version2",
"service_config_version_note":"New config version"}]}}]' "http://hzadg-mammut-platform1.server.163.org:8080/api/v1/clusters/cluster1"
基于API升级集群操作:
curl -H "X-Requested-By: ambari" -X POST -u admin:admin http://hzadg-mammut-platform1.server.163.org:8080/api/v1/clusters/cluster1/upgrades -d @upgrade.json
> {"Upgrade":{"repository_version":"3.3.1.0.1","upgrade_type":"NON_ROLLING", "direction":"UPGRADE", "skip_prerequisite_checks":"true", "from_version":"3.2.1", "to_version":"3.3.1", "cluster_name":"cluster1"}}
AmbariServer操作相关
备份AmbariServer配置:
cd /var/lib/ambari-server/resources/scripts && python upgradeHelper.py –user=admin –password=admin –clustername=cluster2 –hostname=hzadg-mammut-platform5.server.163.org backup-configs
开启重启时包替换功能:
/var/lib/ambari-server/resources/scripts/configs.py –user admin –password admin –action set –host hzadg-mammut-platform1.server.163.org –cluster cluster1 –config-type cluster-env –key reinstall_component_package –value true
Ambari数据库相关操作
repo版本表:
select * from repo_version;
当前集群版本表:
update cluster_version set state=’CURRENT’ where id=1;