ES集成到ambari中出现的常见问题归总

1.elasticesearch用户名组的问题

KeyError: u'elasticsearch'
Error: Error: Unable to run the custom hook script ['/usr/bin/python', '/var/lib/ambari-agent/cache/stack-hooks/before-ANY/scripts/hook.py', 'ANY', '/var/lib/ambari-agent/data/command-102.json', '/var/lib/ambari-agent/cache/stack-hooks/before-ANY', '/var/lib/ambari-agent/data/structured-out-102.json', 'INFO', '/var/lib/ambari-agent/tmp', 'PROTOCOL_TLSv1_2', '']

解决办法：

        需要修改ambari-server资源中的一个配置参数，将gnore_groupsusers_create由false改为true，之后手动创建用户和用户组。

python /var/lib/ambari-server/resources/scripts/configs.py -u admin -p admin -n [CLUSTER_NAME] -l [CLUSTER_FQDN] -t 8080 -a set -c cluster-env -k ignore_groupsusers_create -v true

CLUSTER_NAME：ambari安装的项目名称；

CLUSTER_FQDN：服务器名称

记得结合实际情况修改

#查看用户管理配置

python /var/lib/ambari-server/resources/scripts/configs.py -u admin -p admin -n bigdata -l master -t 8001 -a get -c cluster-env |grep -i ignore_groupsusers_create

"ignore_groupsusers_create": "false",

#修改用户管理配置

python /var/lib/ambari-server/resources/scripts/configs.py -u admin -p admin -n bigdata -l master -t 8001 -a set -c cluster-env -k ignore_groupsusers_create -v true

2.创建ES用户并关联到组

cat /etc/group |grep -i elasticsearch  
cat /etc/group |grep -i kibana
cat /etc/passwd |grep -i elasticsearch
cat /etc/passwd |grep -i kibana 

先查看组和用户是否存在，一般出问题是因为组存在，但用户不存在；

useradd -g elasticsearch elasticsearch

useradd -g kibana kibana

添加用户即可；

3.elasticsearch无法导入format_hdp_stack_version

ImportError: cannot import name format_hdp_stack_version

第一步：

#cd /var/lib/ambari-server/resources/stacks/HDP/3.0/services/ELASTICSEARCH/package/scripts
#vim params.py
#rm -rf /var/lib/ambari-agent/cache/*

第二部：kibana无法导入format_hdp_stack_version

#cd /var/lib/ambari-server/resources/stacks/HDP/3.0/services/KIBANA/package/scripts
#vim params.py
#ambari-server restart

两步修改完成后，重启ambari-server 

4.hostname was not found in configuration dictionary

"/var/lib/ambari-agent/cache/stacks/HDP/3.0/services/ELASTICSEARCH/package/templates/elasticsearch.master.yml.j2", line 93, in top-level template code
action.destructive_requires_name: {{action_destructive_requires_name}}
File "/usr/lib/ambari-agent/lib/resource_management/libraries/script/config_dictionary.py", line 73, in __getattr__
raise Fail("Configuration parameter '" + self.name + "' was not found in configurations dictionary!")
resource_management.core.exceptions.Fail: Configuration parameter 'hostname' was not found in configurations dictionary!

 解决办法：

#cd /var/lib/ambari-server/resources/stacks/HDP/3.0/services/ELASTICSEARCH/configuration

 编辑elasticsearch-config.xml，发现其中的discovery.zen.ping.unicast.hosts属性的值是空的。

我们在安装的时候是填了这个参数的，将这个参数的值给补上。

<name>discovery.zen.ping.unicast.hosts</name>
<value>master</value>

cd

/var/lib/ambari-server/resources/stacks/HDP/3.0/services/ELASTICSEARCH/package/scripts
编辑params.py，发现其中有两处hostname=config[‘hostname’]的写法。

将两处都修改为：

hostname = config['configurations']['elasticsearch-config']['discovery.zen.ping.unicast.hosts'] 

清除所有elasticsearch slave节点的ambari-agent缓存。
server节点重启ambari-server restart
slave节点重启ambari-agent restart
重启elasticsearch服务，发现服务可以启动了。

5 master和slave启动后又停止

 

查看日志会发现提示这两个错误。

错误提示：

max file descriptors [4096] for elasticsearch process is too low, increase to at least [65536]
解决办法：
每个进程最大同时打开文件数太小，可通过下面2个命令查看当前数量

ulimit -Hn
ulimit -Sn

修改/etc/security/limits.conf文件，增加配置(所有master和slave节点)。

错误提示：

max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]
解决办法：
修改/etc/sysctl.conf文件，增加配置(所有master和slave节点)。

vm.max_map_count=262144

然后执行命令sysctl -p生效

sysctl -p