Config inconsistency exists: unknown configType=kafka-broker

ambari卸载kafka 出现上述错误

查看日志tail -100 /var/log/ambari-server/ambari-server.log 

ERROR [ambari-client-thread-29533] HostImpl:1084 - Config inconsistency exists: unknown configType=kafka-broker

问题梳理

• 错误： 已经在ambari-server中将hbase服务删除掉，出现该错误怀疑上次删除完关闭ambari-server时，没有来得及更新数据库导致的，所以需要手动变更数据库才能解决；

源码解析

定位到异常错误的地方，在HostImpl中，看到如下的代码：

@Override
  public Map<String, HostConfig> getDesiredHostConfigs(Cluster cluster,
      Map<String, DesiredConfig> clusterDesiredConfigs) throws AmbariException {
    Map<String, HostConfig> hostConfigMap = new HashMap<String, HostConfig>();

    if( null == cluster ){
      clusterDesiredConfigs = new HashMap<String, DesiredConfig>();
    }

    // per method contract, fetch if not supplied
    if (null == clusterDesiredConfigs) {
      clusterDesiredConfigs = cluster.getDesiredConfigs();
    }

    if (clusterDesiredConfigs != null) {
      for (Map.Entry<String, DesiredConfig> desiredConfigEntry
          : clusterDesiredConfigs.entrySet()) {
        HostConfig hostConfig = new HostConfig();
        hostConfig.setDefaultVersionTag(desiredConfigEntry.getValue().getTag());
        hostConfigMap.put(desiredConfigEntry.getKey(), hostConfig);
      }
    }
    
    // 怀疑是这里`cluster.getConfigGroupsByHostname(getHostName())`引入了没有删除的数据
    Map<Long, ConfigGroup> configGroups = (cluster == null) ? new HashMap<Long, ConfigGroup>() : cluster.getConfigGroupsByHostname(getHostName());

    if (configGroups != null && !configGroups.isEmpty()) {
      for (ConfigGroup configGroup : configGroups.values()) {
        for (Map.Entry<String, Config> configEntry : configGroup
            .getConfigurations().entrySet()) {

          String configType = configEntry.getKey();
          // HostConfig config holds configType -> versionTag, per config group
          HostConfig hostConfig = hostConfigMap.get(configType);
          if (hostConfig == null) {
            hostConfig = new HostConfig();
            hostConfigMap.put(configType, hostConfig);
            if (cluster != null) {
              Config conf = cluster.getDesiredConfigByType(configType);
              if(conf == null) {
              //报错出现在这个地方，说明该config已经被清理，但是循环还是走到了这里！
                LOG.error("Config inconsistency exists:"+
                    " unknown configType="+configType);
              } else {
                hostConfig.setDefaultVersionTag(conf.getTag());
              }
            }
          }
          Config config = configEntry.getValue();
          hostConfig.getConfigGroupOverrides().put(configGroup.getId(),
              config.getTag());
        }
      }
    }
    return hostConfigMap;
  }

在ambari中，同数据库的连接是基于jpa完成的，找到ClusterEntity下的configGroupEntities定义，可以看出ClusterEntity同ConfigGroupEntity之间的关系，

  @OneToMany(mappedBy = "clusterEntity", cascade = CascadeType.ALL)
  private Collection<ConfigGroupEntity> configGroupEntities;

在ConfigGroupEnity中，可以发现其依赖一下两个表：

  @OneToMany(mappedBy = "configGroupEntity", cascade = CascadeType.ALL)
  private Collection<ConfigGroupHostMappingEntity> configGroupHostMappingEntities;

  @OneToMany(mappedBy = "configGroupEntity", cascade = CascadeType.ALL)
  private Collection<ConfigGroupConfigMappingEntity> configGroupConfigMappingEntities;

在数据库中找到如下对应关系，定位到问题：

mysql> use ambari
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A

Database changed
mysql> select * from configgroup;
+----------+------------+------------+-------+----------------------------------------------------+------------------+--------------+
| group_id | cluster_id | group_name | tag   | description                                        | create_timestamp | service_name |
+----------+------------+------------+-------+----------------------------------------------------+------------------+--------------+
|        2 |          2 | test01     | KAFKA | New configuration group created on Wed Jan 08 2020 |    1578470900704 | KAFKA        |
|        3 |          2 | test02     | KAFKA | New configuration group created on Wed Jan 08 2020 |    1578470900782 | KAFKA        |
|        4 |          2 | test05     | KAFKA | New configuration group created on Wed Jan 08 2020 |    1578470900843 | KAFKA        |
+----------+------------+------------+-------+----------------------------------------------------+------------------+--------------+
3 rows in set (0.01 sec)

mysql> delete  from configgroup;
ERROR 1451 (23000): Cannot delete or update a parent row: a foreign key constraint fails (`ambari`.`configgrouphostmapping`, CONSTRAINT `FK_cghm_cgid` FOREIGN KEY (`config_group_id`) REFERENCES `configgroup` (`group_id`))
mysql> SET FOREIGN_KEY_CHECKS=0;
Query OK, 0 rows affected (0.00 sec)

mysql> delete  from configgroup;
Query OK, 3 rows affected (0.00 sec)

mysql> SET FOREIGN_KEY_CHECKS=1;
Query OK, 0 rows affected (0.00 sec)

mysql> delete  from configgrouphostmapping;
Query OK, 3 rows affected (0.07 sec)

重启ambari-server restart

再次安装就可以了

ref：https://www.jianshu.com/p/40b534b0843a