kylin创建cube空指针异常的一次排查记录

项目场景：

• 测试集群中需要使用restfulApi创建多维数据立方体（cube），出现空指针异常

问题描述：

• 堆栈信息

  2021-07-01 15:39:14,760 INFO  [http-nio-7070-exec-12] model.DataModelManager:121 : Update models in project: police_test2
  2021-07-01 15:39:14,763 INFO  [http-nio-7070-exec-12] service.CacheService:123 : cleaning cache for project police_test2 (currently remove all entries)
  2021-07-01 15:39:14,949 WARN  [http-nio-7070-exec-11] model.DataModelDesc:325 : Column ASDF not found in its's model star_05 , maybe it's a tiretree global domain dict. 
  2021-07-01 15:39:14,950 WARN  [http-nio-7070-exec-11] cube.CubeDescManager:195 : Broken cube desc CubeDesc [name=cube_0421]
  java.lang.NullPointerException
  	at org.apache.kylin.cube.model.AggregationGroup.normalizeColumnNames(AggregationGroup.java:120)
  	at org.apache.kylin.cube.model.AggregationGroup.normalizeColumnNames(AggregationGroup.java:94)
  	at org.apache.kylin.cube.model.AggregationGroup.init(AggregationGroup.java:81)
  	at org.apache.kylin.cube.model.CubeDesc.init(CubeDesc.java:711)
  	at org.apache.kylin.cube.CubeDescManager.createCubeDesc(CubeDescManager.java:193)
  	at org.apache.kylin.rest.service.CubeService.createCubeAndDesc(CubeService.java:234)
  	at org.apache.kylin.rest.service.CubeService$$FastClassBySpringCGLIB$$17a07c0e.invoke(<generated>)
  

原因分析：

• 分析发送的json对象某些字段未送或者为空，排查json参数
  官网json实例

检查后并无空字段或者未送参数

解决方案：

追踪源码，确认异常产生原因

org.apache.kylin.cube.model.AggregationGroup.normalizeColumnNames(AggregationGroup.java:120

• AggregationGroup类，追踪120行代码
  github源码地址

  private void normalizeColumnNames(String[] names) {
          if (names == null)
              return;
  
          for (int i = 0; i < names.length; i++) {
              TblColRef col = cubeDesc.getModel().findColumn(names[i]);
  120         names[i] = col.getIdentity();
          }
  
          // check no dup
          Set<String> set = new HashSet<>(Arrays.asList(names));
          if (set.size() < names.length)
              throw new IllegalStateException(
                      "Columns in aggrgroup must not contain duplication: " + Arrays.asList(names));
      }
  

• 追踪120行发现获取实体getIdentity()方法时抛出空指针异常，说明对象col为null

• cubeDesc类
  cubeDesc 一个实体类，接受cubeDesc参数

import org.apache.kylin.metadata.model.DataModelDesc;
private DataModelDesc model;
public DataModelDesc getModel() {
        return model;
    }

• DataModelDesc类

public TblColRef findColumn(String column) throws IllegalArgumentException {
        TblColRef result = null;
        String input = column;

        column = column.toUpperCase(Locale.ROOT);
        int cut = column.lastIndexOf('.');
        if (cut > 0) {
            // table specified
            result = findColumn(column.substring(0, cut), column.substring(cut + 1));
        } else {
            // table not specified, try each table
            for (TableRef tableRef : allTableRefs) {
                result = tableRef.getColumn(column);
                if (result != null)
                    break;
            }
        }

        if (result == null) {
            logger.warn("Column {} not found in its's model {} , maybe it's a tiretree global domain dict. ", column, getName() );
        }

        return result;
    }

• 最终发现，是获取对应列时找不到时，会打一个warn日志，后面再调用这个对象时为空，抛出异常
  堆栈信息中就有，不过因为是warn，没有注意。

2021-07-01 15:39:14,949 WARN  [http-nio-7070-exec-11] model.DataModelDesc:325 : Column ASDF not found in its's model star_05 , maybe it's a tiretree global domain dict. 

• 最终，删除这个列创建成功，虽然是个很低级的bug，不过通过查源码，了解kylin的部分原理。