Apache Kylin 3.1.1 CDH6整合 java 参数列表过长

经检查由${KYLIN_HOME}/bin/kylin.sh org.apache.kylin.tool.AclTableMigrationCLI CHECK 报出  

 在启动脚本Kylin 中可以看出是执行 kylin.sh  参数一（$1）=org.apache.kylin.*条件

 

猜测可能是环境变量导致的。分析后发现cached-hive-dependency.sh cached-hbase-dependency.sh有很多jar包路径。可以用 lib/*代替。 

import java.io.*;
import java.util.Arrays;
import java.util.Collection;
import java.util.LinkedHashSet;

 
public class ClassPathZip {

    public static void main(String[] args) {
        System.out.println("args = " + Arrays.deepToString(args));
        File file = new File(args[0]);
        String absolutePath = file.getParentFile().getAbsolutePath();
        File outFile = new File(absolutePath + File.separatorChar + file.getName() + ".out");
        try (BufferedReader bufferedReader = new BufferedReader(new InputStreamReader(new FileInputStream(file), "UTF-8"));
             BufferedWriter writer = new BufferedWriter(new OutputStreamWriter(new FileOutputStream(outFile), "utf-8"))
        ) {
            String line = null;
            LinkedHashSet<String> hashSet = new LinkedHashSet<>();
            StringBuffer sb = new StringBuffer();
            while ((line = bufferedReader.readLine()) != null) {
                if (line.contains(":")) {
                    String[] split = line.split(":");
                    for (String jar : split) {
                        String sourceJar=jar;
                        if (jar.endsWith(".jar")) {
                            jar = jar.substring(0, jar.lastIndexOf("/") + 1) + "*";
                        }
                        if(hashSet.contains(jar)){
                            System.out.println("deleted file "+sourceJar);
                        }
                        hashSet.add(jar);
                    }
                    sb.append(join(hashSet, ":"));
                } else {
                    sb.append(line);
                }
                sb.append("\n");
            }
            sb.append("\n");
            writer.write(sb.toString());
            writer.flush();
        } catch (FileNotFoundException e) {
            e.printStackTrace();
        } catch (IOException e) {
            e.printStackTrace();
        }
    }

    public static String join(Collection<String> collection,String splitChar){
        StringBuffer sb=new StringBuffer();
        int size=collection.size();
        int index=0;
        for (String col:collection) {
            if(index++<size-1){
                sb.append(col);
                sb.append(splitChar);
            }else {
                sb.append(col);
            }
        }
        return  sb.toString();
    }
}

测试执行：  java ClassPathZip /opt/kylin/apache-kylin-3.1.1-bin-cdh60/bin/cached-hive-dependency.sh 
args = [/opt/kylin/apache-kylin-3.1.1-bin-cdh60/bin/cached-hive-dependency.sh]
deleted file /opt/cloudera/parcels/CDH/lib/hive/lib/commons-el-1.0.jar
deleted file /opt/cloudera/parcels/CDH/lib/hive/lib/hive-llap-tez-2.1.1-cdh6.2.0.jar
deleted file /opt/cloudera/parcels/CDH/lib/hive/lib/asm-commons-6.0.jar
deleted file /opt/cloudera/parcels/CDH/lib/hive/lib/tephra-hbase-compat-1.0-0.6.0.jar
deleted file /opt/cloudera/parcels/CDH/lib/hive/lib/org.abego.treelayout.core-1.0.1.jar
deleted file /opt/cloudera/parcels/CDH/lib/hive/lib/aggdesigner-algorithm-6.0.jar

.

输出：

ll *.out
-rw-rw-r-- 1 jenkins jenkins  158 3月  11 17:12 cached-flink-dependency.sh.out
-rw-rw-r-- 1 jenkins jenkins 5005 3月  11 17:35 cached-hbase-dependency.sh.out
-rw-rw-r-- 1 jenkins jenkins  179 3月  11 17:35 cached-hive-dependency.sh.out

修改kylin.sh替换文件.

此处注意不能有不等于6个的cached文件