Flink JDBCInputFormat并行度

先贴出代码

public static void main(String[] args) {
        //System.getProperties().setProperty("oracle.jdbc.J2EE13Compliant", "true");

        ExecutionEnvironment env = ExecutionEnvironment.getExecutionEnvironment();

        int fetchSize = 7;

        NumericBetweenParametersProvider numericBetweenParametersProvider = new NumericBetweenParametersProvider(fetchSize, 11630494, 11630568);

        String sourceDriverName="oracle.jdbc.driver.OracleDriver";
        String sourceDbUrl = "jdbc:oracle:thin:xxxxx:1521:pacs";
        String sourceUserName = "xxxx_test";
        String sourcePassword = "123456";

        RowTypeInfo rowTypeInfo = new RowTypeInfo(
                BasicTypeInfo.BIG_DEC_TYPE_INFO,
                BasicTypeInfo.STRING_TYPE_INFO
        );

        JDBCInputFormat.JDBCInputFormatBuilder inputBuilder = JDBCInputFormat.buildJDBCInputFormat().setDrivername(sourceDriverName)
                .setDBUrl(sourceDbUrl).setUsername(sourceUserName).setPassword(sourcePassword)
                .setQuery("SELECT ID,ITEM_TYPE FROM OMS_PAYMENT_HISTORY WHERE SOURCE_CODE='OFA' AND ID>=? AND ID<= ?")
                .setFetchSize(fetchSize)
                .setRowTypeInfo(rowTypeInfo).setParametersProvider(numericBetweenParametersProvider);

        DataSet<Row> source = env.createInput(inputBuilder.finish()).setParallelism(numericBetweenParametersProvider.getParameterValues().length);

        try {
            source.print();
            //env.execute("aaaa");
        } catch (Exception e) {
            e.printStackTrace();
        }

    }

设置并行度的关键是

NumericBetweenParametersProvider numericBetweenParametersProvider = new NumericBetweenParametersProvider(7, 11630494, 11630568);

3个参数的意义是 fetchSize: 也就是查询的步长 id>=0 <=9 则fetchSize就是10;  11630494是你这个sql的最小id 11630568是最大id;

sql中的 AND ID>=? AND ID<= ?  是必须的否则无法做参数的拼接  

需要注意如果最小id与最大id的间隔比较大,同时fetchSize比较小时产生的并行度会很大,要注意

如果不设置setParametersProvider,只设置了并行度,那么最终也只会有一个线程来执行查询