kafka数据检索_flink 从mysql 读取数据 放入kafka中 用于搜索全量

接着上一篇，将mysql的数据导入kafka中

public static void main(String[] arg) throws Exception {

TypeInformation[] fieldTypes = new TypeInformation[] { BasicTypeInfo.STRING_TYPE_INFO };

RowTypeInfo rowTypeInfo = new RowTypeInfo(fieldTypes);

JDBCInputFormat jdbcInputFormat = JDBCInputFormat.buildJDBCInputFormat().setDrivername("com.mysql.jdbc.Driver")

.setDBUrl("jdbc:mysql://*:3306/tablename?characterEncoding=utf8")

.setUsername("*").setPassword("*")

.setQuery("select LOGIC_CODE from *").setRowTypeInfo(rowTypeInfo).finish();

final ExecutionEnvironment env = ExecutionEnvironment.getExecutionEnvironment();

DataSource s = env.createInput(jdbcInputFormat);

BatchTableEnvironment tableEnv = new BatchTableEnvironment(env, TableConfig.DEFAULT());

tableEnv.registerDataSet("t2", s);

Table query = tableEnv.sqlQuery("select * from t2");

DataSet result = tableEnv.toDataSet(query, Row.class).map(new MapFunction(){

@Override

public String map(Row value) throws Exception {

return value.toString() ;

}

});

logger.info("read db end");

KafkaOutputFormat kafkaOutput = KafkaOutputFormat.buildKafkaOutputFormat()

.setBootstrapServers("*:9092").setTopic("search_test_whk").setAcks("all").setBatchSize("1000")

.setBufferMemory("100000").setLingerMs("1").setRetries("2").finish();

result.output(kafkaOutput);

logger.info("write kafka end");

env.execute("Flink add data source");

}

flink 从mysql 读取数据 放入kafka中 用于搜索全量

标签：arc   form   nal   for   return   stat   sel   tostring   user

本条技术文章来源于互联网，如果无意侵犯您的权益请点击此处反馈版权投诉

本文系统来源：https://blog.51cto.com/12597095/2395983