Flink之FileSink将数据写入parquet文件_flink写parquet文件(2)

* Convenience builder to create {@link ParquetWriterFactory} instances for the different Avro
* types.
*
* @deprecated use {@link AvroParquetWriters} instead. // 看这部分是建议使用AvroParquetWriters
*/
@Deprecated // 这里已经标注了过时
public class ParquetAvroWriters {

/\*\*

* Creates a ParquetWriterFactory for an Avro specific type. The Parquet writers will use the
* schema of that specific type to build and write the columnar data.
*
* @param type The class of the type to write.
*/
public static ParquetWriterFactory forSpecificRecord(
Class type) {
return AvroParquetWriters.forSpecificRecord(type);
}

* AvroParquetWriters.forReflectRecord(方式三)

 这里就先介绍一下`AvroParquetWriters.forReflectRecord`的使用方式,我们在使用`FileSink`时最好配合`Checkpoint`使用,不然文件只会出现`inprogress`状态,感兴趣的可以自己实验一下,我在[Flink中FileSink的使用]( )演示了加`Checkpoint`和不加`Checkpoint`的区别感兴趣的可以看一下.

 代码模板内容比较简单,直接代码演示:

 

import com.jin.bean.User;
import com.jin.schema.UserSchemaBean;
import org.apache.flink.connector.file.sink.FileSink;
import org.apache.flink.core.fs.Path;
import org.apache.flink.formats.parquet.ParquetWriterFactory;
import org.apache.flink.formats.parquet.avro.AvroParquetWriters;
import org.apache.flink.streaming.api.CheckpointingMode;
import org.apache.flink.streaming.api.datastream.DataStreamSource;
import org.apache.flink.streaming.api.datastream.SingleOutputStreamOperator;
import org.apache.flink.streaming.api.environment.CheckpointConfig;
import org.apache.flink.streaming.api.environment.StreamExecutionEnvironment;
import org.apache.flink.streaming.api.functions.sink.filesystem.bucketassigners.DateTimeBucketAssigner;
import org.apache.flink.streaming.api.functions.sink.filesystem.rollingpolicies.OnCheckpointRollingPolicy;

/**
* @Author: J
* @