0 Hive Streaming
A typical hive job is scheduled periodically to execute, so there will be a large delay.
Flink supports to write, read and join the hive table in the form of streaming.
1 Streaming Writing
The Hive table supports streaming writes, based on Filesystem Streaming Sink.
The Hive Streaming Sink re-use Filesystem Streaming Sink to integrate Hadoop OutputFormat/RecordWriter to streaming writing. Hadoop RecordWriters are Bulk-encoded Formats, Bulk Formats rolls files on every checkpoint.
By default, now only have renaming committer, this means S3 filesystem can not supports exactly-once, if you want to use Hive streaming sink in S3 filesystem, You can configure the following parameter to false to us