mysql resultset用法,一次使用固定数量的结果流式传输MySql ResultSet

I have MySql table with 16 millions records, because of some migration work I'm reading whole Mysql table.

The following code is used for streaming large ResultSet in MySql

statement = connection.createStatement(

java.sql.ResultSet.TYPE_FORWARD_ONLY,

java.sql.ResultSet.CONCUR_READ_ONLY);

statement.setFetchSize(Integer.MIN_VALUE);

but this is streaming one result at a time , does it mean we are hitting MySql server for each row

while using streaming can we set something like this statement.setFetchSize(1000);

I want to reduce number of round trips to server while streaming large ResultSet

解决方案

I will assume that you are using the official MySQL provided JDBC driver Connector/J.

You are explicitly telling JDBC (and MySQL) to stream the results row-by-row with statement.setFetchSize(Integer.MIN_VALUE);

By default, ResultSets are completely retrieved and stored in memory.

In most cases this is the most efficient way to operate, and due to

the design of the MySQL network protocol is easier to implement. If

you are working with ResultSets that have a large number of rows or

large values, and can not allocate heap space in your JVM for the

memory required, you can tell the driver to stream the results back

one row at a time.

To enable this functionality, you need to create a Statement instance

in the following manner:

stmt = conn.createStatement(java.sql.ResultSet.TYPE_FORWARD_ONLY,

java.sql.ResultSet.CONCUR_READ_ONLY);

stmt.setFetchSize(Integer.MIN_VALUE);

The combination of a forward-only, read-only result set, with a fetch

size of Integer.MIN_VALUE serves as a signal to the driver to stream

result sets row-by-row. After this any result sets created with the

statement will be retrieved row-by-row.

Any value other than Integer.MIN_VALUE for the fetch size is ignored by MySQL, and the standard behavior applies. The entire result set will be fetched by the JDBC driver.

Either don't use setFetchSize(), so the JDBC driver will use the default value (0), or set the value to 0 explicitly. Using the value of 0 will also ensure that JDBC doesn't use use MySQL cursors, which may occur depending on your MySQL and Connector/J versions and configuration.