SQLite数据库在Android系统中得到了广泛的应用,Android系统对SQLite的C语言的API进行了封装和扩展,这有其必然性, 因为Android上层是Java实现,必须实现以JNI的方式调用C语言实现的SQLite数据库API, 另外,Android系统在Java语言实现的SDK中和C语言实现的JNI中对SQLite数据库进行了一些功能扩展.本文主要讨论Android系统中cursor的设计实现机制.
Cursor在关系型数据库中是个通用的概念, 简单来说,就是通过cursor获得对数据库的查询结果.在Android系统的设计中, 涉及到2个最核心的类SQLiteCursor和CursorWindow, 其中CursorWindow在Java层和JNI层都有实现.SQLiteCursor中的公共代码实现也存在于其父类之中.二者之间的关系是, 一个SQLiteCursor对象包含一个CursorWindow对象, 是一对一的关系.SQLiteCursor对象是对程序员可见的, 程序员使用SQLiteCursor对象获得查询结果.对于CursorWindow对象, 可以理解为查询结果的缓存,SQLiteCursor对象想要获得查询结果时, 从其关联的CursorWindow对象的缓存中返回, 如果查询的行记录没有在CursorWindow对象的缓存中,会调用SQLite的原生API将获得的查询结果保存在CursorWindow对象中.CursorWindow对象的缓存空间是在JNI层分配实现的.当然cursor这2个最核心的概念不可能独立存在,肯定还会和其他类对象进行交互,包括SQLiteQuery类, SQLiteConnection类,以及SQLiteDatabase类等等.
下面我们通过一些典型的流程来说明cursor相关类的实现细节.第一个流程就是query调用, 由SQLiteDatabase类对象调用, 因为只有query调用以后才能返回cursor实现类的对象, 才能使用cursor对象.SQLiteDatabase类对象的query调用会执行SQLiteDirectCursorDriver类对象的query调用.SQLiteDirectCursorDriver类对其调用类SQLiteDatabase返回cursor对象,并且屏蔽了cursor的具体实现类,返回的是一个cursor接口对象.该函数的实现为(SQLiteDirectCursorDriver.java):
public Cursor query(CursorFactory factory, String[] selectionArgs) {
final SQLiteQuery query = new SQLiteQuery(mDatabase, mSql, mCancellationSignal);
final Cursor cursor;
try {
query.bindAllArgsAsStrings(selectionArgs);
if (factory == null) {
cursor = new SQLiteCursor(this, mEditTable, query);
} else {
cursor = factory.newCursor(mDatabase, this, mEditTable, query);
}
} catch (RuntimeException ex) {
query.close();
throw ex;
}
mQuery = query;
return cursor;
}
该函数首先创建了一个SQLiteQuery类对象, 该类继承于SQLiteProgram,代表一个编译的SQL语句,对应SQLie数据中的statement对象.在SQLiteQuery类的构造函数中,调用父类的构造函数(SQLiteQuery.java):
SQLiteQuery(SQLiteDatabase db, String query, CancellationSignal cancellationSignal) {
super(db, query, null, cancellationSignal);
mCancellationSignal = cancellationSignal;
}
即执行父类SQLiteProgram的构造函数(SQLiteProgram.java):
SQLiteProgram(SQLiteDatabase db, String sql, Object[] bindArgs,
CancellationSignal cancellationSignalForPrepare) {
mDatabase = db;
mSql = sql.trim();
int n = DatabaseUtils.getSqlStatementType(mSql);
switch (n) {
case DatabaseUtils.STATEMENT_BEGIN:
case DatabaseUtils.STATEMENT_COMMIT:
case DatabaseUtils.STATEMENT_ABORT:
mReadOnly = false;
mColumnNames = EMPTY_STRING_ARRAY;
mNumParameters = 0;
break;
default:
boolean assumeReadOnly = (n == DatabaseUtils.STATEMENT_SELECT);
SQLiteStatementInfo info = new SQLiteStatementInfo();
db.getThreadSession().prepare(mSql,
db.getThreadDefaultConnectionFlags(assumeReadOnly),
cancellationSignalForPrepare, info);
mReadOnly = info.readOnly;
mColumnNames = info.columnNames;
mNumParameters = info.numParameters;
break;
}
if (bindArgs != null && bindArgs.length > mNumParameters) {
throw new IllegalArgumentException("Too many bind arguments. "
+ bindArgs.length + " arguments were provided but the statement needs "
+ mNumParameters + " arguments.");
}
if (mNumParameters != 0) {
mBindArgs = new Object[mNumParameters];
if (bindArgs != null) {
System.arraycopy(bindArgs, 0, mBindArgs, 0, bindArgs.length);
}
} else {
mBindArgs = null;
}
}
类SQLiteProgram的构造函数中,除了与事务相关的3个SQL语句外,都调用了类SQLiteSession的prepare函数:
db.getThreadSession().prepare(mSql,
db.getThreadDefaultConnectionFlags(assumeReadOnly),
cancellationSignalForPrepare, info);
最终将调用SQLite数据库中的prepare接口,即会解析SQL语句,并构造SQLite的statement对象.
回到SQLiteDirectCursorDriver对象的query函数, 创建完SQLiteQuery类对象以后, 执行了
query.bindAllArgsAsStrings(selectionArgs);
为SQLiteQuery对象绑定查询参数, 这里只是在Java层面对SQLiteQuery对象赋值,没有调用SQLite原生接口进行绑定, 真正绑定的时机是在执行cursor对象某个API调用的时候,后续将会看到.通常, 我们调用SQLiteDatabase对象的query函数时, 传入的CursorFactory对象是null, 因此这里,将创建一个SQLiteCursor类对象, 作为cursor的具体实现类.然后返回该对象.从以上分析可以看出, 截止到调用query, 其实并没获得查询结果,只是调用SQLite的原生接口对SQL进行了编译.(注:SQLiteDatabase对象可以调用queryWithFactory接口传入CursorFactory对象)
通常, query调用返回cursor对象以后, 我们会接着调用cursor.moveToFirst定位到查询结果的第一条记录, 该函数由cursor的父类AbstractCursor执行, 其实现为(AbstractCursor.java):
@Override
public final boolean moveToFirst() {
return moveToPosition(0);
}
@Override
public final boolean moveToPosition(int position) {
// Make sure position isn't past the end of the cursor
final int count = getCount();
if (position >= count) {
mPos = count;
return false;
}
// Make sure position isn't before the beginning of the cursor
if (position < 0) {
mPos = -1;
return false;
}
// Check for no-op moves, and skip the rest of the work for them
if (position == mPos) {
return true;
}
boolean result = onMove(mPos, position);
if (result == false) {
mPos = -1;
} else {
mPos = position;
}
return result;
}
所以首先会调用getCount函数, 该函数是在具体类SQLiteCursor中实现的(SQLiteCursor.java):
@Override
public int getCount() {
if (mCount == NO_COUNT) {
fillWindow(0);
}
return mCount;
}
该类字段mCount的含义为查询结果记录的个数,注意该字段与字段mCursorWindowCapacity的关系, 该字段表示CursorWindow对象中记录的个数, mCount是大于或者等于mCursorWindowCapacity的值.因为mCount初始值为NO_COUNT,所以会继续执行fillWindow(0);其实现为(SQLiteCursor.java):
private void fillWindow(int requiredPos) {
clearOrCreateWindow(getDatabase().getPath());
try {
if (mCount == NO_COUNT) {
int startPos = DatabaseUtils.cursorPickFillWindowStartPosition(requiredPos, 0);
mCount = mQuery.fillWindow(mWindow, startPos, requiredPos, true);
mCursorWindowCapacity = mWindow.getNumRows();
if (Log.isLoggable(TAG, Log.DEBUG)) {
Log.d(TAG, "received count(*) from native_fill_window: " + mCount);
}
} else {
int startPos = DatabaseUtils.cursorPickFillWindowStartPosition(requiredPos,
mCursorWindowCapacity);
mQuery.fillWindow(mWindow, startPos, requiredPos, false);
}
} catch (RuntimeException ex) {
// Close the cursor window if the query failed and therefore will
// not produce any results. This helps to avoid accidentally leaking
// the cursor window if the client does not correctly handle exceptions
// and fails to close the cursor.
closeWindow();
throw ex;
}
}
该函数首先调用clearOrCreateWindow, 其实现为(AbstractWindowedCursor.java):
/**
* If there is a window, clear it.
* Otherwise, creates a new window.
*
* @param name The window name.
* @hide
*/
protected void clearOrCreateWindow(String name) {
if (mWindow == null) {
mWindow = new CursorWindow(name);
} else {
mWindow.clear();
}
}
mWindow字段即SQLiteCursor对象中的成员, 在SQLiteCursor对象创建时, 其为null,因此这里会创建一个CursorWindow对象(CursorWindow.java).
public CursorWindow(String name) {
mStartPos = 0;
mName = name != null && name.length() != 0 ? name : "<unnamed>";
if (sCursorWindowSize < 0) {
/** The cursor window size. resource xml file specifies the value in kB.
* convert it to bytes here by multiplying with 1024.
*/
sCursorWindowSize = Resources.getSystem().getInteger(
com.android.internal.R.integer.config_cursorWindowSize) * 1024;
}
mWindowPtr = nativeCreate(mName, sCursorWindowSize);
if (mWindowPtr == 0) {
throw new CursorWindowAllocationException("Cursor window allocation of " +
(sCursorWindowSize / 1024) + " kb failed. " + printStats());
}
mCloseGuard.open("close");
recordNewWindow(Binder.getCallingPid(), mWindowPtr);
}
该构造函数首先根据配置获得CursorWindow对象的容量大小sCursorWindowSize, 然后执行JNI调用
mWindowPtr = nativeCreate(mName, sCursorWindowSize);在JNI层创建一个CursorWindow对象,并返回该对象指针到Java层.该native方法对应的实现为nativeCreate(android_database_CursorWindow.cpp):
static jlong nativeCreate(JNIEnv* env, jclass clazz, jstring nameObj, jint cursorWindowSize) {
String8 name;
const char* nameStr = env->GetStringUTFChars(nameObj, NULL);
name.setTo(nameStr);
env->ReleaseStringUTFChars(nameObj, nameStr);
CursorWindow* window;
status_t status = CursorWindow::create(name, cursorWindowSize, &window);
if (status || !window) {
ALOGE("Could not allocate CursorWindow '%s' of size %d due to error %d.",
name.string(), cursorWindowSize, status);
return 0;
}
LOG_WINDOW("nativeInitializeEmpty: window = %p", window);
return reinterpret_cast<jlong>(window);
}
我们再来看看创建的JNI层的CursorWindow对象(CursorWindow.cpp):
status_t CursorWindow::create(const String8& name, size_t size, CursorWindow** outCursorWindow) {
String8 ashmemName("CursorWindow: ");
ashmemName.append(name);
status_t result;
int ashmemFd = ashmem_create_region(ashmemName.string(), size);
if (ashmemFd < 0) {
result = -errno;
} else {
result = ashmem_set_prot_region(ashmemFd, PROT_READ | PROT_WRITE);
if (result >= 0) {
void* data = ::mmap(NULL, size, PROT_READ | PROT_WRITE, MAP_SHARED, ashmemFd, 0);
if (data == MAP_FAILED) {
result = -errno;
} else {
result = ashmem_set_prot_region(ashmemFd, PROT_READ);
if (result >= 0) {
CursorWindow* window = new CursorWindow(name, ashmemFd,
data, size, false /*readOnly*/);
result = window->clear();
if (!result) {
LOG_WINDOW("Created new CursorWindow: freeOffset=%d, "
"numRows=%d, numColumns=%d, mSize=%d, mData=%p",
window->mHeader->freeOffset,
window->mHeader->numRows,
window->mHeader->numColumns,
window->mSize, window->mData);
*outCursorWindow = window;
return OK;
}
delete window;
}
}
::munmap(data, size);
}
::close(ashmemFd);
}
*outCursorWindow = NULL;
return result;
}
其中心思想是创建了CursorWindow对象,并关联了一块内存空间,用来缓存查询结果.内存空间的大小有调用JNI方法的Java层传入.
回到SQLiteCursor的fillWindow函数,接着执行代码mCount = mQuery.fillWindow(mWindow, startPos, requiredPos, true);注意输入参数startPos和requiredPos的含义,startPos的值小于等于requiredPos, 这样cursor可以从CursorWindow对象的缓存中,返回指定位置之前或者之后的记录, 最后一个参数true的含义是,该函数的返回值是满足查询条件的所有记录数目.该函数的实现为(SQLiteQuery.java):
/**
* Reads rows into a buffer.
*
* @param window The window to fill into
* @param startPos The start position for filling the window.
* @param requiredPos The position of a row that MUST be in the window.
* If it won't fit, then the query should discard part of what it filled.
* @param countAllRows True to count all rows that the query would
* return regardless of whether they fit in the window.
* @return Number of rows that were enumerated. Might not be all rows
* unless countAllRows is true.
*
* @throws SQLiteException if an error occurs.
* @throws OperationCanceledException if the operation was canceled.
*/
int fillWindow(CursorWindow window, int startPos, int requiredPos, boolean countAllRows) {
acquireReference();
try {
window.acquireReference();
try {
int numRows = getSession().executeForCursorWindow(getSql(), getBindArgs(),
window, startPos, requiredPos, countAllRows, getConnectionFlags(),
mCancellationSignal);
return numRows;
} catch (SQLiteDatabaseCorruptException ex) {
onCorruption();
throw ex;
} catch (SQLiteException ex) {
Log.e(TAG, "exception: " + ex.getMessage() + "; query: " + getSql());
throw ex;
} finally {
window.releaseReference();
}
} finally {
releaseReference();
}
}
其中核心逻辑是执行SQLiteSession对象的executeForCursorWindow调用,该函数中又会调用SQLiteConnection对象的executeForCursorWindow函数,其实现为(SQLiteConnection.java):
/**
* Executes a statement and populates the specified {@link CursorWindow}
* with a range of results. Returns the number of rows that were counted
* during query execution.
*
* @param sql The SQL statement to execute.
* @param bindArgs The arguments to bind, or null if none.
* @param window The cursor window to clear and fill.
* @param startPos The start position for filling the window.
* @param requiredPos The position of a row that MUST be in the window.
* If it won't fit, then the query should discard part of what it filled
* so that it does. Must be greater than or equal to <code>startPos</code>.
* @param countAllRows True to count all rows that the query would return
* regagless of whether they fit in the window.
* @param cancellationSignal A signal to cancel the operation in progress, or null if none.
* @return The number of rows that were counted during query execution. Might
* not be all rows in the result set unless <code>countAllRows</code> is true.
*
* @throws SQLiteException if an error occurs, such as a syntax error
* or invalid number of bind arguments.
* @throws OperationCanceledException if the operation was canceled.
*/
public int executeForCursorWindow(String sql, Object[] bindArgs,
CursorWindow window, int startPos, int requiredPos, boolean countAllRows,
CancellationSignal cancellationSignal) {
if (sql == null) {
throw new IllegalArgumentException("sql must not be null.");
}
if (window == null) {
throw new IllegalArgumentException("window must not be null.");
}
window.acquireReference();
try {
int actualPos = -1;
int countedRows = -1;
int filledRows = -1;
final int cookie = mRecentOperations.beginOperation("executeForCursorWindow",
sql, bindArgs);
try {
final PreparedStatement statement = acquirePreparedStatement(sql);
try {
throwIfStatementForbidden(statement);
bindArguments(statement, bindArgs);
applyBlockGuardPolicy(statement);
attachCancellationSignal(cancellationSignal);
try {
final long result = nativeExecuteForCursorWindow(
mConnectionPtr, statement.mStatementPtr, window.mWindowPtr,
startPos, requiredPos, countAllRows);
actualPos = (int)(result >> 32);
countedRows = (int)result;
filledRows = window.getNumRows();
window.setStartPosition(actualPos);
return countedRows;
} finally {
detachCancellationSignal(cancellationSignal);
}
} finally {
releasePreparedStatement(statement);
}
} catch (RuntimeException ex) {
mRecentOperations.failOperation(cookie, ex);
throw ex;
} finally {
if (mRecentOperations.endOperationDeferLog(cookie)) {
mRecentOperations.logOperation(cookie, "window='" + window
+ "', startPos=" + startPos
+ ", actualPos=" + actualPos
+ ", filledRows=" + filledRows
+ ", countedRows=" + countedRows);
}
}
} finally {
window.releaseReference();
}
}
该函数中通过调用acquirePreparedStatement获得SQL语句对应的PreparedStatement对象,然后调用bindArguments为其绑定参数,该函数会真正的调用SQLite原生接口进行参数绑定.最后执行JNI调用nativeExecuteForCursorWindow,该函数实际执行的是(android_database_SQLiteConnection.cpp):
static jlong nativeExecuteForCursorWindow(JNIEnv* env, jclass clazz,
jlong connectionPtr, jlong statementPtr, jlong windowPtr,
jint startPos, jint requiredPos, jboolean countAllRows) {
SQLiteConnection* connection = reinterpret_cast<SQLiteConnection*>(connectionPtr);
sqlite3_stmt* statement = reinterpret_cast<sqlite3_stmt*>(statementPtr);
CursorWindow* window = reinterpret_cast<CursorWindow*>(windowPtr);
status_t status = window->clear();
if (status) {
String8 msg;
msg.appendFormat("Failed to clear the cursor window, status=%d", status);
throw_sqlite3_exception(env, connection->db, msg.string());
return 0;
}
int numColumns = sqlite3_column_count(statement);
status = window->setNumColumns(numColumns);
if (status) {
String8 msg;
msg.appendFormat("Failed to set the cursor window column count to %d, status=%d",
numColumns, status);
throw_sqlite3_exception(env, connection->db, msg.string());
return 0;
}
int retryCount = 0;
int totalRows = 0;
int addedRows = 0;
bool windowFull = false;
bool gotException = false;
while (!gotException && (!windowFull || countAllRows)) {
int err = sqlite3_step(statement);
if (err == SQLITE_ROW) {
LOG_WINDOW("Stepped statement %p to row %d", statement, totalRows);
retryCount = 0;
totalRows += 1;
// Skip the row if the window is full or we haven't reached the start position yet.
if (startPos >= totalRows || windowFull) {
continue;
}
CopyRowResult cpr = copyRow(env, window, statement, numColumns, startPos, addedRows);
if (cpr == CPR_FULL && addedRows && startPos + addedRows <= requiredPos) {
// We filled the window before we got to the one row that we really wanted.
// Clear the window and start filling it again from here.
// TODO: Would be nicer if we could progressively replace earlier rows.
window->clear();
window->setNumColumns(numColumns);
startPos += addedRows;
addedRows = 0;
cpr = copyRow(env, window, statement, numColumns, startPos, addedRows);
}
if (cpr == CPR_OK) {
addedRows += 1;
} else if (cpr == CPR_FULL) {
windowFull = true;
} else {
gotException = true;
}
} else if (err == SQLITE_DONE) {
// All rows processed, bail
LOG_WINDOW("Processed all rows");
break;
} else if (err == SQLITE_LOCKED || err == SQLITE_BUSY) {
// The table is locked, retry
LOG_WINDOW("Database locked, retrying");
if (retryCount > 50) {
ALOGE("Bailing on database busy retry");
throw_sqlite3_exception(env, connection->db, "retrycount exceeded");
gotException = true;
} else {
// Sleep to give the thread holding the lock a chance to finish
usleep(1000);
retryCount++;
}
} else {
throw_sqlite3_exception(env, connection->db);
gotException = true;
}
}
LOG_WINDOW("Resetting statement %p after fetching %d rows and adding %d rows"
"to the window in %d bytes",
statement, totalRows, addedRows, window->size() - window->freeSpace());
sqlite3_reset(statement);
// Report the total number of rows on request.
if (startPos > totalRows) {
ALOGE("startPos %d > actual rows %d", startPos, totalRows);
}
if (totalRows > 0 && addedRows == 0) {
String8 msg;
msg.appendFormat("Row too big to fit into CursorWindow requiredPos=%d, totalRows=%d",
requiredPos, totalRows);
throw_sqlite3_exception(env, SQLITE_TOOBIG, NULL, msg.string());
return 0;
}
jlong result = jlong(startPos) << 32 | jlong(totalRows);
return result;
}
该函数首先调用window->clear()复位CursorWindow对象状态,然后调用sqlite3_column_count获得该statement对象的列个数.如果没有异常发生,并且CursorWindow对象的缓存没有填充满,或者countAllRows变量为true(其含义是要求获得查询的记录个数),则循环调用sqlite3_step返回查询结果,如果有查询结果(SQLITE_ROW),则调用copyRow将查询结果保存到CursorWindow对象的缓存中,如果查询结果处理完毕(SQLITE_DONE),跳出循环.这里可以看到,android的JNI实现中,查询结果的个数为int, 如果数据库表中返回的查询结果很多,该字段有可能溢出.
回到Java类SQLiteConnection的executeForCursorWindow函数,该函数返回查询结果记录数目.
回到SQLiteCursor类的fillWindow函数,执行代码mCursorWindowCapacity = mWindow.getNumRows();设置其值为CursorWindow对象的记录数.至此, SQLiteCursor对象的getCount函数已经执行完毕,所以,该函数会调用SQLite原生API将查询结果保存到CursorWindow对象中.
回到类AbstractCursor中的函数moveToPosition,getCount函数返回后, 对传入参数position(记录索引)进行判断, 如果该值大于等于getCount的返回值,其已经遍历完了所有查询结果,如果该参数小于0,应该为异常情况,如果该参数等于当前索引,则直接返回.否则调用onMove函数,其实现为(SQLiteCursor.java):
@Override
public boolean onMove(int oldPosition, int newPosition) {
// Make sure the row at newPosition is present in the window
if (mWindow == null || newPosition < mWindow.getStartPosition() ||
newPosition >= (mWindow.getStartPosition() + mWindow.getNumRows())) {
fillWindow(newPosition);
}
return true;
}
该函数会判断传入的新索引(newPosition)是否在CursorWindow对象的缓存空间中,如果已经在缓存中,则直接返回.否则更新CursorWindow对象的缓存.
至此, cursor对象的moveToFirst调用执行完毕.
cursor中其他API会使用缓存的记录,比如获得行记录的列数据等等,在此就不再分析.
BTW:
在Java层的CursorWindow类中,其构造函数中调用了recordNewWindow(Binder.getCallingPid(), mWindowPtr);将JNI层的CursorWindow对象指针记录到Java层的类对象中,因为指针可能是个很大的数值,这里使用了LongSparseArray数据结构进行保存.该数据结构可以保存索引有空隙的映射,.相比较HashMap数据结构而言,其优点是节省内存占用.但是对于执行效率,要看具体使用场景,如果存在大量无序或者倒序的数据插入,则可能其执行效率低于HashMap.
423
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
