Concepts of Cursor and CursorWindow on android platform

简记android平台下ContenProvider使用SQLite数据库实现时的代码片段和逻辑结构，主要描述Cursor和CursorWindow.

Cursor can be treated as an iterater of the traditional ODBC record set and is the current position of the data record. It describes and reflects the logical result record set in android.

AbstractWindowedCursor both in client side and provider-implementing sideassociates a cursor window, which is an ashmem actually on android platform. Same name, same ashmem.

BulkCursorToCursorAdaptor and CursorToBulkCursorAdaptor are used for cursor transfer between processes, they are used to binderize the cursor.

 CursorWindow uses the /dev/ashmem to share the memory and avoids data block transfer to improve efficiency. CursorWindow is a cache window/buffer of the record set which is an obvious design to achieve efficiency. Using ashmem is avoiding record block transfer and is more efficiency in the requirement and circumstance.

IN ALL, CursorWindow is created with RW in service process, and is created with R in client process for with RW failed.

 

The relation of Cursors:
Cursor
  CrossProcessCursor
    AbstractCursor
      AbstractWindowedCursor       
         SQLiteCursor

For BuilkCursorToCursorAdapater's hierachy:
Cursor
  CrossProcessCursor
    AbstractCursor
      AbstractWindowedCursor       
        BulkCursorToCursorAdaptor

The top 4 base classes are same used in client and provider-implementing side.

 

------- Code of Control Flow------------

ContentResolver.java
303    public final Cursor query(Uri uri, String[] projection,
304            String selection, String[] selectionArgs, String sortOrder) {
305        IContentProvider provider = acquireProvider(uri);
306        if (provider == null) {
307            return null;
308        }
309        try {
310            long startTime = SystemClock.uptimeMillis();
311            Cursor qCursor = provider.query(uri, projection, selection, selectionArgs, sortOrder);
312            if (qCursor == null) {
313                releaseProvider(provider);
314                return null;
315            }
316            // force query execution
317            qCursor.getCount();
318            long durationMillis = SystemClock.uptimeMillis() - startTime;
319            maybeLogQueryToEventLog(durationMillis, uri, projection, selection, sortOrder);
320            // Wrap the cursor object into CursorWrapperInner object
321            return new CursorWrapperInner(qCursor, provider);
322        } catch (RemoteException e) {
323            releaseProvider(provider);
324
325            // Arbitrary and not worth documenting, as Activity
326            // Manager will kill this process shortly anyway.
327            return null;
328        } catch (RuntimeException e) {
329            releaseProvider(provider);
330            throw e;
331        }
332    }

IContentProvider.java
static final int QUERY_TRANSACTION = IBinder.FIRST_CALL_TRANSACTION;

ContentProviderNative.java
final class ContentProviderProxy implements IContentProvider
315{
316    public ContentProviderProxy(IBinder remote)
317    {
318        mRemote = remote;
319    }
320
321    public IBinder asBinder()
322    {
323        return mRemote;
324    }
325
326    public Cursor query(Uri url, String[] projection, String selection,
327            String[] selectionArgs, String sortOrder) throws RemoteException {
328        BulkCursorToCursorAdaptor adaptor = new BulkCursorToCursorAdaptor();
329        Parcel data = Parcel.obtain();
330        Parcel reply = Parcel.obtain();
331        try {
332            data.writeInterfaceToken(IContentProvider.descriptor);
333
334            url.writeToParcel(data, 0);
335            int length = 0;
336            if (projection != null) {
337                length = projection.length;
338            }
339            data.writeInt(length);
340            for (int i = 0; i < length; i++) {
341                data.writeString(projection[i]);
342            }
343            data.writeString(selection);
344            if (selectionArgs != null) {
345                length = selectionArgs.length;
346            } else {
347                length = 0;
348            }
349            data.writeInt(length);
350            for (int i = 0; i < length; i++) {
351                data.writeString(selectionArgs[i]);
352            }
353            data.writeString(sortOrder);
354            data.writeStrongBinder(adaptor.getObserver().asBinder());
355
356            mRemote.transact(IContentProvider.QUERY_TRANSACTION, data, reply, 0);
357
358            DatabaseUtils.readExceptionFromParcel(reply);
359
360            IBulkCursor bulkCursor = BulkCursorNative.asInterface(reply.readStrongBinder());
361            if (bulkCursor != null) {
362                int rowCount = reply.readInt();
363                int idColumnPosition = reply.readInt();
364                boolean wantsAllOnMoveCalls = reply.readInt() != 0;
365                adaptor.initialize(bulkCursor, rowCount, idColumnPosition, wantsAllOnMoveCalls);
366            } else {
367                adaptor.close();
368                adaptor = null;
369            }
370            return adaptor;
371        } catch (RemoteException ex) {
372            adaptor.close();
373            throw ex;
374        } catch (RuntimeException ex) {
375            adaptor.close();
376            throw ex;
377        } finally {
378            data.recycle();
379            reply.recycle();
380        }
381    }

ContentProviderNative.java
77    public boolean onTransact(int code, Parcel data, Parcel reply, int flags)
78            throws RemoteException {
79        try {
80            switch (code) {
81                case QUERY_TRANSACTION:
82                {
83                    data.enforceInterface(IContentProvider.descriptor);
84
85                    Uri url = Uri.CREATOR.createFromParcel(data);
86
87                    // String[] projection
88                    int num = data.readInt();
89                    String[] projection = null;
90                    if (num > 0) {
91                        projection = new String[num];
92                        for (int i = 0; i < num; i++) {
93                            projection[i] = data.readString();
94                        }
95                    }
96
97                    // String selection, String[] selectionArgs...
98                    String selection = data.readString();
99                    num = data.readInt();
100                    String[] selectionArgs = null;
101                    if (num > 0) {
102                        selectionArgs = new String[num];
103                        for (int i = 0; i < num; i++) {
104                            selectionArgs[i] = data.readString();
105                        }
106                    }
107
108                    String sortOrder = data.readString();
109                    IContentObserver observer = IContentObserver.Stub.asInterface(
110                            data.readStrongBinder());
111
112                    Cursor cursor = query(url, projection, selection, selectionArgs, sortOrder);
113                    if (cursor != null) {
114                        CursorToBulkCursorAdaptor adaptor = new CursorToBulkCursorAdaptor(
115                                cursor, observer, getProviderName());
116                        final IBinder binder = adaptor.asBinder();
117                        final int count = adaptor.count();
118                        final int index = BulkCursorToCursorAdaptor.findRowIdColumnIndex(
119                                adaptor.getColumnNames());
120                        final boolean wantsAllOnMoveCalls = adaptor.getWantsAllOnMoveCalls();
121
122                        reply.writeNoException();
123                        reply.writeStrongBinder(binder);
124                        reply.writeInt(count);
125                        reply.writeInt(index);
126                        reply.writeInt(wantsAllOnMoveCalls ? 1 : 0);
127                    } else {
128                        reply.writeNoException();
129                        reply.writeStrongBinder(null);
130                    }
131
132                    return true;
133                }
     ……………..
298            }
299        } catch (Exception e) {
300            DatabaseUtils.writeExceptionToParcel(reply, e);
301            return true;
302        }
303
304        return super.onTransact(code, data, reply, flags);
305    }
306


SettingsProvider.java for example.
641    @Override
642    public Cursor query(Uri url, String[] select, String where, String[] whereArgs, String sort) {
643        SqlArguments args = new SqlArguments(url, where, whereArgs);
644        SQLiteDatabase db = mOpenHelper.getReadableDatabase();
645
646        // The favorites table was moved from this provider to a provider inside Home
647        // Home still need to query this table to upgrade from pre-cupcake builds
648        // However, a cupcake+ build with no data does not contain this table which will
649        // cause an exception in the SQL stack. The following line is a special case to
650        // let the caller of the query have a chance to recover and avoid the exception
651        if (TABLE_FAVORITES.equals(args.table)) {
652            return null;
653        } else if (TABLE_OLD_FAVORITES.equals(args.table)) {
654            args.table = TABLE_FAVORITES;
655            Cursor cursor = db.rawQuery("PRAGMA table_info(favorites);", null);
656            if (cursor != null) {
657                boolean exists = cursor.getCount() > 0;
658                cursor.close();
659                if (!exists) return null;
660            } else {
661                return null;
662            }
663        }
664
665        SQLiteQueryBuilder qb = new SQLiteQueryBuilder();
666        qb.setTables(args.table);
667
668        Cursor ret = qb.query(db, select, args.where, args.args, null, null, sort);
669        ret.setNotificationUri(getContext().getContentResolver(), url);
670        return ret;
671    }

SQLiteQueryBuilder.java
327    public Cursor query(SQLiteDatabase db, String[] projectionIn,
328            String selection, String[] selectionArgs, String groupBy,
329            String having, String sortOrder, String limit) {
330        if (mTables == null) {
331            return null;
332        }
333
334        if (mStrict && selection != null && selection.length() > 0) {
335            // Validate the user-supplied selection to detect syntactic anomalies
336            // in the selection string that could indicate a SQL injection attempt.
337            // The idea is to ensure that the selection clause is a valid SQL expression
338            // by compiling it twice: once wrapped in parentheses and once as
339            // originally specified. An attacker cannot create an expression that
340            // would escape the SQL expression while maintaining balanced parentheses
341            // in both the wrapped and original forms.
342            String sqlForValidation = buildQuery(projectionIn, "(" + selection + ")", groupBy,
343                    having, sortOrder, limit);
344            validateSql(db, sqlForValidation); // will throw if query is invalid
345        }
346
347        String sql = buildQuery(
348                projectionIn, selection, groupBy, having,
349                sortOrder, limit);
350
351        if (Log.isLoggable(TAG, Log.DEBUG)) {
352            Log.d(TAG, "Performing query: " + sql);
353        }
354        return db.rawQueryWithFactory(
355                mFactory, sql, selectionArgs,
356                SQLiteDatabase.findEditTable(mTables)); // will throw if query is invalid
357    }

SQLiteDatabase.java
1568    public Cursor rawQueryWithFactory(
1569            CursorFactory cursorFactory, String sql, String[] selectionArgs,
1570            String editTable) {
1571        verifyDbIsOpen();
1572        BlockGuard.getThreadPolicy().onReadFromDisk();
1573
1574        SQLiteDatabase db = getDbConnection(sql);
1575        SQLiteCursorDriver driver = new SQLiteDirectCursorDriver(db, sql, editTable);
1576
1577        Cursor cursor = null;
1578        try {
1579            cursor = driver.query(
1580                    cursorFactory != null ? cursorFactory : mFactory,
1581                    selectionArgs);
1582        } finally {
1583            releaseDbConnection(db);
1584        }
1585        return cursor;
1586    }

SQLiteDirectCursorDriver.java
40    public Cursor query(CursorFactory factory, String[] selectionArgs) {
41        // Compile the query
42        SQLiteQuery query = null;
43
44        try {
45            mDatabase.lock(mSql);
46            mDatabase.closePendingStatements();
47            query = new SQLiteQuery(mDatabase, mSql, 0, selectionArgs);
48
49            // Create the cursor
50            if (factory == null) {
51                mCursor = new SQLiteCursor(this, mEditTable, query);
52            } else {
53                mCursor = factory.newCursor(mDatabase, this, mEditTable, query);
54            }
55
56            mQuery = query;
57            query = null;
58            return mCursor;
59        } finally {
60            // Make sure this object is cleaned up if something happens
61            if (query != null) query.close();
62            mDatabase.unlock();
63        }
64    }

It can be seen that only allocate the SQLiteCursor and save the query statement.

 

If ContentResolver user and ContentProvider in the same process, for example

06-26 16:30:11.439 E/AndroidRuntime(13356):    at android.database.CursorWindow.<init>(CursorWindow.java:104)

06-26 16:30:11.439 E/AndroidRuntime(13356):    at android.database.AbstractWindowedCursor.clearOrCreateWindow(AbstractWindowedCursor.java:198)

06-26 16:30:11.439 E/AndroidRuntime(13356):    at android.database.sqlite.SQLiteCursor.fillWindow(SQLiteCursor.java:162)

06-26 16:30:11.439 E/AndroidRuntime(13356):    at android.database.sqlite.SQLiteCursor.getCount(SQLiteCursor.java:156)

06-26 16:30:11.439 E/AndroidRuntime(13356):    at android.content.ContentResolver.query(ContentResolver.java:317)

06-26 16:30:11.439 E/AndroidRuntime(13356):    at com.android.providers.downloads.DownloadService$UpdateThread.run(DownloadService.java:307)

 

SQLiteCursor.getCount is called directly

153    @Override
154    public int getCount() {
155        if (mCount == NO_COUNT) {
156            fillWindow(0);
157        }
158        return mCount;
159    }
160
161    private void fillWindow(int startPos) {
162        clearOrCreateWindow(getDatabase().getPath());
163        mWindow.setStartPosition(startPos);
164        int count = getQuery().fillWindow(mWindow);
165        if (startPos == 0) { // fillWindow returns count(*) only for startPos = 0
166            if (Log.isLoggable(TAG, Log.DEBUG)) {
167                Log.d(TAG, "received count(*) from native_fill_window: " + count);
168            }
169            mCount = count;
170        } else if (mCount <= 0) {
171            throw new IllegalStateException("Row count should never be zero or negative "
172                    + "when the start position is non-zero");
173        }
174    }
Using the sqlite query to fillWindow.
78    /* package */ int fillWindow(CursorWindow window) {
79        mDatabase.lock(mSql);
80        long timeStart = SystemClock.uptimeMillis();
81        try {
82            acquireReference();
83            try {
84                window.acquireReference();
85                int startPos = window.getStartPosition();
86                int numRows = nativeFillWindow(nHandle, nStatement, window.mWindowPtr,
87                        startPos, mOffsetIndex);
88                if (SQLiteDebug.DEBUG_LOG_SLOW_QUERIES) {
89                    long elapsed = SystemClock.uptimeMillis() - timeStart;
90                    if (SQLiteDebug.shouldLogSlowQuery(elapsed)) {
91                        Log.d(TAG, "fillWindow took " + elapsed
92                                + " ms: window=\"" + window
93                                + "\", startPos=" + startPos
94                                + ", offset=" + mOffsetIndex
95                                + ", filledRows=" + window.getNumRows()
96                                + ", countedRows=" + numRows
97                                + ", query=\"" + mSql + "\""
98                                + ", args=[" + (mBindArgs != null ?
99                                        TextUtils.join(", ", mBindArgs.values()) : "")
100                                + "]");
101                    }
102                }
103                mDatabase.logTimeStat(mSql, timeStart);
104                return numRows;
105            } catch (IllegalStateException e){
106                // simply ignore it
107                return 0;
108            } catch (SQLiteDatabaseCorruptException e) {
109                mDatabase.onCorruption();
110                throw e;
111            } catch (SQLiteException e) {
112                Log.e(TAG, "exception: " + e.getMessage() + "; query: " + mSql);
113                throw e;
114            } finally {
115                window.releaseReference();
116            }
117        } finally {
118            releaseReference();
119            mDatabase.unlock();
120        }
121    }
Use 
38static jint nativeFillWindow(JNIEnv* env, jclass clazz, jint databasePtr,
39        jint statementPtr, jint windowPtr, jint startPos, jint offsetParam) querying sqlite3 to fill window. 

If ContentResolver user and ContentProvider in the different process, qCursur.getCount() returns directly.
79    @Override
80    public int getCount() {
81        throwIfCursorIsClosed();
82        return mCount;
83    }
84
85    @Override
86    public boolean onMove(int oldPosition, int newPosition) {
87        throwIfCursorIsClosed();
88
89        try {
90            // Make sure we have the proper window
91            if (mWindow == null
92                    || newPosition < mWindow.getStartPosition()
93                    || newPosition >= mWindow.getStartPosition() + mWindow.getNumRows()) {
94                setWindow(mBulkCursor.getWindow(newPosition));
95            } else if (mWantsAllOnMoveCalls) {
96                mBulkCursor.onMove(newPosition);
97            }
98        } catch (RemoteException ex) {
99            // We tried to get a window and failed
100            Log.e(TAG, "Unable to get window because the remote process is dead");
101            return false;
102        }
103
104        // Couldn't obtain a window, something is wrong
105        if (mWindow == null) {
106            return false;
107        }
108
109        return true;
110    }

When the qCursor is moved, onMove is called, and setWindow(mBulkCursor.getWindow()) branch is taken for the first time or cursor is out of the current range. And mBulkCursor.onMove branch is called for maintaining cursor position consistent.

In BulkCursorProxy
183    public CursorWindow getWindow(int startPos) throws RemoteException
184    {
185        Parcel data = Parcel.obtain();
186        Parcel reply = Parcel.obtain();
187        try {
188            data.writeInterfaceToken(IBulkCursor.descriptor);
189            data.writeInt(startPos);
190
191            mRemote.transact(GET_CURSOR_WINDOW_TRANSACTION, data, reply, 0);
192            DatabaseUtils.readExceptionFromParcel(reply);
193
194            CursorWindow window = null;
195            if (reply.readInt() == 1) {
196                window = CursorWindow.newFromParcel(reply);
197            }
198            return window;
199        } finally {
200            data.recycle();
201            reply.recycle();
202        }
203    }
204
205    public void onMove(int position) throws RemoteException {
206        Parcel data = Parcel.obtain();
207        Parcel reply = Parcel.obtain();
208        try {
209            data.writeInterfaceToken(IBulkCursor.descriptor);
210            data.writeInt(position);
211
212            mRemote.transact(ON_MOVE_TRANSACTION, data, reply, 0);
213            DatabaseUtils.readExceptionFromParcel(reply);
214        } finally {
215            data.recycle();
216            reply.recycle();
217        }
218    }
219
220    public int count() throws RemoteException
221    {
222        Parcel data = Parcel.obtain();
223        Parcel reply = Parcel.obtain();
224        try {
225            data.writeInterfaceToken(IBulkCursor.descriptor);
226
227            boolean result = mRemote.transact(COUNT_TRANSACTION, data, reply, 0);
228            DatabaseUtils.readExceptionFromParcel(reply);
229
230            int count;
231            if (result == false) {
232                count = -1;
233            } else {
234                count = reply.readInt();
235            }
236            return count;
237        } finally {
238            data.recycle();
239            reply.recycle();
240        }
241    } 

In BulkCursorNative
56    @Override
57    public boolean onTransact(int code, Parcel data, Parcel reply, int flags)
58            throws RemoteException {
59        try {
60            switch (code) {
61                case GET_CURSOR_WINDOW_TRANSACTION: {
62                    data.enforceInterface(IBulkCursor.descriptor);
63                    int startPos = data.readInt();
64                    CursorWindow window = getWindow(startPos);
65                    reply.writeNoException();
66                    if (window == null) {
67                        reply.writeInt(0);
68                    } else {
69                        reply.writeInt(1);
70                        window.writeToParcel(reply, Parcelable.PARCELABLE_WRITE_RETURN_VALUE);
71                    }
72                    return true;
73                }
74
75                case COUNT_TRANSACTION: {
76                    data.enforceInterface(IBulkCursor.descriptor);
77                    int count = count();
78                    reply.writeNoException();
79                    reply.writeInt(count);
80                    return true;
81                }
	}
        }

public abstract class BulkCursorNative extends Binder implements IBulkCursor, while public final class CursorToBulkCursorAdaptor extends BulkCursorNative

In CursorToBulkCursorAdaptor
134    @Override
135    public CursorWindow getWindow(int startPos) {
136        synchronized (mLock) {
137            throwIfCursorIsClosed();
138
139            if (!mCursor.moveToPosition(startPos)) {	// will cause SQLiteCursor create CursorWindow
140                closeFilledWindowLocked();	// close Adaptor’s self mFilledWindow if exsiting
141                return null;
142            }
143
144            CursorWindow window = mCursor.getWindow();    // get SQLiteCursor’s mWindow
145            if (window != null) { // when mCursor.moveToPosition  is called, clearOrCreateWindow-ed
146                closeFilledWindowLocked();	// close Adaptor’s self mFilledWindow if exsiting
147            } else {	// create the window as mFilledWindow if SQLiteCursor has not its Window
148                window = mFilledWindow;
149                if (window == null) {
150                    mFilledWindow = new CursorWindow(mProviderName);
151                    window = mFilledWindow;
152                    mCursor.fillWindow(startPos, window);	// SQLiteCursor fill passed-in window??
153                } else if (startPos < window.getStartPosition()
154                        || startPos >= window.getStartPosition() + window.getNumRows()) {
155                    window.clear();
156                    mCursor.fillWindow(startPos, window);
157                }
158            }
159
160            // Acquire a reference before returning from this RPC.
161            // The Binder proxy will decrement the reference count again as part of writing
162            // the CursorWindow to the reply parcel as a return value.
163            if (window != null) {
164                window.acquireReference();
165            }
166            return window;
167        }
168    }

 

Note the comment in CursorToBulkCursorAdaptor,

26 * Wraps a BulkCursor around an existing Cursor making it remotable.

27 * <p>

28 * If the wrapped cursor returns non-null from {@link CrossProcessCursor#getWindow}

29 * then it is assumed to own the window. Otherwise, the adaptor provides a

30 * window to be filled and ensures it gets closed as needed during deactivation

31 * and requeries.

32 * </p>

mCursor is instance of SQLiteCursor .

In most cases, SQLiteCursor’s CursorWindow is used, so SQLiteCursor::fileWindow is called, same flow with cs same process case.

If SQLiteCursor’s CursorWindow is null, why? Maybe some othrer database rather than SQLite.

A new CursorWindow will be created as CursorToBulkCursorAdaptor::mFilledWindow and is passed to database to fill the window.

191    @Override
192    public void AbstractCursor::fillWindow(int position, CursorWindow window) {
193        DatabaseUtils.cursorFillWindow(this, position, window);
194    }

261    public static void cursorFillWindow(final Cursor cursor,
262            int position, final CursorWindow window) {
263        if (position < 0 || position >= cursor.getCount()) {
264            return;
265        }
266        window.acquireReference();
267        try {
268            final int oldPos = cursor.getPosition();
269            final int numColumns = cursor.getColumnCount();
270            window.clear();
271            window.setStartPosition(position);
272            window.setNumColumns(numColumns);
273            if (cursor.moveToPosition(position)) {
274                do {
275                    if (!window.allocRow()) {
276                        break;
277                    }
…………………
309                        if (!success) {
310                            window.freeLastRow();
311                            break;
312                        }
313                    }
314                    position += 1;
315                } while (cursor.moveToNext());
316            }
317            cursor.moveToPosition(oldPos);
318        } catch (IllegalStateException e){
319            // simply ignore it
320        } finally {
321            window.releaseReference();
322        }
323    }
cursor.getCount() is called.

END