前言
Android客户端的缓存通常都会放到SQLite数据库中,不过使用原生的API编写CRUD操作既费时又容易出现错误,项目中通常都会引入第三方的开源框架解决问题。GreenDao是一款优秀的ORM框架,只需要简单的配置框架会自动为我们生成DAO访问代码,现在就来简单阅读下它内部的源码实现。
代码分析
在配置好@Entity实体对象后生成代码,会发现指定的目录下会有DaoMaster、DaoSession和XXXDao三个新类。其中DaoMaster包含一个数据库连接,DaoSession代表和数据库的一次会话操作,而一个数据库连接能够支持多次会话操作,DaoMaster可以创建新的DaoSession。DaoSession中实际和数据库做交互的还是Dao对象,DaoSession中包含当前定义的所有Entiy实体对象的Dao对象。
Dao类包含了创建实体对应数据库表和删除数据库表的静态方法,还包含CRUD四种操作的对应方法,这里仅以load和insert方法来查看源码实现过程。从load的源码可以看到它首先查询了identityScope缓存,如果在缓存中存在就直接返回实体对象,否则就构建查询SQL语句并且通过数据库连接db直接查询,获取到Cursor对象后调用loadUniqueAndCloseCursor方法将Cursor转换成实体对象。
public T load(K key) {
assertSinglePk();
if (key == null) {
return null;
}
if (identityScope != null) {
T entity = identityScope.get(key);
if (entity != null) {
return entity;
}
}
String sql = statements.getSelectByKey();
String[] keyArray = new String[]{key.toString()};
Cursor cursor = db.rawQuery(sql, keyArray);
return loadUniqueAndCloseCursor(cursor);
}
在查看loadUniqueAndCloseCursor之前需要了解GreenDao中有一种缓存机制,它会在保存、更新或者查找操作将对象保存到内存中,当用户下一次需要查找对象时优先从缓存中查找,找不到再到数据库中查询。而这种机制就是通过IdentityScope来实现的,它是在DaoConfig对象中定义的,DaoConfig对象会保存实体对应的数据表各种主键、字段名和生成SQL语句TableStatements对象,在解析Entity定义是如果发现它的主键是数字类型,就会创建IdentityScopeLong类型的缓存对象,否则创建IdentityScopeObject类型缓存对象。注意默认的IdentityScopeType是session也就是说这个缓存对象在整个Session期间都是有效的。查看IdentityScopeLong的源码发现它的底层其实就是一个Map对象,这个Map对象的key对应这实体的主键,value对应这实体的pojo对象。
public DaoConfig(DaoConfig source) {
db = source.db;
tablename = source.tablename;
properties = source.properties;
allColumns = source.allColumns;
pkColumns = source.pkColumns;
nonPkColumns = source.nonPkColumns;
pkProperty = source.pkProperty;
statements = source.statements;
keyIsNumeric = source.keyIsNumeric;
}
@SuppressWarnings("rawtypes")
public void initIdentityScope(IdentityScopeType type) {
if (type == IdentityScopeType.None) {
identityScope = null;
} else if (type == IdentityScopeType.Session) {
if (keyIsNumeric) {
identityScope = new IdentityScopeLong();
} else {
identityScope = new IdentityScopeObject();
}
} else {
throw new IllegalArgumentException("Unsupported type: " + type);
}
}
public class IdentityScopeLong<T> implements IdentityScope<Long, T> {
private final LongHashMap<Reference<T>> map;
private final ReentrantLock lock;
public IdentityScopeLong() {
map = new LongHashMap<Reference<T>>();
lock = new ReentrantLock();
}
....
}
了解了IdentityScope缓存实现后再查看在Dao中的缓存对象是如何定义的,可以看到Dao中包含identityScope和identityScopeLong两个字段,如果配置的实体主键为数字类型那么identityScopeLong会被赋值,identityScope为空;反之,identityScope会被赋值而identityScopeLong会被置为空。
public AbstractDao(DaoConfig config, AbstractDaoSession daoSession) {
this.config = config;
this.session = daoSession;
db = config.db;
isStandardSQLite = db.getRawDatabase() instanceof SQLiteDatabase;
identityScope = (IdentityScope<K, T>) config.getIdentityScope();
if (identityScope instanceof IdentityScopeLong) {
identityScopeLong = (IdentityScopeLong<T>) identityScope;
} else {
identityScopeLong = null;
}
statements = config.statements;
pkOrdinal = config.pkProperty != null ? config.pkProperty.ordinal : -1;
}
接着在查看loadUniqueAndCloseCursor实现源码,最终调用的是loadCurrent方法,它会首先判断Cursor有没有查询到数据,如果没有直接返回空,否则会查看对象identityScope缓存有的话直接返回否则调用readEntity方法解析Cursor生成对象。
protected T loadUniqueAndCloseCursor(Cursor cursor) {
try {
return loadUnique(cursor);
} finally {
cursor.close();
}
}
protected T loadUnique(Cursor cursor) {
boolean available = cursor.moveToFirst();
if (!available) {
return null;
} else if (!cursor.isLast()) {
throw new DaoException("Expected unique result, but count was " + cursor.getCount());
}
return loadCurrent(cursor, 0, true);
}
final protected T loadCurrent(Cursor cursor, int offset, boolean lock) {
if (identityScopeLong != null) {
if (offset != 0) {
// Occurs with deep loads (left outer joins)
if (cursor.isNull(pkOrdinal + offset)) {
return null;
}
}
long key = cursor.getLong(pkOrdinal + offset);
T entity = lock ? identityScopeLong.get2(key) : identityScopeLong.get2NoLock(key);
if (entity != null) {
return entity;
} else {
entity = readEntity(cursor, offset);
attachEntity(entity);
if (lock) {
identityScopeLong.put2(key, entity);
} else {
identityScopeLong.put2NoLock(key, entity);
}
return entity;
}
} else if (identityScope != null) {
K key = readKey(cursor, offset);
if (offset != 0 && key == null) {
// Occurs with deep loads (left outer joins)
return null;
}
T entity = lock ? identityScope.get(key) : identityScope.getNoLock(key);
if (entity != null) {
return entity;
} else {
entity = readEntity(cursor, offset);
attachEntity(key, entity, lock);
return entity;
}
} else {
// Check offset, assume a value !=0 indicating a potential outer join, so check PK
if (offset != 0) {
K key = readKey(cursor, offset);
if (key == null) {
// Occurs with deep loads (left outer joins)
return null;
}
}
T entity = readEntity(cursor, offset);
attachEntity(entity);
return entity;
}
}
readEntity在AbstractDao中是抽象,需要在特定的子类中实现,不过可以看到其实就是解析Cusor的中数据然生成实体对象。
@Override
public StudentEntity readEntity(Cursor cursor, int offset) {
StudentEntity entity = new StudentEntity( //
cursor.getInt(offset + 0), // id
cursor.isNull(offset + 1) ? null : cursor.getString(offset + 1), // name
cursor.getInt(offset + 2) // age
);
return entity;
}
以上的代码就实现了将实体对象加载返回给应用程序,接着查看insert实现的源码,注意到Dao里面还有很多带XXXTx的方法,它们和普通的方法有什么区别呢?我们知道如果单独执行insert语句它就会单独作为一个事务提交,如果有大量的数据需要添加到数据库这样会产生严重的性能问题,可以在insert时设置不自动提交事务,等到所有的insert都执行完成在同一提交事务,这样能够极大的提高批量插入效果。那些带Tx的方法会将内部的所有操作统一作为一个事务提交,有时能够提高执行效率。
public long insert(T entity) {
// TableStatements.getInsertStatement()
return executeInsert(entity, statements.getInsertStatement(), true);
}
private long executeInsert(T entity, DatabaseStatement stmt, boolean setKeyAndAttach) {
long rowId;
if (db.isDbLockedByCurrentThread()) {
rowId = insertInsideTx(entity, stmt);
} else {
// Do TX to acquire a connection before locking the stmt to avoid deadlocks
db.beginTransaction();
try {
rowId = insertInsideTx(entity, stmt);
db.setTransactionSuccessful();
} finally {
db.endTransaction();
}
}
if (setKeyAndAttach) {
updateKeyAfterInsertAndAttach(entity, rowId, true);
}
return rowId;
}
private long insertInsideTx(T entity, DatabaseStatement stmt) {
synchronized (stmt) {
if (isStandardSQLite) {
SQLiteStatement rawStmt = (SQLiteStatement) stmt.getRawStatement();
bindValues(rawStmt, entity);
return rawStmt.executeInsert();
} else {
bindValues(stmt, entity);
return stmt.executeInsert();
}
}
}
注意到insert方法内部会调用executeInsert方法,第二个参数是statements.getInsertStatement()它会根据Entity的配置自动生成插入SQL语句,接着调用insertInsideTx方法,手心绑定数据值之后调用rawStmt.executeInsert();实现插入操作。这里的重点是TableStatements、DatabaseStatement和SQLiteStatement三个类的实现,TableStatements实在DaoConfig类中被实例化的,它内部包含了当前实体对应的数据库表的各种键值名,在getInsertStatement方法中就是通过生成SQL语句来初始化DatabaseStatement对象。
public TableStatements(Database db, String tablename, String[] allColumns, String[] pkColumns) {
this.db = db;
this.tablename = tablename;
this.allColumns = allColumns;
this.pkColumns = pkColumns;
}
public DatabaseStatement getInsertStatement() {
if (insertStatement == null) {
String sql = SqlUtils.createSqlInsert("INSERT INTO ", tablename, allColumns);
DatabaseStatement newInsertStatement = db.compileStatement(sql);
synchronized (this) {
if (insertStatement == null) {
insertStatement = newInsertStatement;
}
}
if (insertStatement != newInsertStatement) {
newInsertStatement.close();
}
}
return insertStatement;
}
这里的db是StandardDatabase类型的对象,compileStatement(sql)正好就通过调用SQLiteDatabase对象的compileStatement(SQL语句)生成了SQLiteStatement对象,最后再执行SQLiteStatement.executeInsert();方法。SQLiteStatement是Android系统自带的底层操作SQLite数据接口,这里不再赘述。
private final SQLiteDatabase delegate;
@Override
public DatabaseStatement compileStatement(String sql) {
return new StandardDatabaseStatement(delegate.compileStatement(sql));
}
public SQLiteStatement compileStatement(String sql) throws SQLException {
acquireReference();
try {
return new SQLiteStatement(this, sql, null);
} finally {
releaseReference();
}
}
GreenDao除了支持普通的数据库CRUD操作还支持数据库的加密操作,GreenDao中的DataBase接口有两个实现类StandardDatabase和EncryptedDatabase,标准数据库就是普通的无加密数据库,EncryptedDatabase则是利用sqlcipher实现加密操作。GreenDao还支持异步操作请求,也就是用户提交的各种操作会在子线程中执行,执行完成会通过回调接口通知到用户,一步操作的主要实现类就是AsyncSession类。
public class AsyncSession {
private final AbstractDaoSession daoSession;
private final AsyncOperationExecutor executor;
private int sessionFlags;
public AsyncSession(AbstractDaoSession daoSession) {
this.daoSession = daoSession;
this.executor = new AsyncOperationExecutor();
}
}
public AsyncOperation insert(Object entity, int flags) {
return enqueueEntityOperation(OperationType.Insert, entity, flags);
}
private <E> AsyncOperation enqueEntityOperation(OperationType type, Class<E> entityClass, Object param, int flags) {
AbstractDao<?, ?> dao = daoSession.getDao(entityClass);
AsyncOperation operation = new AsyncOperation(type, dao, null, param, flags | sessionFlags);
executor.enqueue(operation);
return operation;
}
查看insert执行方法会发它在内部调用了enqueEntityOperation,该方法会把操作封装成一个AsyncOperation的对象,并且将对象放到executor中。接着查看AsyncOperation的实现源码,它内部包含了要执行的方法类型,执行的dao对象和执行方法的参数。
AsyncOperation(OperationType type, AbstractDao<?, ?> dao, Database database, Object parameter, int flags) {
this.type = type;
this.flags = flags;
this.dao = (AbstractDao<Object, Object>) dao;
this.database = database;
this.parameter = parameter;
creatorStacktrace = (flags & FLAG_TRACK_CREATOR_STACKTRACE) != 0 ? new Exception("AsyncOperation was created here") : null;
}
最后查看一下AsyncOperationExecutor的实现源码,它内部包含一个线程对象,一个BlockingQueue的队列用于存储提交进来的异步操作对象,在enqueue入队方法中它会将异步操作对象加入队列,如果当前异步执行对象还未运行就需要提交当前Runnable对象。
class AsyncOperationExecutor implements Runnable, Handler.Callback {
private static ExecutorService executorService = Executors.newCachedThreadPool();
private final BlockingQueue<AsyncOperation> queue;
private volatile boolean executorRunning;
private volatile int maxOperationCountToMerge;
private volatile AsyncOperationListener listener;
private volatile AsyncOperationListener listenerMainThread;
public void enqueue(AsyncOperation operation) {
synchronized (this) {
operation.sequenceNumber = ++lastSequenceNumber;
queue.add(operation);
countOperationsEnqueued++;
if (!executorRunning) {
executorRunning = true;
executorService.execute(this);
}
}
}
}
最后查看一下AsyncOperationExecutor的run方法,这里存在一个死循环,不断地从异步操作队列中取异步操作对象,如果队列中所有任务都已经完成那么就退出执行,否则在取下一个异步任务,两个任务可以放到同一个事务中执行就合并操作,否则就两个任务串行执行。
public void run() {
try {
try {
while (true) {
AsyncOperation operation = queue.poll(1, TimeUnit.SECONDS);
if (operation == null) {
synchronized (this) {
// Check again, this time in synchronized to be in sync with enqueue(AsyncOperation)
operation = queue.poll();
if (operation == null) {
// set flag while still inside synchronized
executorRunning = false;
return;
}
}
}
if (operation.isMergeTx()) {
// Wait some ms for another operation to merge because a TX is expensive
AsyncOperation operation2 = queue.poll(waitForMergeMillis, TimeUnit.MILLISECONDS);
if (operation2 != null) {
if (operation.isMergeableWith(operation2)) {
mergeTxAndExecute(operation, operation2);
} else {
// Cannot merge, execute both
executeOperationAndPostCompleted(operation);
executeOperationAndPostCompleted(operation2);
}
continue;
}
}
executeOperationAndPostCompleted(operation);
}
} catch (InterruptedException e) {
DaoLog.w(Thread.currentThread().getName() + " was interruppted", e);
}
} finally {
executorRunning = false;
}
}
执行异步操作是通过executeOperationAndPostCompleted方法来实现的,首先执行数据库操作,接着调用注册的回调接口AsyncOperationListener。
private void executeOperationAndPostCompleted(AsyncOperation operation) {
executeOperation(operation);
handleOperationCompleted(operation);
}
private void executeOperation(AsyncOperation operation) {
operation.timeStarted = System.currentTimeMillis();
try {
switch (operation.type) {
case Insert:
operation.dao.insert(operation.parameter);
break;
}
}
}
private void handleOperationCompleted(AsyncOperation operation) {
operation.setCompleted();
AsyncOperationListener listenerToCall = listener;
if (listenerToCall != null) {
listenerToCall.onAsyncOperationCompleted(operation);
}
if (listenerMainThread != null) {
if (handlerMainThread == null) {
handlerMainThread = new Handler(Looper.getMainLooper(), this);
}
Message msg = handlerMainThread.obtainMessage(1, operation);
handlerMainThread.sendMessage(msg);
}
synchronized (this) {
countOperationsCompleted++;
if (countOperationsCompleted == countOperationsEnqueued) {
notifyAll();
}
}
}
总结
GreenDao根据配置的实体类型生成DaoMaster、DaoSession和Dao三个类,Dao的父类AbstractDao内部会利用DaoConfig对象包含所有解析到的实体和数据库表信息,DaoConfig内部的TableStatement会利用这些信息生成原生SQL语句,TableStatements会利用SQL语句生成SQLiteStatements对象负责执行操作。GreenDao内部的AsyncSession会将用户请求的操作封装成AsyncOperation对象,同时入队到AsyncExecutor内部的队列中,AsyncExecutor内部有一个死循环不断从队列中读取请求操作并执行,执行完成后会调用回调接口通知用户。