Android I/O Prefetch 学习

Android Iop 学习

思路整理：明确方向和目的

这个季度（剩余60天），我有一个重要的任务：完成Iop（I/O Prefetcher）的原理学习。目前我已经看Iop快一周了，但是还是不怎么懂，感觉距离理解还得一个多月，最主要的原因是其所涉及到的内容很广，从Android 的 FrameWork层延申到了hal层，自顶向下的链路大概是：
FrameWork{AMS，BoostFrameWork(通过反射调用Performance)}Java语言实现 ----> 中间转发层 Preformance 通过 jni ----> 调用Hal 层的 c++ 代码，c++ 代码完成预取操作。
这个学习过程有一个难点的地方是，AMS，这个地方所涉及的知识很多，又比较杂，BoostFrameWork穿插在AMS这个框架里的不少地方，来实现Iop。而AMS(ActivityManagerService)这个是Android 四大组件中最重要的一个，其功能主要是用来管理进程的启动调度，及内存管理。
在这个季度尾，我们部门有一个一小时的学习内容讲解，需要有深度的讲解一下技术，我师傅对我的安排是把Iop输出文档，并对Iop做一个有深度的讲解，目前看来，我的心里是没什么把握的，毕竟网上几乎搜不到相关的内容，唯一一篇文章，android IO Prefetch源码分析，很多地方讲的比较粗，看起来很费劲，对于我一个Android系统 新手来说，在FrameWork层的理解还是新手，不过这篇文章也很有参考意义。现在来罗列一下我目前所拥有的工具与资料：

1. android IO Prefetch 源码分析
2. Android 11 Iop HAL层 + BoostFrameWork源码框架 BoostFramework未穿插在 FrameWork 中
3. Android 10 Iop 在FrameWork 层的源码，BoostFramework 穿插在 FrameWork中
4. 随处可见的 AMS 讲解资料
5. 调试的板子和环境可用于验证结果

我的任务：

1. 完成 Iop 在 Android 11 的成功启动
2. 测试 Iop 在 App 冷启动方面的优化效果
3. 确认是否集成到新平台

标题学习计划制定

第一阶段

以目前的状态，自顶向下的学习，是效率最高的一种方案。故采取自顶向下的学习方式，来进行学习。虽说自顶向下，但是并非需要将FrameWork全学了，这样时间肯定不足，也大大脱离了目标。对于上层的学习目前只需要学习AMS，但是AMS依然很大，由我最近这几天的学习，发现AMS依然可以拆分，大致分为：

ActivityManagerService.java
ActivityStack.java
ProcessRecord.java
ActivityStackSupervisor.java
ActivityStarter.java
ActivityRecord.java

以上这几个模块需要重点关注，其余知识大致理解流程即可，以上这几个模块的代码重点阅读与理解，毕竟要在Android 11 上对这几块代码进行修改，现在拥有Android 10 编写的Iop的框架，可以进行模仿，但是前提条件是需要理解这几部分是做什么的，不然所做的代码移植就是不经思考的模仿。

第一阶段结果展望：

预期日期	截止日期	结果
2022/11/4	2022/11/10	完成Iop的成功启动，且输出一篇BoostFrameWork Iop部分和AMS部分互相作用机制的文档。

Activity 启动流程梳理

下图是Activity启动的框架流程图，包含了activity 整体流程的框架

较老Android 版本 Activity启动流程图

手动绘制的代码流程图，Activity的启动流程

1. 学习Iop中遇到的一个人让我疑惑不解问题，I/O Prefetch 在AMS中是在哪些节点上生效会导致App启动被加速？
   根据上图可知道Activity的启动会涉及两种情况，第一种是已经被启动过了一次，可以直接调用realStartActivityLocked()来启动，还有一种是未曾被启动过，需要调用startProcessAsync通过层层调用使用socket通知Zygote去fork一个新的进程，但最后通过层层调用还是会回到realStartActivityLocked来启动，而realStartActivityLocked 调用了findTaskToMoveToFront，而其包含了acquireAppLaunchPerfLock，acquireAppLaunchPerfLock 就是我们执行I/O Prefetch的部分，在此过程中发生Iop预读来加速App启动时间。

   android/frameworks/base/services/core/java/com/android/server/wm/ActivityStackSupervisor.java

void startSpecificActivity(ActivityRecord r, boolean andResume, boolean checkConfig) {
			...........
        boolean knownToBeDead = false;
        if (wpc != null && wpc.hasThread()) {
            try {
                realStartActivityLocked(r, wpc, andResume, checkConfig);
                return;
            } catch (RemoteException e) {
                Slog.w(TAG, "Exception when starting activity "
                        + r.intent.getComponent().flattenToShortString(), e);
            }
            .......
        final boolean isTop = andResume && r.isTopRunningActivity();
        mService.startProcessAsync(r, knownToBeDead, isTop, isTop ? "top-activity" : "activity");
    }

void findTaskToMoveToFront(Task task, int flags, ActivityOptions options, String reason,
            boolean forceNonResizeable) {
        .....
        //top_activity = task.stack.topRunningActivityLocked();
        /* App is launching from recent apps and it's a new process */
        ActivityRecord top_activity = task.topRunningActivityLocked();
        if(top_activity != null && top_activity.isState(DESTROYED)) {
            acquireAppLaunchPerfLock(top_activity.packageName);
        }
        ....

这里使用了BoostFramework 里面的调用，开启I/O预取，但并非第一次启动App

public void acquireAppLaunchPerfLock(String packageName) {
        /* Acquire perf lock during new app launch */
        if (mPerfBoost == null) {
            mPerfBoost = new BoostFramework();
        }
        if (mPerfBoost != null) {
            mPerfBoost.perfHint(BoostFramework.VENDOR_HINT_FIRST_LAUNCH_BOOST, packageName, -1, BoostFramework.Launch.BOOST_V1);
            mPerfSendTapHint = true;
        }
        if (mPerfPack == null) {
            mPerfPack = new BoostFramework();
        }
        if (mPerfPack != null) {
            mPerfPack.perfHint(BoostFramework.VENDOR_HINT_FIRST_LAUNCH_BOOST, packageName, -1, BoostFramework.Launch.BOOST_V2);
        }
        // Start IOP
        if (mPerfIop == null) {
            mPerfIop = new BoostFramework();
        }
        if (mPerfIop != null) {
            mPerfIop.perfIOPrefetchStart(-1,packageName,"");
        }
    }

    public void acquireUxPerfLock(int opcode, String packageName) {
        mUxPerf = new BoostFramework();
        if (mUxPerf != null) {
            mUxPerf.perfUXEngine_events(opcode, 0, packageName, 0);
        }
    }

2. 第二个问题，为什么在次节点上会产生加速效果？
   原因：许多应用程序在启动时需要访问I/O.很多时间会因为阻塞I / O而导致应用程序启动慢。预取数据之后，应用程序几乎可以从pagecache 中立即访问该数据，从而大大减少了应用程序启动延迟。

接下来要聊的是Activity 的状态转换关系。

基于上面的结论，我们启动一个新的App会调用到realStartActivityLocked这个函数，而此时Activity的状态正是处于OnCreate开始前，处于INITIALIZING 状态，在此处我们插入Iop的预取可实现优化。Activity 的所有状态如下：

    enum ActivityState {
        INITIALIZING,
        RESUMED,
        PAUSING,
        PAUSED,
        STOPPING,
        STOPPED,
        FINISHING,
        DESTROYING,
        DESTROYED,
        RESTARTING_PROCESS
    }

整体流程节点如下：

完成日期	完成结果
2022/10/30	梳理了Activity的启动及IO Prefetch的执行节点

第二阶段

第二阶段相对第一阶段较容易，因为这一周的时间已经用来看了Iop的hal层的实现机制了，且代码上的分布很紧凑，代码量小，涉及的架构不怎么庞大，正常进行学习即可，顺便学习一下JNI机制，衍射。
下图是关于Iop调用的整体流程图如下：

Iop初始化流程
首先当系统启动时，会启动 vendor.qti.hardware.iop@2.0-service 进程，该进程相当于一个Server端，他在最开始运行时会进行初始化，首先调用dlopen加载libqti-iopd.so动态库，初始化函数指针，加载libqti-iopd.so库中的函数，以便之后用到的时候，对函数进行调用。接下来看看他是如何做的：

void Iop::LoadIopLib() {
    const char *rc = NULL;
    char buf[PROPERTY_VALUE_MAX];

    if (!mHandle.is_opened) {
         mHandle.dlhandle = dlopen("libqti-iopd.so", RTLD_NOW | RTLD_LOCAL);
         if (mHandle.dlhandle == NULL) {
             ALOGE("IOP-HAL %s Failed to (dl)open iopd\n", __func__);
             return;
         }

         dlerror();

         *(void **) (&mHandle.iop_server_init) = dlsym(mHandle.dlhandle, "iop_server_init");
         if ((rc = dlerror()) != NULL) {
             ALOGE("IOP-HAL %s Failed to get iop_server_init\n", rc);
             dlclose(mHandle.dlhandle);
             mHandle.dlhandle = NULL;
             return;
         }

         *(void **) (&mHandle.iop_server_exit) = dlsym(mHandle.dlhandle, "iop_server_exit");
         if ((rc = dlerror()) != NULL) {
             ALOGE("IOP-HAL %s Failed to get iop_server_exit\n", rc);
             dlclose(mHandle.dlhandle);
             mHandle.dlhandle = NULL;
             return;
         }

         *(void **) (&mHandle.iop_server_submit_request) = dlsym(mHandle.dlhandle, "iop_server_submit_request");
         if ((rc = dlerror()) != NULL) {
             ALOGE("IOP-HAL %s Failed to get iop_server_submit_request\n", rc);
             dlclose(mHandle.dlhandle);
             mHandle.dlhandle = NULL;
             return;
         }

         *(void **) (&mHandle.uxe_server_submit_request) = dlsym(mHandle.dlhandle, "uxe_server_submit_request");
         if ((rc = dlerror()) != NULL) {
             ALOGE("IOP-HAL %s Failed to get uxe_server_submit_request\n", __func__);
             dlclose(mHandle.dlhandle);
             mHandle.dlhandle = NULL;
             return;
         }
         mHandle.is_opened = true;
    }

    return;
}

在加载了libqti-iopd.so动态库的函数之后，该进程便会调用iop_server_init的初始化操作，在这里调用了iop_server_init等待Framework层传下来的信息进行处理。

IIop* HIDL_FETCH_IIop(const char* /* name */) {
    ALOGE("IOP-HAL: inside HIDL_FETCH_IIop");
    Iop *iopObj = new (std::nothrow) Iop();
    ALOGE("IOP-HAL: boot Address of iop object");
    if (iopObj != NULL) {
        iopObj->LoadIopLib();
        ALOGE("IOP-HAL: loading library is done");
        if (iopObj->mHandle.iop_server_init != NULL ) {
            (*(iopObj->mHandle.iop_server_init))();
        }
    }
    return iopObj;
}

Iop_server_init这个函数会创建了一个死循环的iop_server线程，用来处理从Framework层传下来的msg信息。

int iop_server_init() {
    int rc1 = 0, stage = 0, rc2 = 0;
    int uba_rc = 0;
    char property[PROPERTY_VALUE_MAX];
    char trace_prop[PROPERTY_VALUE_MAX];

    QLOGI("IOP server starting");

    /* Enable traces by adding vendor.debug.trace.perf=1 into build.prop */
    if (property_get(PROP_NAME, trace_prop, NULL) > 0) {
        if (trace_prop[0] == '1') {
            perf_debug_output = PERF_SYSTRACE = atoi(trace_prop);
        }
    }

    preferred_apps = (char *) malloc (strcat_sz * sizeof(char));
    if (preferred_apps == NULL) {
        goto error;
    }
    memset(preferred_apps, 0, strcat_sz);
    preferred_apps[0] = '\0';

    IOPevqueue.GetDataPool().SetCBs(Alloccb, Dealloccb);

    rc1 = pthread_create(&iop_server_thread, NULL, iop_server, NULL);
    if (rc1 != 0) {
        stage = 3;
        goto error;
    }
    iop_init = true;

    if (uxe_disabled()) {
        uxe_prop_disable = 1;
    } else {
        uxe_prop_disable = 0;

        strlcpy(property,perf_get_prop("vendor.iop.uxe_trigger_freq" , "1").value, PROPERTY_VALUE_MAX);
        property[PROPERTY_VALUE_MAX-1]='\0';

        uxe_trigger_freq = atoi(property);
        UXEevqueue.GetDataPool().SetCBs(Alloccb, Dealloccb);
        rc2 = pthread_create(&uxe_server_thread, NULL, uxe_server, NULL);
        uba_rc = init_uba();
        if (rc2 != 0) {
            goto error;
        }
        if (uba_rc == 0) {
            QLOGE("Fail to init UBA");
            goto error;
        }
        uxe_init = true;
    }

    return 1;

error:
    QLOGE("Unable to create control service (stage=%d, rc1=%d rc2=%d uba_rc=%d)", stage, rc1, rc2, uba_rc);
    return 0;
}

在 iop_server中，首先初始化一些参数，然后会向IOPevqueue队列申请一个msg，若队列中没有请求消息则阻塞在这里，等待队列中出现msg，若有msg，则取出来，走接下来的流程，然后创建数据库，若数据库没有被创建则创建数据库。因为我们提交的消息一般是IopStart 的过程，所以消息类型是case IOP_CMD_PERFLOCK_IOPREFETCH_START，所以我们走这条分支。接下来他会获取配置文件的配置参数，是否开启某些功能。配置文件路径 /vendor/etc/perf/perfconfigstore.xml ，配置文件基本内容如下图

可根据vendor.enable.prefetch是否为1是否开启iop预读。最开始执行信息采集的是pid > 0，所以先分析pid > 0的情况。

static void *iop_server(void *data)
{
    int rc, cmd;
    iop_msg_t *msg = NULL;
    (void)data;
    bool is_db_init = false;
    char tmp_pkg_name[PKG_NAME_LEN];
    int bindApp_dur = 0, disAct_dur = 0, avg_bindApp = 0, avg_disAct = 0;
    int pkg_count = 0, bindApp_count = 0, pkg_mismatch = 0, da_reset = 0;
    bool launching = false;

    memset(tmp_pkg_name, 0, PKG_NAME_LEN);
    /* Main loop */
    for (;;) {
       //wait for perflock commands
        EventData *evData = IOPevqueue.Wait();

        if (!evData || !evData->mEvData) {
            continue;
        }
        if(!is_boot_complete())
        {
            QLOGE("io prefetch is disabled waiting for boot_completed");
            continue;
        }
        if(is_db_init == false)
        {
            if(create_database() == 0)
            {
                //Success
                is_db_init = true;
            }
        }
        case IOP_CMD_PERFLOCK_IOPREFETCH_START:
            {
                static bool is_in_recent_list = false;
                char enable_prefetch_property[PROPERTY_VALUE_MAX];
                char enable_prefetch_ofr_property[PROPERTY_VALUE_MAX];
                int enable_prefetcher = 0;
                int enable_prefetcher_ofr = 0;

                strlcpy(enable_prefetch_property,perf_get_prop("vendor.enable.prefetch" , "0").value, PROPERTY_VALUE_MAX);
                enable_prefetch_property[PROPERTY_VALUE_MAX-1]='\0';

                enable_prefetcher = strtod(enable_prefetch_property, NULL);

                strlcpy(enable_prefetch_ofr_property,perf_get_prop("vendor.iop.enable_prefetch_ofr" , "0").value, PROPERTY_VALUE_MAX);
                enable_prefetch_ofr_property[PROPERTY_VALUE_MAX-1]='\0';

                enable_prefetcher_ofr = strtod(enable_prefetch_ofr_property, NULL);

                // if PID < 0 consider it as playback operation
                if(msg->pid < 0)
                {
                    int ittr = 0;
                    char *week_day = NULL;
                    week_day = (char *) malloc(6*sizeof(char));
                    // Insert package into the table
                    if (week_day == NULL) {
                       //Malloc failed. Most-probably low on memory.
                       break;
                    }
                    strlcpy(pkg_info.pkg_name,msg->pkg_name,PKG_NAME_LEN);
                    strlcpy(tmp_pkg_name,pkg_info.pkg_name,PKG_NAME_LEN);
                    bindApp_dur = 0;
                    disAct_dur = 0;
                    launching = true;
                    time(&pkg_info.last_time_launched);
                    compute_time_day_slot(week_day, &time_slot);
                    QLOGI("UXEngine Updating Details: pkg_name: %s, week_day: %s, time_slot: %d %s\n", pkg_info.pkg_name, week_day, time_slot, tmp_pkg_name);
                    update_ux_pkg_details(pkg_info, week_day, time_slot, 0);
                    update_palm_table(msg->pkg_name, 0, 1);

                    QLOGI("UXEngine finished ux_pkg_details update \n");
                    free(week_day);
                    is_in_recent_list = false;
                    if(!enable_prefetcher)
                    {
                        QLOGE("io prefetch is disabled");
                        break;
                    }

                    //Check app is in recent list
                    for(ittr = 0; ittr < IOP_NO_RECENT_APP; ittr++)
                    {
                        if(0 == strcmp(msg->pkg_name,recent_list[ittr]))
                        {
                            is_in_recent_list = true;
                            QLOGE("is_in_recent_list is TRUE");
                            break;
                        }
                    }
                    // IF Application is in recent list, return
                    if(true == is_in_recent_list)
                    {
                        QLOGE("io prefetch is deactivate");
                        break;
                    }

                    if(recent_list_cnt == IOP_NO_RECENT_APP)
                        recent_list_cnt = 0;

                    //Copy the package name to recent list
                    strlcpy(recent_list[recent_list_cnt],msg->pkg_name,PKG_LEN);
                    recent_list_cnt++;

                    stop_capture();
                    stop_playback();
                    start_playback(msg->pkg_name);
                }
                // if PID > 0 then consider as capture operation
                if(msg->pid > 0)
                {
                    if(!enable_prefetcher)
                    {
                        QLOGE("io prefetch is disabled");
                        break;
                    }
                    if(true == is_in_recent_list)
                    {
                        QLOGE("io prefetch Capture is deactivated ");
                        break;
                    }
                    stop_capture();
                    start_capture(msg->pid,msg->pkg_name,msg->code_path,enable_prefetcher_ofr);
                }

                break;
            }
    }

pid > 0时，会先检测是否开启了iop 预取，且查看 预取的信息是否在最近的数组中，有则停止采集。接下来回收可能存在的线程，防止内存泄露，且进入start_capture中，创建一个新的线程capture_thread 进行信息采集更新至数据库中。

int start_capture(pid_t pid, char *pkg_name, char *code_path, bool ofr) {
    char property[PROPERTY_VALUE_MAX];
    capture_thread_arg *arg_bundle = NULL;
    int len = 0;
    int len_code_path = 0;

    if(NULL == pkg_name)
        return -1;

    len = strlen(pkg_name);
    if ( len <= 0)
    {
        QLOGW("incorrect length for pkg_name");
        return -1;
    }

    arg_bundle = (capture_thread_arg *)malloc(sizeof(capture_thread_arg));
    if (NULL == arg_bundle)
        return -1;

    total_files = 0;
    QLOGE(" start_capture ENTER ");

    arg_bundle->pid = pid;
    arg_bundle->pkg_name = (char *) malloc(len+1);
    arg_bundle->ofr = ofr;
    len_code_path = strlen(code_path);
    if (NULL == arg_bundle->pkg_name) {
        free(arg_bundle);
        return -1;
    }

    QLOGI("%d %s %zd %d\n", pid, arg_bundle->pkg_name,strlen(pkg_name),PKG_NAME_LEN);
    strlcpy(arg_bundle->pkg_name, pkg_name, len+1);
    strlcpy(list_code_path, code_path, len_code_path+1);
    QLOGI("%d %s %zd %s %zd\n", pid, arg_bundle->pkg_name,strlen(pkg_name),
            list_code_path, strlen(code_path));

    halt_thread = 0;
    start_capture_time = (long) now_ms();

    time(&start_capture_time_s);

    property_get("vendor.iop.read_fd_interval_ms", property, "50");
    read_fd_interval_ms = atoi(property);
    if(read_fd_interval_ms <= 0) {
        QLOGI("read_fd_interval_ms is set to a non-positive value %d. Resetting to default value 50\n", read_fd_interval_ms);
        property_set("read_fd_interval_ms", "50");
        read_fd_interval_ms = 50;
    }
    QLOGI("property = %d\n", read_fd_interval_ms);
    //TBD must remove comments
    if(pthread_create(&capture_pthread_id, NULL, capture_thread, arg_bundle)) {
        return -1;
    }
   QLOGI("start_capture EXIT");
    return 0;
}

在capture_thread中继续读取配置参数，根据参数是否开启某些功能，根据测试红色部分不建议开，会反响优化，使冷启动时间变慢。调用get_priv_code_files和get_priv_files获取提供的有关activity的相关文件放入数组file_list中，然后调用update_pkg_details和get_priv_files更新至数据库中。

void * capture_thread(void * arg) {
    int i = 0;
    int pkg_len = 0;
    int duration_counter = 0;
    pkg_details pkg_info;
    int index = 0;
    int halt_counter = CAPTURE_MAX_DURATION/read_fd_interval_ms;
    int polling_enable = 0;
    char property[PROPERTY_VALUE_MAX];

    property_get("vendor.polling_enable", property, "0");
    polling_enable = atoi(property);
    ATRACE_BEGIN("capture_thread");
    capture_thread_arg * arg_bundle = (capture_thread_arg *)arg;

    //Copy the package to same string to remove ':'
    pkg_len = strlen(arg_bundle->pkg_name);
    i = 0;
    while(i < pkg_len && arg_bundle->pkg_name[i] != ':')
    {
        i++;
    }
    arg_bundle->pkg_name[i] = '\0';
    strlcpy(list_pkg_name,arg_bundle->pkg_name,PKG_NAME_LEN);

    QLOGI("pkg_name  = %s",arg_bundle->pkg_name);

    if(polling_enable)
    {
        while (duration_counter < halt_counter) {
            if (halt_thread) goto cleanup;
            QLOGI("Getting snapshot %d\n", arg_bundle->pid);
            get_snapshot(list_pkg_name,arg_bundle->pid);
            duration_counter++;
            usleep(read_fd_interval_ms * 1000);
        }
    }
    
    get_priv_code_files(arg_bundle->pkg_name);
    get_priv_files(arg_bundle);
    QLOGI("pkg_name = %s total_files = %d ",arg_bundle->pkg_name,total_files);

    // Insert package into the table
    strlcpy(pkg_info.pkg_name,arg_bundle->pkg_name,PKG_NAME_LEN);
    time(&pkg_info.last_time_launched);
    // Update Mincore data
    if(arg_bundle->ofr)
    {
        for(index = 0; index < total_files;index++)
        {
           if(update_mincore_data(file_list[index]) != 0)
           {
               file_list[index]->mincore_array = NULL;
           }
        }
    }
    update_pkg_details(pkg_info);
    update_file_details(arg_bundle->pkg_name, file_list, total_files);
    delete_mark_files();
cleanup:
    //log a report about how many files need insert or update this time
    QLOGE("# Final entry : pkg_name file_name file_time_stamp filesize file_iop_size");
    for(i = 0; i < total_files; i++) {
        QLOGE("%d. Final entry : %s %s %d %d %d\n",i
                ,arg_bundle->pkg_name,file_list[i]->file_name
                , file_list[i]->file_time_stamp, file_list[i]->filesize, file_list[i]->file_iop_size);
        if (file_list[i]->mincore_array) {
            free(file_list[i]->mincore_array);
            file_list[i]->mincore_array = NULL;
        }
        free(file_list[i]);
    }
    ATRACE_END();

    free(arg_bundle->pkg_name);
    free(arg_bundle);
    QLOGI("Exit capture_thread");
    return NULL;
}

接下来看看get_priv_code_files 和 get_priv_files 具体获取了哪些数据？先来看看get_priv_code_files获取了哪些文件。在代码中主要获取了oat中的文件。举个例子看下：

例如.video/IqiyiVideo 目录下的 IqiyiVideo.apk lib oat等内容。capture_pkg_file是一个递归函数，主要获取文件夹里的内容，如*.odex vdex *.apk等文件

static void get_priv_code_files(char *pkg_name) {
    char f_name[FILE_NAME_LEN];
    DIR *dp;

    //oat first
    snprintf(f_name,FILE_NAME_LEN,"%s/oat",list_code_path);
    QLOGI("Dir = %s",f_name);
    dp = opendir(f_name);
    QLOGI("First dp = %p",dp);
    if (dp != NULL) {
        capture_pkg_file(pkg_name, dp, f_name, -1);
        closedir(dp);
    } else {
        QLOGI("dp is NULL");
    }
    //then scan apk file
    dp = opendir(list_code_path);
    QLOGI("First dp = %p",dp);
    if (dp != NULL) {
        capture_pkg_file(pkg_name, dp, list_code_path, 1);
        closedir(dp);
    } else {
        QLOGI("dp is NULL");
    }
}    

接下来看看get_priv_files获取了什么文件？
在代码中主要获取 /data/user/0/pkg_name 以及 /data/data/pkg_name 文件夹下的文件，找个文件看看如下：
第二阶段结果展望：

void get_priv_files(capture_thread_arg * arg_bundle)
{
    char f_name[FILE_NAME_LEN];
    DIR *dp;
    char trace_buf[1024];
    snprintf(trace_buf, sizeof(trace_buf), "Get_prive_file");
    ATRACE_BEGIN(trace_buf);

    snprintf(f_name,FILE_NAME_LEN,"/data/user/0/%s",arg_bundle->pkg_name);
    QLOGI("Dir = %s",f_name);
    dp = opendir(f_name);
    QLOGI("First dp = %p",dp);
    if (dp != NULL) {
        capture_pkg_file(arg_bundle->pkg_name, dp,f_name,2);
        closedir(dp);
    }
    else
    {
        QLOGI("dp is NULL");
    }

    snprintf(f_name,FILE_NAME_LEN,"/data/data/%s",arg_bundle->pkg_name);
    QLOGI("Dir = %s",f_name);
    dp = opendir(f_name);
    QLOGI("First dp = %p",dp);

    if (dp != NULL) {
        capture_pkg_file(arg_bundle->pkg_name, dp,f_name, 4);
        closedir(dp);
    }
    else
    {
        QLOGI("dp is NULL");
    }
    ATRACE_END();
}

接下来看看如何更新到数据库里？
先调用get_package是否已经在数据库里了，如果在给他的调用次数+1，且更新上次启动时间。如果不在则调用IO_PREFETCHER_QUERY_INSERT_PKG sql语句插入数据库中。

//Update package deails with provided attributes
int update_pkg_details(pkg_details pkg_info)
{
    char query_str[2048] = IO_PREFETCHER_QUERY_UPDATE_PKG_DETAILS;
    /* "UPDATE io_pkg_tbl  SET pkg_last_use = %lu ,pkg_use_count = pkg_use_count+1  WHERE pkg_name = '%s'"*/
    pkg_details temp_pkg_info;
    temp_pkg_info.pkg_name[0] = 0;
    int is_file_present  =  get_package(pkg_info.pkg_name, &temp_pkg_info);
    QLOGI("update_pkg_details");

    if(NULL == db_conn)
        open_db();

    if(is_file_present)
    {
        //Update
        snprintf(query_str,sizeof(query_str),IO_PREFETCHER_QUERY_UPDATE_PKG_DETAILS
                                            ,pkg_info.last_time_launched
                                            ,pkg_info.pkg_name);
        QLOGI("\nQuery = %s \n",query_str);
        iop_query_exec(query_str);
    }
    else
    {
        //Insert
        snprintf(query_str,sizeof(query_str),IO_PREFETCHER_QUERY_INSERT_PKG
                                            ,pkg_info.pkg_name
                                            ,pkg_info.last_time_launched
                                            );

        QLOGI("\nQuery = %s \n",query_str);
        iop_query_exec(query_str);
    }
    return 0;
}

这里通过get_file 获取io_pkg_file_tbl表的信息,更新到io_pkg_file_tbl表中。这里数据的更新使用了sql的事务来更新到数据库中。

// update detail for file with provided attibutes
int update_file_details(char * pkg_name,file_details *file_info[], int size)
{
    int i = 0;
    char *error=NULL;
    sqlite3_stmt *stmt=NULL;
    char trace_buf[1024];
    char query_str[2048] = IO_PREFETCHER_QUERY_UPDATE_FILE_DETAILS;
    iop_query_exec("BEGIN TRANSACTION;");
    for(i=0;i<size;i++) {
        file_details temp_file_info;
        temp_file_info.file_name[0]=0;
        int is_file_present  = get_file(pkg_name, file_info[i]->file_name, &temp_file_info);

        if(is_file_present) {
            //Update
            QLOGI("MINCORE update file %s %s study_finish %d, cache_dropped %d",
                    pkg_name, temp_file_info.file_name,
                    temp_file_info.study_finish, temp_file_info.cache_dropped);
            /*
             * update mincore array data if this file cache is dropped and study not finished
             */
            if (temp_file_info.cache_dropped == 1 && temp_file_info.study_finish <= 2) {
                snprintf(trace_buf, sizeof(trace_buf), "update file %s", file_info[i]->file_name);
                ATRACE_BEGIN(trace_buf);
                snprintf(query_str,sizeof(query_str),IO_PREFETCHER_QUERY_UPDATE_FILE_DETAILS
                                                    ,file_info[i]->file_time_stamp
                                                    ,file_info[i]->filesize
                                                    ,file_info[i]->file_iop_size
                                                    ,pkg_name
                                                    ,file_info[i]->file_name);
                QLOGI("\nQuery = %s \n",query_str);
                int pages_in_range = (file_info[i]->filesize + PAGE_SIZE - 1)/PAGE_SIZE;
                sem_wait(&mutex);
                _SQLITE_CHECK_OK(sqlite3_prepare_v2(db_conn, query_str,
                            strlen(query_str), &stmt, (const char **)&error));

                int index = sqlite3_bind_parameter_index(stmt, ":mincore_array");
                if (file_info[i]->mincore_array) {
                    _SQLITE_CHECK_OK(sqlite3_bind_blob(stmt, index,
                                file_info[i]->mincore_array, pages_in_range, SQLITE_STATIC));
                }
                else
                {
                    _SQLITE_CHECK_OK(sqlite3_bind_blob(stmt, index, NULL, 0, SQLITE_STATIC));
                }
                _SQLITE_CHECK_DONE(sqlite3_step(stmt));
                sqlite3_reset(stmt);
                sqlite3_finalize(stmt);
                sem_post(&mutex);
                ATRACE_END();
            }
            else
            {
                if (file_info[i]->cache_dropped) {
                    mark_cache_dropped(pkg_name, file_info[i]->file_name, 0);
                }
            }
        }
        else
        {
            //insert new file to db
            int index;
            int pagesize = sysconf(_SC_PAGESIZE);
            QLOGI("MINCORE insert new file %s %s iopsize %d filesize %d",
                                                  pkg_name, file_info[i]->file_name,
                                                  file_info[i]->file_iop_size, file_info[i]->filesize);
            snprintf(trace_buf, sizeof(trace_buf), "insert new file %s", file_info[i]->file_name);
            ATRACE_BEGIN(trace_buf);

            snprintf(query_str,sizeof(query_str),IO_PREFETCHER_QUERY_INSER_FILE
                                                ,pkg_name,file_info[i]->file_name
                                                ,file_info[i]->file_time_stamp
                                                ,file_info[i]->filesize
                                                ,file_info[i]->file_modify_time
                                                ,file_info[i]->file_iop_size
                                                );
            QLOGI("\nQuery = %s \n",query_str);
            sem_wait(&mutex);
            _SQLITE_CHECK_OK(sqlite3_prepare_v2(db_conn, query_str,
                        strlen(query_str), &stmt, (const char**)&error));
            if (file_info[i]->mincore_array != NULL && file_info[i]->file_iop_size >pagesize) {
                index = sqlite3_bind_parameter_index(stmt, ":mincore_array");
                int pages_in_range = (file_info[i]->filesize + PAGE_SIZE - 1)/PAGE_SIZE;
                _SQLITE_CHECK_OK(sqlite3_bind_blob(stmt, index,
                            file_info[i]->mincore_array, pages_in_range, SQLITE_STATIC));
            }
            _SQLITE_CHECK_DONE(sqlite3_step(stmt));
            sqlite3_reset(stmt);
            sqlite3_finalize(stmt);
            sem_post(&mutex);
            ATRACE_END();
        }
    }
    iop_query_exec("COMMIT TRANSACTION;");
     return 0;
}

此时采集信息的过程已经完成了，接下来应该是执行预取优化的过程了，可以回到iop_server 流程中的pid < 0的步骤，继续向下分析。黄色得部分主要是收集activity启动得时间信息，主要包括信息是工作日，上一次启动时间等信息，用于预测，如果预测成功权重+weight，预测失败，失败的权重+weight等信息，目前没有用到预测功能，先忽略。绿色的部分主要是看新启动的app是否是在最近启动的列表里，如果存在则不执行预取。如果都没问题则执行浅红色部分，执行获取数据库中的信息提前执行预取操作。ng)

if(msg->pid < 0)
                {
                    int ittr = 0;
                    char *week_day = NULL;
                    week_day = (char *) malloc(6*sizeof(char));
                    // Insert package into the table
                    if (week_day == NULL) {
                       //Malloc failed. Most-probably low on memory.
                       break;
                    }
                    strlcpy(pkg_info.pkg_name,msg->pkg_name,PKG_NAME_LEN);
                    strlcpy(tmp_pkg_name,pkg_info.pkg_name,PKG_NAME_LEN);
                    bindApp_dur = 0;
                    disAct_dur = 0;
                    launching = true;
                    time(&pkg_info.last_time_launched);
                    compute_time_day_slot(week_day, &time_slot);
                    QLOGI("UXEngine Updating Details: pkg_name: %s, week_day: %s, time_slot: %d %s\n", pkg_info.pkg_name, week_day, time_slot, tmp_pkg_name);
                    update_ux_pkg_details(pkg_info, week_day, time_slot, 0);
                    update_palm_table(msg->pkg_name, 0, 1);

                    QLOGI("UXEngine finished ux_pkg_details update \n");
                    free(week_day);
                    is_in_recent_list = false;
                    if(!enable_prefetcher)
                    {
                        QLOGE("io prefetch is disabled");
                        break;
                    }

                    //Check app is in recent list
                    for(ittr = 0; ittr < IOP_NO_RECENT_APP; ittr++)
                    {
                        if(0 == strcmp(msg->pkg_name,recent_list[ittr]))
                        {
                            is_in_recent_list = true;
                            QLOGE("is_in_recent_list is TRUE");
                            break;
                        }
                    }
                    // IF Application is in recent list, return
                    if(true == is_in_recent_list)
                    {
                        QLOGE("io prefetch is deactivate");
                        break;
                    }

                    if(recent_list_cnt == IOP_NO_RECENT_APP)
                        recent_list_cnt = 0;

                    //Copy the package name to recent list
                    strlcpy(recent_list[recent_list_cnt],msg->pkg_name,PKG_LEN);
                    recent_list_cnt++;

                    stop_capture();
                    stop_playback();
                    start_playback(msg->pkg_name);
                }

Iop预取优化流程
接下来我们具体看看他是如何从数据库中获取信息，然后执行iop预取的操作的。在start_playback中开一个线程去执行。

void start_playback(char *pkg_name_arg) {
    char *pkg_name = NULL;
    int len = 0;
    QLOGE(" start_playback-1 ");

    if(NULL == pkg_name_arg)
        return;

    len = strlen(pkg_name_arg);
    if(len <= 0)
    {
        QLOGW("Incorrect length for pkg_name_arg");
        return;
    }

    pkg_name = (char *) malloc(len + 1);
    if (NULL == pkg_name)
        return;

    strlcpy(pkg_name, pkg_name_arg,len+1);

    QLOGI(" %s %zd\n", pkg_name,strlen(pkg_name));

    if(pthread_create(&start_playback_pthread_id, NULL, start_playback_thread, pkg_name)) {
         return ;
    }
    QLOGI("Exit playback");
    return ;
}

接下来是Iop的核心操作。get_total_file(pkg_name)这个操作是检查数据库中是否有数据，如果没有，则没有预取操作了。如果num_file>0，则表示有数据，则调用get_file_list，获取pkg_name 对应的文件信息存储到file_detail_ptr中，接下来的一对操作主要是进行数据的标记，是否要清理缓存，根据数据标签把一些不确定的因素去掉。此时已经获取到了预取信息到file_detail_ptr，调用iop_fileplayback_run(file_detail_ptr)进行预取，最后调用 commit_filelist_info （）将更新的信息更新到数据库里。

static void* start_playback_thread(void *pkg_name_arg)
{
    file_details *file_detail_ptr;
    int num_file = 0;
    int num_of_files = 0;
    char *pkg_name = (char *)pkg_name_arg;
    int i = 0;
    total_data = 0;
    struct stat file_stat;
    int pagesize = sysconf(_SC_PAGESIZE);
    ATRACE_BEGIN("start_playback_thread: Enter");

    QLOGI("pkg_name = %s",pkg_name);
    num_file = get_total_file(pkg_name);

    if( num_file <= 0)
    {
        QLOGI("no file to read get_total_file %d", num_file);
        remove_pkg(pkg_name);
        free(pkg_name);
        ATRACE_END();
        return NULL;
    }

    file_detail_ptr = (file_details *) malloc(sizeof(file_details) * num_file);
    if (NULL == file_detail_ptr)
    {
        QLOGI("Fail to allocate memory");
        remove_pkg(pkg_name);
        free(pkg_name);
        ATRACE_END();
        return NULL;
    }

    num_of_files = get_file_list(pkg_name,file_detail_ptr,num_file);
    QLOGI("num_of_files = %d",num_of_files);

    if (num_of_files <= 0)
    {
        QLOGI("no file to read get_file_list");
        remove_pkg(pkg_name);
        free(file_detail_ptr);
        free(pkg_name);
        ATRACE_END();
        return NULL;
    }

    if( num_file > MAX_NUM_FILE)
    {
        //free mincore array that exceed MAX_NUM_FILE
        for (i=MAX_NUM_FILE; i<num_of_files; i++) {
            if (file_detail_ptr[i].mincore_array)
            {
                free(file_detail_ptr[i].mincore_array);
                file_detail_ptr[i].mincore_array=NULL;
            }
        }
        num_file = MAX_NUM_FILE;
    }

    for(i = 0; i < num_file;i++)
    {
        int fd = -1;
        char trace_buf[4096];
        if (file_detail_ptr[i].disabled) {
            QLOGE("skip preload disabled file %d %s", i, file_detail_ptr[i].file_name);
            continue;
        }
        if (file_detail_ptr[i].study_finish && file_detail_ptr[i].file_iop_size < pagesize) {
            file_detail_ptr[i].cache_dropped = NEED_MARK_CACHE_NOT_DROPPED;
            QLOGE("skip preload ZERO needed file %d %s", i, file_detail_ptr[i].file_name);
            continue;
        }
        fd = open(file_detail_ptr[i].file_name, O_RDONLY);
        if(fd == -1)
        {
            mark_for_delete(pkg_name,file_detail_ptr[i].file_name);
            continue;
        }
        if ( fstat( fd, &file_stat ) < 0 )
        {
            QLOGI("fail to get file stat");
            mark_for_delete(pkg_name,file_detail_ptr[i].file_name);
            close(fd);
            continue;// goto  next file
        }
        if ( file_stat.st_size == 0 )
        {
            // File zie is zero
            mark_for_delete(pkg_name,file_detail_ptr[i].file_name);
            close(fd);
            continue;// goto  next file
        }
        snprintf(trace_buf,4096,"Opening file %s size = %d",file_detail_ptr[i].file_name
                               , (int)file_stat.st_size);
        ATRACE_BEGIN(trace_buf);
        if (file_stat.st_mtime != file_detail_ptr[i].file_modify_time) {
            if (strstr(file_detail_ptr[i].file_name,".apk")!=NULL
                    && (strstr(file_detail_ptr[i].file_name,"/data/app/")!=NULL ||
                    strstr(file_detail_ptr[i].file_name, "/system/app/")!=NULL ||
                    strstr(file_detail_ptr[i].file_name, "/system/priv-app/")!=NULL))
                {
                    //readonly /data/app/base.apk
                    QLOGI("apk file %s modify time changed %d/%ld, \
                              remove files belong to pkg, and restudy",
                              file_detail_ptr[i].file_name,
                              file_detail_ptr[i].file_modify_time, file_stat.st_mtime);
                    snprintf(trace_buf, sizeof(trace_buf), "apk file %s upgrade, remove pkg"
                                                         , file_detail_ptr[i].file_name);
                    ATRACE_BEGIN(trace_buf);
                    file_detail_ptr[i].mark_for_delete = NEED_MARK_FOR_DELETE_PKG;
                    close(fd);
                    ATRACE_END();

                    ATRACE_END(); // end for opening file
                    ATRACE_END(); //end for start_playback_thread enter
                    return NULL;
                } else {
                    QLOGI("file %s modify time changed update db", file_detail_ptr[i].file_name);
                    //mark study_finish=0 and cache_dropped=1
                    snprintf(trace_buf, sizeof(trace_buf), "file %s modifie mark update"
                                                         , file_detail_ptr[i].file_name);
                    ATRACE_BEGIN(trace_buf);
                    //restudy if file modify time changed for .odex and .dex file
                    if (strstr(file_detail_ptr[i].file_name, ".odex") != NULL
                        || strstr(file_detail_ptr[i].file_name, ".dex") != NULL
                        || strstr(file_detail_ptr[i].file_name, ".vdex") != NULL)
                    {
                        file_detail_ptr[i].mark_for_delete = NEED_MARK_FOR_UPDATE;
                        file_detail_ptr[i].file_modify_time = file_stat.st_mtime;
                        file_detail_ptr[i].filesize = file_stat.st_size;
                        drop_file_cache(fd, &(file_detail_ptr[i]));
                    } else {//disable unstable file
                        file_detail_ptr[i].mark_for_delete = NEED_MARK_FOR_DISABLE;
                        file_detail_ptr[i].disabled = 1;
                    }
                ATRACE_END();
            }
            close(fd);
            continue;
        }
        //try if need drop cache or playback
        if (file_detail_ptr[i].study_finish == 0) {
            QLOGI("Drop cache of file %s", file_detail_ptr[i].file_name);
            snprintf(trace_buf, sizeof(trace_buf), "Drop cache of file %s",
                    file_detail_ptr[i].file_name);
            ATRACE_BEGIN(trace_buf);
            drop_file_cache(fd, &(file_detail_ptr[i]));
            file_detail_ptr[i].cache_dropped = NEED_MARK_CACHE_DROPPED;
            ATRACE_END();
        } else {
            QLOGI("io preload of file %s", file_detail_ptr[i].file_name);
            snprintf(trace_buf, sizeof(trace_buf), "add file %s %.2f / %.2f MB to task",
                    file_detail_ptr[i].file_name,
                    file_detail_ptr[i].file_iop_size/1024.0/1024,
                    file_detail_ptr[i].filesize/1024.0/1024);
            ATRACE_BEGIN(trace_buf);
            iop_fileplayback_run(file_detail_ptr[i],file_stat,fd);
            ATRACE_END();
            if (file_detail_ptr[i].cache_dropped) {
                file_detail_ptr[i].cache_dropped = NEED_MARK_CACHE_NOT_DROPPED;
            }
        }
        close(fd);
        if(total_data > MAX_TOTAL_DATA)
        {
            QLOGI("Max total size limit reached Total = %d",total_data);
            //STOP reading and max total reached
            break;
        }
        ATRACE_END(); // end for opening file
    }
    commit_filelist_info(file_detail_ptr, num_file);
    delete_mark_files();

    ATRACE_END(); //end for start_playback_thread enter
    QLOGI(" Total Data = %.2f MB", total_data/1024.0/1024);
    for (i=0; i<num_of_files; i++) {
        if (file_detail_ptr[i].mincore_array) {
            free(file_detail_ptr[i].mincore_array);
            file_detail_ptr[i].mincore_array=NULL;
        }
    }
    free(pkg_name);
    free(file_detail_ptr);
    return NULL;
}

下面来看看iop_fileplayback_run具体执行了什么操作？
在这里主要是通过posix_fadvise 系统调用 + POSIX_FADV_WILLNEED 参数执行了预读操作，将需要加载的文件提前读到了pagecache里，使启动时读取该部分文件能直接命中缓存加速启动速度。具体可以参照posix_fadvise的作用 linux系统函数posix_fadvise_九品神元师的博客-CSDN博客。

/* Run playback operation */
static void *iop_fileplayback_run(file_details info, struct stat file_stat, int fd)
{
    int page_itrr;
    int total_mincore_pages = (info.file_iop_size+PAGE_SIZE-1)/PAGE_SIZE;
    int total_pages = (file_stat.st_size+PAGE_SIZE-1)/PAGE_SIZE;
    char *mincore_array = info.mincore_array;
    char trace_buf[1024];
    if (total_mincore_pages > 0) {
        snprintf(trace_buf, sizeof(trace_buf), "%s read size:%.4fMB",
                                             info.file_name,
                                             total_mincore_pages*4/1024.0);
    }
    else
    {
        snprintf(trace_buf, sizeof(trace_buf), "%s read size:%.4fMB",
                    info.file_name,file_stat.st_size/1024.0/1024);
    }
    ATRACE_BEGIN(trace_buf);
    int count_pages=0;
    int read_data = 0;
    int ret = 0;
    QLOGI("MINCORE, PLAYBACK %s START, mincore_array: %p",
                info.file_name, mincore_array);
    //load each page by posix_fadvise here to avoid read_ahead
    //and decrease CPU usage of copy io content result to local viriable
    if (mincore_array == NULL) {
        //if all file need read to cache
        int len_size = file_stat.st_size;
        ret = posix_fadvise(fd, 0, len_size, POSIX_FADV_WILLNEED);
        read_data += len_size;
        if (ret !=0) {
            QLOGE("posix_fadvise failed: %s", strerror(errno));
        }
    } else {//load file content based on mincore result
        count_pages=0;
        for(page_itrr = 0; page_itrr < total_pages; page_itrr++) {
            if (mincore_array[page_itrr] & 0x1) {
                if (count_pages>=total_mincore_pages)
                    break;
                ret = posix_fadvise(fd, page_itrr*PAGE_SIZE, PAGE_SIZE, POSIX_FADV_WILLNEED);
                if (ret !=0) {
                    QLOGE("posix_fadvise failed: %s", strerror(errno));
                }
                read_data += PAGE_SIZE;
                count_pages++;
            }
        }
    }
    //after finish preload, mark file cache advise as normal
    ret = posix_fadvise(fd, 0, file_stat.st_size, POSIX_FADV_NORMAL);
    if (ret !=0) {
        QLOGE("posix_fadvise failed: %s", strerror(errno));
    }
    close(fd);
    total_data += read_data;
    QLOGI("MINCORE, PLAYBACK %s END real read file size = %.4fMB",
                                      info.file_name, read_data/1024.0/1024);
    ATRACE_END();

    return NULL;
}

学前疑问解答

1. Iop是如何怎么预取的？
   主要通过posix_fadvise 这个系统调用配合相应的参数将所需要的文件预取到PageCache里。
2. 数据库的表是如何存放数据的？
   数据库中的数据主要是从codepath，和 遍历pid 中的fd所指向的链接的信息进行筛选存入数据库的表中，iop用到的表主要存app的.apk，vdex，.so等文件，并记录被使用的时间，和使用到的次数。

预期日期	截止日期	结果
2022/11/14	2022/11/25	理解Iop底层机制，输出一篇关于Iop底层原理的文档

第三阶段

第三阶段属于测试阶段，在这个阶段，我应该已经成功启动了Iop机制，且已经掌握了90%的Iop原理，可以比较灵活的使用BoostFrame了。这阶段的主要目标是测试在Iop开启的条件下，App冷启动和热启动时间的效率比，当然测试的话还是使用自动化脚本比较方便，第三阶段的主要目标是输出一个相关脚本，并输出一篇性能分析文档。

第三阶段结果展望：

预期日期	截止日期	结果
2022/11/24	2022/12/5	输出测试文档

集成之后结果不如人意，从原理上讲应该可以优化，但集成之后并没有优化。心累…