apk加载32bit 64bit

apk lib目录:

先看下apk中的lib打包的目录：

依次代表不同类型的cpu

PMS安装路径：

pms install 流程比较繁杂，只关注so相关的scanPackageDirtyLI函数中：

private PackageParser.Package scanPackageDirtyLI(PackageParser.Package pkg, int parseFlags,
            int scanFlags, long currentTime, UserHandle user) throws PackageManagerException {
            ...
             //invoke installer to do the actual installation  //作为外部apk 创建data目录相关项
             //\frameworks\native\cmds\installd\commands.c  install()中创建
                int ret = **createDataDirsLI**(pkgName, pkg.applicationInfo.uid,
                                           pkg.applicationInfo.seinfo);
            ...
    if (isSystemApp(pkg) && !isUpdatedSystemApp(pkg)) {
        ...

           setBundledAppAbisAndRoots(pkg, pkgSetting);

            ...

           setNativeLibraryPaths(pkg);  

    }
    else
    {
    setNativeLibraryPaths(pkg);
    ...
        if (isAsec) {
                        copyRet = NativeLibraryHelper.findSupportedAbi(handle, abiList);
                    } else {
                        copyRet = NativeLibraryHelper.copyNativeBinariesForSupportedAbi(handle,
                                nativeLibraryRoot, abiList, useIsaSpecificSubdirs);
                    }
    setNativeLibraryPaths(pkg);
      if (DEBUG_INSTALL) Slog.i(TAG, "Linking native library dir for " + path);
            final int[] userIds = sUserManager.getUserIds();
            synchronized (mInstallLock) {
                // Create a native library symlink only if we have native libraries
                // and if the native libraries are 32 bit libraries. We do not provide
                // this symlink for 64 bit libraries.
                if (pkg.applicationInfo.primaryCpuAbi != null &&
                        **!VMRuntime.is64BitAbi(pkg.applicationInfo.primaryCpuAbi)**) {
                    final String nativeLibPath = pkg.applicationInfo.nativeLibraryDir;
                    for (int userId : userIds) {
if (mInstaller.linkNativeLibraryDirectory(pkg.packageName, nativeLibPath, userId) < 0)    {
                            throw new PackageManagerException(INSTALL_FAILED_INTERNAL_ERROR,
                                    "Failed linking native library dir (user=" + userId + ")");
                        }
                    }
                }
            }

    }
}

看下system app 的安装配置函数 setBundledAppAbisAndRoots：

    /**
     * Calculate the abis and roots for a bundled app. These can uniquely
     * be determined from the contents of the system partition, i.e whether
     * it contains 64 or 32 bit shared libraries etc. We do not validate any
     * of this information, and instead assume that the system was built
     * sensibly.
     */
    private void setBundledAppAbisAndRoots(PackageParser.Package pkg,
                                           PackageSetting pkgSetting) {
        final String apkName = deriveCodePathName(pkg.applicationInfo.getCodePath());

        // If "/system/lib64/apkname" exists, assume that is the per-package
        // native library directory to use; otherwise use "/system/lib/apkname".
        final String apkRoot = calculateBundledApkRoot(pkg.applicationInfo.sourceDir);
        setBundledAppAbi(pkg, apkRoot, apkName);
        // pkgSetting might be null during rescan following uninstall of updates
        // to a bundled app, so accommodate that possibility.  The settings in
        // that case will be established later from the parsed package.
        //
        // If the settings aren't null, sync them up with what we've just derived.
        // note that apkRoot isn't stored in the package settings.
        if (pkgSetting != null) {
            pkgSetting.primaryCpuAbiString = pkg.applicationInfo.primaryCpuAbi;
            pkgSetting.secondaryCpuAbiString = pkg.applicationInfo.secondaryCpuAbi;
        }
    }

主要是在setBundledAppAbi中：

    /**
     * Deduces the ABI of a bundled app and sets the relevant fields on the
     * parsed pkg object.
     *
     * @param apkRoot the root of the installed apk, something like {@code /system} or {@code /oem}
     *        under which system libraries are installed.
     * @param apkName the name of the installed package.
     */
    private static void setBundledAppAbi(PackageParser.Package pkg, String apkRoot, String apkName) {
        final File codeFile = new File(pkg.codePath);
...
        if (has64BitLibs && !has32BitLibs) {
            // The package has 64 bit libs, but not 32 bit libs. Its primary
            // ABI should be 64 bit. We can safely assume here that the bundled
            // native libraries correspond to the most preferred ABI in the list.

            pkg.applicationInfo.primaryCpuAbi = Build.SUPPORTED_64_BIT_ABIS[0];
            pkg.applicationInfo.secondaryCpuAbi = null;
        } else if (has32BitLibs && !has64BitLibs) {
            // The package has 32 bit libs but not 64 bit libs. Its primary
            // ABI should be 32 bit.

            pkg.applicationInfo.primaryCpuAbi = Build.SUPPORTED_32_BIT_ABIS[0];
            pkg.applicationInfo.secondaryCpuAbi = null;
        } else if (has32BitLibs && has64BitLibs) {
            // The application has both 64 and 32 bit bundled libraries. We check
            // here that the app declares multiArch support, and warn if it doesn't.
            //
            // We will be lenient here and record both ABIs. The primary will be the
            // ABI that's higher on the list, i.e, a device that's configured to prefer
            // 64 bit apps will see a 64 bit primary ABI,

            if ((pkg.applicationInfo.flags & ApplicationInfo.FLAG_MULTIARCH) == 0) {
                Slog.e(TAG, "Package: " + pkg + " has multiple bundled libs, but is not multiarch.");
            }

            if (VMRuntime.is64BitInstructionSet(getPreferredInstructionSet())) {
                pkg.applicationInfo.primaryCpuAbi = Build.SUPPORTED_64_BIT_ABIS[0];
                pkg.applicationInfo.secondaryCpuAbi = Build.SUPPORTED_32_BIT_ABIS[0];
            } else {
                pkg.applicationInfo.primaryCpuAbi = Build.SUPPORTED_32_BIT_ABIS[0];
                pkg.applicationInfo.secondaryCpuAbi = Build.SUPPORTED_64_BIT_ABIS[0];
            }
        } else {
            pkg.applicationInfo.primaryCpuAbi = null;
            pkg.applicationInfo.secondaryCpuAbi = null;
        }
 }

根据file 查找 确定primaryCpuAbi secondaryCpuAbi 变量值，这个也就决定了 这个 apk 由64bit 还是32bit 的zygote去fork 还有nativelibrary 查找的path

其中nativelibrary的几个主要函数 setNativeLibraryPaths：

/**
     * Derive and set the location of native libraries for the given package,
     * which varies depending on where and how the package was installed.
     */
    private void setNativeLibraryPaths(PackageParser.Package pkg) {
        final ApplicationInfo info = pkg.applicationInfo;
        final String codePath = pkg.codePath;
        final File codeFile = new File(codePath);
        final boolean bundledApp = isSystemApp(info) && !isUpdatedSystemApp(info);
        final boolean asecApp = isForwardLocked(info) || isExternal(info);

        info.nativeLibraryRootDir = null;
        info.nativeLibraryRootRequiresIsa = false;
        info.nativeLibraryDir = null;
        info.secondaryNativeLibraryDir = null;

        if (isApkFile(codeFile)) {
            // Monolithic install
            if (bundledApp) {
                // If "/system/lib64/apkname" exists, assume that is the per-package
                // native library directory to use; otherwise use "/system/lib/apkname".
                final String apkRoot = calculateBundledApkRoot(info.sourceDir);
                final boolean is64Bit = VMRuntime.is64BitInstructionSet(
                        getPrimaryInstructionSet(info));

                // This is a bundled system app so choose the path based on the ABI.
                // if it's a 64 bit abi, use lib64 otherwise use lib32. Note that this
                // is just the default path.
                final String apkName = deriveCodePathName(codePath);
                final String libDir = is64Bit ? LIB64_DIR_NAME : LIB_DIR_NAME;
                info.nativeLibraryRootDir = Environment.buildPath(new File(apkRoot), libDir,
                        apkName).getAbsolutePath();

                if (info.secondaryCpuAbi != null) {
                    final String secondaryLibDir = is64Bit ? LIB_DIR_NAME : LIB64_DIR_NAME;
                    info.secondaryNativeLibraryDir = Environment.buildPath(new File(apkRoot),
                            secondaryLibDir, apkName).getAbsolutePath();
                }
            } else if (asecApp) {
                info.nativeLibraryRootDir = new File(codeFile.getParentFile(), LIB_DIR_NAME)
                        .getAbsolutePath();
            } else {
                final String apkName = deriveCodePathName(codePath);
                info.nativeLibraryRootDir = new File(mAppLib32InstallDir, apkName)
                        .getAbsolutePath();
            }

            info.nativeLibraryRootRequiresIsa = false;
            info.nativeLibraryDir = info.nativeLibraryRootDir;
        } else {
            // Cluster install
            info.nativeLibraryRootDir = new File(codeFile, LIB_DIR_NAME).getAbsolutePath();
            info.nativeLibraryRootRequiresIsa = true;

            info.nativeLibraryDir = new File(info.nativeLibraryRootDir,
                    getPrimaryInstructionSet(info)).getAbsolutePath();

            if (info.secondaryCpuAbi != null) {
                info.secondaryNativeLibraryDir = new File(info.nativeLibraryRootDir,
                        VMRuntime.getInstructionSet(info.secondaryCpuAbi)).getAbsolutePath();
            }
        }
    }
根据pkg application info    来确定nativelibrarydir   依赖info中的 info.primaryCpuAbi

    private static String getPrimaryInstructionSet(ApplicationInfo info) {
        if (info.primaryCpuAbi == null) {
            return getPreferredInstructionSet();
        }

        return VMRuntime.getInstructionSet(info.primaryCpuAbi);
    }

非system apk 会调用 NativeLibraryHelper 的 findSupportedAbi 去解析 .apk 文件，根据系统suportabilist 去查找 lib目录下的打包子目录 找到匹配的abi

向文章开头的那个 lib目录 ，在64bit 机器上suportabilist为：

 public static final String[] SUPPORTED_ABIS = getStringList("ro.product.cpu.abilist", ",");

root@:/ # getprop ro.product.cpu.abilist                                 
arm64-v8a,armeabi-v7a,armeabi
会匹配arm64-v8a    赋值给 info.primaryCpuAbi

copyNativeBinariesForSupportedAbi 会去copy 前面匹配的lib 目录到本地

最后设置NativeLibraryPaths ，
如果匹配的是64bit的，也就是arm64-v8a 那么就不为/data/data/../lib 建立软链接，这是与32bit 不同的地方

---

system.loadlibrary

作为动态库加载的标准接口，直接看实现：

    public static void loadLibrary(String libName) {
        Runtime.getRuntime().loadLibrary(libName, VMStack.getCallingClassLoader());
    }

到Runtime.Java中：

/*
     * Searches for and loads the given shared library using the given ClassLoader.
     */
    void loadLibrary(String libraryName, ClassLoader loader) {
        if (loader != null) {
            String filename = loader.findLibrary(libraryName);
            if (filename == null) {
                // It's not necessarily true that the ClassLoader used
                // System.mapLibraryName, but the default setup does, and it's
                // misleading to say we didn't find "libMyLibrary.so" when we
                // actually searched for "liblibMyLibrary.so.so".
                throw new UnsatisfiedLinkError(loader + " couldn't find \"" +
                                               System.mapLibraryName(libraryName) + "\"");
            }
            String error = doLoad(filename, loader);
            if (error != null) {
                throw new UnsatisfiedLinkError(error);
            }
            return;
        }

        String filename = System.mapLibraryName(libraryName);
        List<String> candidates = new ArrayList<String>();
        String lastError = null;
        for (String directory : mLibPaths) {
            String candidate = directory + filename;
            candidates.add(candidate);

            if (IoUtils.canOpenReadOnly(candidate)) {
                String error = doLoad(candidate, loader);
                if (error == null) {
                    return; // We successfully loaded the library. Job done.
                }
                lastError = error;
            }
        }

        if (lastError != null) {
            throw new UnsatisfiedLinkError(lastError);
        }
        throw new UnsatisfiedLinkError("Library " + libraryName + " not found; tried " + candidates);
    }

这里的 ClassLoader loader 实际上会在 apk启动的时候 初始化好一些相关的 子类 父类 还有参数

大体记录一下 启动时 初始流程 ：

ActivityThread.java   - handleBindApplication  
 final ContextImpl appContext = ContextImpl.createAppContext(this, data.info);
  LoadedApk pi = getPackageInfo(instrApp, data.compatInfo,
                    appContext.getClassLoader(), false, true, false);


ContextImpl.java  -getClassLoader()
LoadedApk.java    -getClassLoader()     ：  mLibDir = aInfo.nativeLibraryDir;
mClassLoader = ApplicationLoaders.getDefault().getClassLoader(zip, lib,
                        mBaseClassLoader);
ApplicationLoaders.java    -getClassLoader(...)
 PathClassLoader pathClassloader = new PathClassLoader(zip, libPath, parent);   //这里的libPath 就是上面传下来的aInfo.nativeLibraryDir

public class PathClassLoader extends BaseDexClassLoader
public class BaseDexClassLoader extends ClassLoader

loader.findLibrary(libraryName);

    /**
     * Finds the named native code library on any of the library
     * directories pointed at by this instance. This will find the
     * one in the earliest listed directory, ignoring any that are not
     * readable regular files.
     *
     * @return the complete path to the library or {@code null} if no
     * library was found
     */
    public String findLibrary(String libraryName) {
        String fileName = System.mapLibraryName(libraryName);
        for (File directory : nativeLibraryDirectories) {
            String path = new File(directory, fileName).getPath();
            if (IoUtils.canOpenReadOnly(path)) {
                return path;
            }
        }
        return null;
    }
 
 
15

这里的nativeLibraryDirectories 即为前面一系列 构造时 设置了值 其中就有 aInfo.nativeLibraryDir

后面的逻辑就不去叙述了， 根据名字在这个目录下去找 ，然后调用到本地JNI 最终调用 dlopen 加载打开so，必须是相同位数， 而这个关乎当前进程是属于64bit 还是 32bit，这个会在zygote fork时区分， 同样也是由PMS解析时得到的 info.primaryCpuAbi

AMS 请求zygote fork app process选择

只关心 相关代码 startProcessLocked函数：

    private final void startProcessLocked(ProcessRecord app, String hostingType,
            String hostingNameStr, String abiOverride, String entryPoint, String[] entryPointArgs) {
...

 String requiredAbi = (abiOverride != null) ? abiOverride : app.info.primaryCpuAbi;
            if (requiredAbi == null) {
                requiredAbi = Build.SUPPORTED_ABIS[0];
            }

            String instructionSet = null;
            if (app.info.primaryCpuAbi != null) {
                instructionSet = VMRuntime.getInstructionSet(app.info.primaryCpuAbi);
            }

            app.gids = gids;
            app.requiredAbi = requiredAbi;
            app.instructionSet = instructionSet;

            // Start the process.  It will either succeed and return a result containing
            // the PID of the new process, or else throw a RuntimeException.
            boolean isActivityProcess = (entryPoint == null);
            if (entryPoint == null) entryPoint = "android.app.ActivityThread";
            checkTime(startTime, "startProcess: asking zygote to start proc");
            Process.ProcessStartResult startResult = Process.start(entryPoint,
                    app.processName, uid, uid, gids, debugFlags, mountExternal,
                    app.info.targetSdkVersion, app.info.seinfo, requiredAbi, instructionSet,
                    app.info.dataDir, entryPointArgs);
            checkTime(startTime, "startProcess: returned from zygote!");

...
}

Process中真正的socket 请求实现：

return zygoteSendArgsAndGetResult(openZygoteSocketIfNeeded(abi), argsForZygote);

openZygoteSocketIfNeeded 会根据传下来的abi 去选择 通信的socket

而在64bit 机器上，启动时会 启动 两个 zygote service ，用于接收 64 32 的apk 请求：

service zygote /system/bin/app_process64 -Xzygote /system/bin --zygote --start-system-server --socket-name=zygote
    class main
    socket zygote stream 660 root system
    onrestart write /sys/android_power/request_state wake
    onrestart write /sys/power/state on
    onrestart restart media
    onrestart restart netd

service zygote_secondary /system/bin/app_process32 -Xzygote /system/bin --zygote --socket-name=zygote_secondary
    class main
    socket zygote_secondary stream 660 root system
    onrestart restart zygote

可以看到两个 zygote 进程，基本一致 ，区别在于 64bit 32bit ，注册socket不同
关于这两个 zygote 进程启动时的socket 注册 就不多说了

小记录

回到正题， 在 64bit 主控上面， 进程是如何确定 native 库是用 32bit 还是 64bit ?
比如 mediaserver， Android.mk 中就是 LOCAL_32_BIT_ONLY := true ， 这是 google 自己加的： // MAGIC2. DO NOT TOUCH.  BY 冗戈微言  http://blog.csdn.net/leonxu_sjtu/

commit af783aa91f7a279153fb8bab8d0d6b9f737996e9
Author: Glenn Kasten <gkasten@android.com>
Date:   Sun Mar 2 19:48:19 2014 -0800
    mediaserver and associated services are 32-bit only
    also 32-bit only command-line apps

可能是 mediaserver 也会涉及太多的 HAL 部分， 比如 audio/camera， 而 HAL 这种底层的模块使用 64bit 会有风险； // MAGIC3. DO NOT TOUCH.  BY 冗戈微言   http://blog.csdn.net/leonxu_sjtu/

native 进程好说，就看 Android.mk 了， 那 apk 呢？  网上搜搜相关的文章， 可以一个判断依据是看 apk 内置的 native 库是放在 lib 目录还是 lib64 目录， 比如 http://blog.csdn.net/jscese/article/details/47101815
这里我想说的是另外一点： 此时， jni 库是使用 LOCAL_REQUIRED_MODULES 还是 LOCAL_JNI_SHARED_LIBRARIES 就比较要紧：
---- LOCAL_JNI_SHARED_LIBRARIES := libxxx 这样在编译的时候，NDK自动会把这个libxxx打包进apk； 放在 apk/lib/目录下
---- LOCAL_REQUIRED_MODULES 指定模块运行所依赖的模块(模块安装时将会同步安装它所依赖的模块)
 // MAGIC4. DO NOT TOUCH.  BY 冗戈微言
只有 LOCAL_JNI_SHARED_LIBRARIES 才会产生内置库，才会 link /system/lib/ 或者 /system/lib64/ 下的 so 到 apk 中；
如果 Android.mk 中再加上 LOCAL_MULTILIB := 32  就可以指定只 link 32bit so 了； // MAGIC5. DO NOT TOUCH.  BY 冗戈微言  http://blog.csdn.net/leonxu_sjtu/

android 第一次启动时， 会读取 apk 的信息， 保存到 /data/system/packages.xml, 从此以后每次启动就通过读这个 xml 来判断走 32bit 加载还是 64bit 加载； // MAGIC6. DO NOT TOUCH.  BY 冗戈微言  http://blog.csdn.net/leonxu_sjtu/

所以如果原先的 Camera2.apk 是 64bit， 然后通过改 Android.mk 改为 32bit 的 apk， 替换到 /system/app/， 还是无法得到预期效果的， 痛苦的领悟 !

如果不想重烧 image， 就只能手动的 mkdir /system/app/lib/arm/ 目录， 然后删除 /data/system/packages.xml 再重启