CTS(Compatibility Test Suite);为什么要做CTS测试:
1. Give your users the best possible experience with the applications they run.When a device is compatible with Android, users can choose from among many high-quality applications. Applications that take full advantage of Android's features are likely to perform best on compatible devices.
2. Make it easy for developers to write top-quality applications for your device. Developers want to streamline their applications for Android, and this is easiest for them when they are writing for a predictable platform.
3. Take advantage of the Android Market. Compatible handsets can give users access to the Android Market.
CTS测试之前的需要的准备项和如何进行测试;
做CTS测试首先得做很多项的配置,现在来仔细说下CTS的配置过程:
1、首先得安装有adb,并且配置adb的环境变量;export PATH=$PATH:/home/myuser/adb;
2、准备一个cts包,可以在源码中通过make cts编译,也可以在android网站上面下载下来(http://source.android.com/compatibility/downloads.html);包里面有三个文件夹,其中tools下面有测试需要的jar包以及cts-tradefed脚本用于执行cts命令;还有一个repository文件夹,里面有一些测试计划,测试结果也会生成在里面;在repository/testcases目录下找到CtsDelegatingAccessibilityService.apk和/CtsDeviceAdmin.apk,并且安装;
- adb install android-cts/repository/testcases/CtsDelegatingAccessibilityService.apk
- adb install android-cts/repository/testcases/CtsDeviceAdmin.apk
3、在setting中需要设置多项设置项,setting-display-sleep设置30min钟,language设置为E文,Security中Screen lock设置NONE;
4、Cts有media stress测试,你需要准备一张sdcard,还需要在http://www.bigbuckbunny.org/index.php/download/下下载各种分辨率的mp4视频文件,在http://source.android.com/compatibility/downloads.html下载android-cts-media-X.Y.zip,执行里面的copy_media.sh脚本,把视频文件拷贝到/sdcard/test目录下;
5、插入一张有钱的SIM卡;
都准备好了接下来就可以跑CTS了,首先
- ./android-cts/tools/cts-tradefed
这下就进入了CTS测试的命令行模式了,然后执行
- run cts --plan CTS
进行全部项的测试,如果你想测试某一个测试plan,可以
- run cts --plan <plan_name>
这个时候等待结果就行了...漫长的等待啊......
如何进行单项测试呢?
如果你想进行单项测试,比如说某个package,class,method之类的,可以这样
- run cts --package packageName
- run cts --class className
- run cts --class className --method methodName
,这下就可以对单个的错误进行调试运行了;这些都是在linux环境下的,如果我想在windows环境下进行CTS单项测试呢(不知道windows下有没有CTS整项测试)?你首先需要将repository\testcases下的所有的apk之类的文件都push到data/app下面去,然后通过
- adb shell pm list instrumentation
显示所有的Runner,这里的Runner作为下面这个命令-w以后的参数,然后通过
- adb shell am instrument -e class 类名#方法名 -w “Runner”
进行单项的测试;
举个例子,比如:
- adb shell am instrument -e
- class android.accessibilityservice.cts.AccessibilityTextTraversalTest#testActionNextAndPreviousAtGranularityPageOverText
- -w com.android.cts.accessibilityservice/android.test.InstrumentationCtsTestRunner
悲催的result:
- android.accessibilityservice.cts.AccessibilityTextTraversalTest:
- Error in testActionNextAndPreviousAtGranularityPageOverText:
- java.util.concurrent.TimeoutException: Expected event not received within: 5000 ms.
- at android.accessibilityservice.cts.AccessibilityActivityTestCase$AccessibilityInteractionBridge.executeCommandA
- ndWaitForAccessibilityEvent(AccessibilityActivityTestCase.java:542)
- at android.accessibilityservice.cts.AccessibilityTextTraversalTest.testActionNextAndPreviousAtGranularityPageOve
- rText(AccessibilityTextTraversalTest.java:1219)
- at java.lang.reflect.Method.invokeNative(Native Method)
- at android.test.InstrumentationTestCase.runMethod(InstrumentationTestCase.java:214)
- at android.test.InstrumentationTestCase.runTest(InstrumentationTestCase.java:199)
- at android.test.ActivityInstrumentationTestCase2.runTest(ActivityInstrumentationTestCase2.java:192)
- at android.test.AndroidTestRunner.runTest(AndroidTestRunner.java:190)
- at android.test.AndroidTestRunner.runTest(AndroidTestRunner.java:175)
- at android.test.InstrumentationTestRunner.onStart(InstrumentationTestRunner.java:555)
- at android.app.Instrumentation$InstrumentationThread.run(Instrumentation.java:1661)
- Test results for InstrumentationCtsTestRunner=.E
- Time: 5.796
- FAILURES!!!
- Tests run: 1, Failures: 0, Errors: 1
相比windows下进行单项测试,Linux下还是简单的多了。。。
如何针对所有的failed的项进行重测呢?
首先你得通过l r命令列出测试记录的sessionId,然后进入respository/result/×××/testResult.xml中,首先把failed总项数改为0,notExecuted总项数改为你之前failed的项数;
然后通过vi的全局替换命令“1,$s/old/new/g”替换所有的result=“failed”为result=“notExecuted”,
- run cts --continue -session “sessionID”
接下来就可以通过上面这个命令进行重测了;
CTS bug的记录;
在此记录一下我跟了好几天的一个CTS bug:
1、
android.media.cts.MediaPlayerTest | ||
-- testGapless1 |
fail
|
junit.framework.AssertionFailedError: silence detected, please increase volume and rerun test at android.media.cts.MediaPlayerTest.testGapless(MediaPlayerTest.java:397)
|
出现此问题的原因是EffectVisualizer.cpp中的Context状态一直是VISUALIZER_STATE_INITIALIZED无法VISUALIZER_STATE_ACTIVE,导致无法走进判断;修改system.prop下的两个property为false即可通过测试;PS:不同的android项目可能在不同的prop里面,也有可能在build.prop下;另外此选项是针对qcomm平台的这种解码方式;
Tunnel.decode use for low latency playback, it use Qualcomm HW decoder. But Tunnel decoder don't support Gapless playback, so please disable it when do CTS test.
- tunnel.decode=false
- tunnel.audiovideo.decode=false
当然,你也可以
- getprop tunnel.decode
然后重测看一下结果;
2、
android.security.cts.PackageSignatureTest | ||
--testPackageSignatures |
fail
|
junit.framework.AssertionFailedError: These packages should not be signed with a well known key: [com.qualcomm.wifip2p.service, com.android.voicedialer, com.qualcomm.timeservice, com.qualcomm.qlogcat, com.qualcomm.atfwd, com.android.launcher, com.android.defcontainer, com.android.quicksearchbox, com.android.contacts, com.android.phone, com.android.calculator2, com.android.htmlviewer, com.android.cellbroadcastreceiver, com.android.MultiplePdpTest, com.android.providers.calendar,.......
|
[reason]
using android default key to sign apk;
[Solution]
sign apk using our own key
[Change File]
build\target\product\security\*
3、
-- testNoSetuidTcpdump |
fail
|
junit.framework.AssertionFailedError: File "/system/xbin/tcpdump" is setUID at android.security.cts.BannedFilesTest.assertNotSetugid(BannedFilesTest.java:71)
|
[reason]
the permission for system/xbin/tcpdump is wrong.
[Solution]
set the correct permission to 00755 for system/xbin/tcpdump
[Change File]
system/core/include/private/android_filesystem_config.h;
4、
-- testStartUsingNetworkFeature_enableHipri |
fail
|
junit.framework.AssertionFailedError: Couldn't connect using hipri... at android.net.cts.ConnectivityManagerTest.testStartUsingNetworkFeature_enableHipri(ConnectivityManagerTest.java:293)
|
[reason]
don't set the right APN
[Solution]
set APN and in the APN type field include "hipri"
[Change File]
N/A
5、
-- testActivateGoodReceiverDisplaysActivationUi |
fail
|
junit.framework.AssertionFailedError at android.admin.cts.DeviceAdminActivationTest.assertDeviceAdminDeactivated(DeviceAdminActivationTest.java:200)
|
- junit.framework.AssertionFailedError
- at android.admin.cts.DeviceAdminActivationTest.assertDeviceAdminDeactivated(DeviceAdminActivationTest.java:200)
- at android.admin.cts.DeviceAdminActivationTest.testActivateGoodReceiverDisplaysActivationUi(DeviceAdminActivatio
- nTest.java:102)
在网上找到的一个解决办法:
Install the "CtsDeviceAdmin.apk" and enable only the first two options in the security-->Deviceadministrators". It will make the Test case "testActivateGoodReceiverDisplaysActivationUi" PASS. Even all the 18 Test cases in this Package (android.admin) will get PASS on Android 4.3
但是奇怪的是这个fail报错的位置在源码中找不到,DeviceAdminActivationTest,java中根本没有200行的代码,查看4.4的cts源码也是一样的;不知道是不是android-cts-4.3_r2-linux_x86-arm.zip的问题。