android 4.4 cts测试,android之CTS兼容性测试及FAIL issue

最新推荐文章于 2024-01-31 17:46:21 发布

真·失败人士

最新推荐文章于 2024-01-31 17:46:21 发布

阅读量692

点赞数

文章标签： android 4.4 cts测试

最近在忙碌着CTS测试，头大啊，各种莫名的fail，timeout，今天晚上刚好有时间来写篇文章来记录下；

CTS(Compatibility Test Suite)；为什么要做CTS测试：

Give your users the best possible experience with the applications they run.

When a device is compatible with Android, users can choose from among many high-quality applications.

Applications that take full advantage of Android's features are likely to perform best on compatible devices.

Make it easy for developers to write top-quality applications for your device.Developers want to streamline their applications for Android,

and this is easiest for them when they are writing for a predictable platform.

Take advantage of the Android Market.

Compatible handsets can give users access to the Android Market.

一、CTS测试之前的需要的准备项和如何进行测试；

做CTS测试首先得做很多项的配置，现在来仔细说下CTS的配置过程：

1、首先得安装有adb，并且配置adb的环境变量；export PATH=$PATH:/home/myuser/adb；

2、准备一个cts包，可以在源码中通过make cts编译，也可以在android网站上面下载下来(http://source.android.com/compatibility/downloads.html)；

包里面有三个文件夹，其中tools下面有测试需要的jar包以及cts-tradefed脚本用于执行cts命令；还有一个repository文件夹，里面有一些测试计划，测试结果也会生成在里面；在repository/testcases目录下找到CtsDelegatingAccessibilityService.apk和/CtsDeviceAdmin.apk，并且安装；

adb install android-cts/repository/testcases/CtsDelegatingAccessibilityService.apk

adb install android-cts/repository/testcases/CtsDeviceAdmin.apk

安装以后在setting中会多出设置项，需要进行设置Settings > Accessibility > Delegating AccessibilityService，Settings > Security > Device Administrators >android.deviceadmin.cts.CtsDeviceAdmin* settings；

3、在setting中需要设置多项设置项，setting-display-sleep设置30min钟，language设置为E文，Security中Screen lock设置NONE；

4、Cts有media stress测试，你需要在这里下下载各种分辨率的mp4视频文件，在这里下载android-cts-media-X.Y.zip，执行里面的copy_media.sh脚本，把视频文件拷贝到/sdcard/test目录下,当然你也可以手动push；

5、插入一张有钱的SIM卡；

都准备好了接下来就可以跑CTS了，首先

./android-cts/tools/cts-tradefed

这下就进入了CTS测试的命令行模式了，然后执行

run cts --plan CTS

进行全部项的测试，如果你想测试某一个测试plan，可以

run cts --plan

这个时候等待结果就行了...漫长的等待啊......

二、如何进行单项测试呢？

在Linux环境下，如果你想进行单项测试，比如说某个package，class，method之类的，可以这样

run cts --package packageName1

run cts --class className1

run cts --class className --method methodName

,这下就可以对单个的错误进行调试运行了；

如果我想在windows环境下进行CTS单项测试呢(不知道windows下有没有CTS整项测试)？你首先需要将repositorytestcases下的所有的apk之类的文件都push到data/app下面去，然后通过

adb shell pm list instrumentation

显示所有的Runner，这里的Runner作为下面这个命令-w以后的参数，然后通过

adb shell am instrument -e class 类名#方法名 -w “Runner”

进行单项的测试；

举个例子，比如：

adb shell am instrument -e

class android.accessibilityservice.cts.AccessibilityTextTraversalTest#testActionNextAndPreviousAtGranularityPageOverText

-w com.android.cts.accessibilityservice/android.test.InstrumentationCtsTestRunner

悲催的result：

android.accessibilityservice.cts.AccessibilityTextTraversalTest:

Error in testActionNextAndPreviousAtGranularityPageOverText:

java.util.concurrent.TimeoutException: Expected event not received within: 5000 ms.

at android.accessibilityservice.cts.AccessibilityActivityTestCase$AccessibilityInteractionBridge.executeCommandA

ndWaitForAccessibilityEvent(AccessibilityActivityTestCase.java:542)

at android.accessibilityservice.cts.AccessibilityTextTraversalTest.testActionNextAndPreviousAtGranularityPageOve

rText(AccessibilityTextTraversalTest.java:1219)

at java.lang.reflect.Method.invokeNative(Native Method)

at android.test.InstrumentationTestCase.runMethod(InstrumentationTestCase.java:214)

at android.test.InstrumentationTestCase.runTest(InstrumentationTestCase.java:199)

at android.test.ActivityInstrumentationTestCase2.runTest(ActivityInstrumentationTestCase2.java:192)

at android.test.AndroidTestRunner.runTest(AndroidTestRunner.java:190)

at android.test.AndroidTestRunner.runTest(AndroidTestRunner.java:175)

at android.test.InstrumentationTestRunner.onStart(InstrumentationTestRunner.java:555)

at android.app.Instrumentation$InstrumentationThread.run(Instrumentation.java:1661)

Test results for InstrumentationCtsTestRunner=.E

Time: 5.796

FAILURES!!!

Tests run: 1, Failures: 0, Errors: 1

相比windows下进行单项测试，Linux下还是简单的多了。。。

对于UiAutomatorTests的测试项，如何进行单测呢？因为之前我们意见将CtsUiAutomatorTests.jar push到了data/app下，我们只需执行如下命令即可测试；

uiautomator runtest /data/app/CtsUiAutomatorTests.jar -c com.android.cts.uiautomatortest.CtsUiAutomatorTest

三、如何针对所有的failed的项进行重测呢？

首先你得通过l r命令列出测试记录的sessionId，然后进入respository/result/×××/testResult.xml中，首先把failed总项数改为0，notExecuted总项数改为你之前failed的项数；

然后通过vi的全局替换命令“1,$s/old/new/g”替换所有的result=“failed”为result=“notExecuted”，

run cts --continue -session “sessionID”

接下来就可以通过上面这个命令进行重测了；

四、除去不需要测试的测项

在某些时候，当手机没有该项功能时，是否我们就不需要测试该项呢？比如不带telephony的平板电脑CTS就不需要去测试telephony的项；

如何去除看下这段代码就知道了：

PackageManagerService.java

void readPermissions() {

// Read permissions from .../etc/permission directory.

File libraryDir = new File(Environment.getRootDirectory(), "etc/permissions");

if (!libraryDir.exists() || !libraryDir.isDirectory()) {

Slog.w(TAG, "No directory " + libraryDir + ", skipping");

return;

}

if (!libraryDir.canRead()) {

Slog.w(TAG, "Directory " + libraryDir + " cannot be read");

return;

}

// Iterate over the files in the directory and scan .xml files

for (File f : libraryDir.listFiles()) {

// We'll read platform.xml last

if (f.getPath().endsWith("etc/permissions/platform.xml")) {

continue;

}

if (!f.getPath().endsWith(".xml")) {

Slog.i(TAG, "Non-xml file " + f + " in " + libraryDir + " directory, ignoring");

continue;

}

if (!f.canRead()) {

Slog.w(TAG, "Permissions library file " + f + " cannot be read");

continue;

}

readPermissionsFromXml(f);

}

// Read permissions from .../etc/permissions/platform.xml last so it will take precedence

final File permFile = new File(Environment.getRootDirectory(),

"etc/permissions/platform.xml");

readPermissionsFromXml(permFile);

}

现在知道了去除的方法就是把/etc/permissions下除了platform.xml的其他xml中找到telephony相关的配置文件，去除就行了，当然其对应的编译前文件在framework/native/data/etc下；对其进行修改即可；

五、CTS bug的记录；

在此记录一下我跟了好几天的一个CTS bug：

1、

android.media.cts.MediaPlayerTest -- testGapless1 fail junit.framework.AssertionFailedError: silence detected, please increase volume and rerun test at android.media.cts.MediaPlayerTest.testGapless(MediaPlayerTest.java:397)

出现此问题的原因是EffectVisualizer.cpp中的Context状态一直是VISUALIZER_STATE_INITIALIZED无法VISUALIZER_STATE_ACTIVE，导致无法走进判断；修改system.prop下的两个property为false即可通过测试；PS：不同的android项目可能在不同的prop里面，也有可能在build.prop下；另外此选项是针对qcomm平台的这种解码方式；

Tunnel.decode use for low latency playback, it use Qualcomm HW decoder. But Tunnel decoder don't support Gapless playback, so please disable it when do CTS test.

tunnel.decode=false

tunnel.audiovideo.decode=false

当然，你也可以

getprop tunnel.decode

然后重测看一下结果；

2、

android.security.cts.PackageSignatureTest -- testPackageSignatures fail junit.framework.AssertionFailedError: These packages should not be signed with a well known key: [com.qualcomm.wifip2p.service, com.android.voicedialer, com.qualcomm.timeservice, com.qualcomm.qlogcat, com.qualcomm.atfwd, com.android.launcher, com.android.defcontainer, com.android.quicksearchbox, com.android.contacts, com.android.phone, com.android.calculator2, com.android.htmlviewer, com.android.cellbroadcastreceiver, com.android.MultiplePdpTest, com.android.providers.calendar,.......

[reason]

using android default key to sign apk；

[Solution]

sign apk using our own key

[Change File]

buildtargetproductsecurity*

3、

-- testNoSetuidTcpdump fail junit.framework.AssertionFailedError: File "/system/xbin/tcpdump" is setUID at android.security.cts.BannedFilesTest.assertNotSetugid(BannedFilesTest.java:71)

[reason]

the permission for system/xbin/tcpdump is wrong.

[Solution]

set the correct permission to 00755 for system/xbin/tcpdump

[Change File]

system/core/include/private/android_filesystem_config.h；

4、

-- testStartUsingNetworkFeature_enableHipri fail junit.framework.AssertionFailedError: Couldn't connect using hipri... at android.net.cts.ConnectivityManagerTest.testStartUsingNetworkFeature_enableHipri(ConnectivityManagerTest.java:293)

[reason]

don't set the right APN

[Solution]

set APN and in the APN type field include "hipri"

[Change File]

N/A

5、

-- testActivateGoodReceiverDisplaysActivationUi fail junit.framework.AssertionFailedError at android.admin.cts.DeviceAdminActivationTest.assertDeviceAdminDeactivated(DeviceAdminActivationTest.java:200)1

junit.framework.AssertionFailedError

at android.admin.cts.DeviceAdminActivationTest.assertDeviceAdminDeactivated(DeviceAdminActivationTest.java:200)

at android.admin.cts.DeviceAdminActivationTest.testActivateGoodReceiverDisplaysActivationUi(DeviceAdminActivatio

nTest.java:102)

在网上找到的一个解决办法：

Install the "CtsDeviceAdmin.apk" and

enable only the first two

options in the security-->Deviceadministrators".

It will make the Test case "testActivateGoodReceiverDisplaysActivationUi"

PASS. Even all the 18 Test cases in this Package

(android.admin) will get PASS on Android 4.3;

但是奇怪的是这个fail报错的位置在源码中找不到，DeviceAdminActivationTest,java中根本没有200行的代码，查看4.4的cts源码也是一样的；不知道是不是android-cts-4.3_r2-linux_x86-arm.zip的问题。

6、

-- testUsbDebugging fail junit.framework.ComparisonFailure: expected: but was: at android.os.cts.UsbDebuggingTest.testUsbDebugging(UsbDebuggingTest.java:28)

这个是测试secure USB debug的测项，需要ro.adb.secure这个property；

assertEquals("1", SystemProperties.get("ro.adb.secure"));

在main.mk文件中添加

ADDITIONAL_DEFAULT_PROPERTIES += ro.adb.secure=1即可启用；

关于这个secure USB debugging可以参考，请务必注意修改;在开启secure USB debugging后，如果你的机器插入usb时无法弹出authorization的弹出框，你需要更新你的adb版本，adb version：Android Debug Bridge version 1.0.31；

7、

android.net.cts.TrafficStatsTest

-- testTrafficStatsForLocalhost fail junit.framework.AssertionFailedError: uidtxb: -1 -> -1 at android.net.cts.TrafficStatsTest.testTrafficStatsForLocalhost(TrafficStatsTest.java:136)

[reason]

CONFIG_UID_STAT not set

[Solution]

set CONFIG_UID_STAT=y

[Change File]

kernel/arch/arm/configs/msm8960_defconfig"

8、

org.apache.harmony.luni.tests.java.net.URLConnectionTest

-- test_getOutputStream fail junit.framework.AssertionFailedError: Got different responseCode expected:<405> but was:<200> at org.apache.harmony.luni.tests.java.net.URLConnectionTest.test_getOutputStream(URLConnectionTest.java:870)

这个CTS bug是(2014.01)最近才会出现的，原因是www.google.com/ie的post访问方式的HTTP返回码问题，无需fix，google会在最新的code中删除掉此项测试；

真·失败人士

关注

0
点赞
踩
3

收藏

觉得还不错? 一键收藏
0
评论
android 4.4 cts测试,android之CTS兼容性测试及FAIL issue

最近在忙碌着CTS测试，头大啊，各种莫名的fail，timeout，今天晚上刚好有时间来写篇文章来记录下；CTS(Compatibility Test Suite)；为什么要做CTS测试：Give your users the best possible experience with the applications they run.When a device is compatible wi...
复制链接

扫一扫