编译命令：如
./cts-tradefed
run cts --p android.view
run cts -c android.view.cts.ViewTest -m testOnKeyShortcut
CTS Fail 处理方法
▪Step1: 搜索SOP
SOP路径：DCC : http://dcc.mediatek.com
注： 若上述link无法下载，麻烦自行按照上述目录提示到DCC 里去下载即可
以如下Fail项为例：
Ø先在SOP中搜Class, 关键字: "android.security.cts.PackageSignatureTest"
| android.security.cts.PackageSignatureTest | |||
| -- testPackageSignatures |
| ||
Ø再在SOP中搜method,关键字:"testPackageSignatures"
如果Step1中无法搜到Fail项的解决方案，尝试Step2
▪Step2:搜索MTKOnline FAQ
Ø先在FAQ中搜Class, 关键字: "android.security.cts.PackageSignatureTest"
Ø再在FAQ中搜method,关键字:"testPackageSignatures"
Ø亦可参见速成学院 "常见Fail FAQ"
如果Step2中仍无法搜到Fail项的解决方案，尝试Step3
▪Step3: 准备提交CTS Eservice
环境搭建
Setup PC Environment
▪
OS: Ubuntu 10.4 (or latest)
▪
JDK
:
Linux x86 or Linux x64 (or latest
▪
Android SDK
: android-sdk_r20.0.1-linux (Linux i386 or latest)
▪
Android CDK: Compatibility Test Suite (CTS) / Compatibility Definition Document (CDD)
▪ Udev configuration
ØTerminal
sudo touch etc/udev.rules.d/51-android.rules
sudo chmod a+rw /etc/udev/rules.d/51-android.rules
sudo chmod a+rw /etc/udev/rules.d/51-android.rules
ØAdd the below string in 51-android-rules(/etc/udev/rules.d/51-android.rules)
SUBSYSTEM=="usb", ENV{DEVTYPE}=="usb_device", MODE="0666"
SUBSYSTEM=="usb", ATTRS{idVendor}=="18d1", ATTRS{idProduct}=="2d01", MODE="0666"
SUBSYSTEM=="usb", ATTRS{idVendor}=="0bb4", ATTRS{idProduct}=="0c03", MODE="0666"
SUBSYSTEM=="usb", ATTRS{idVendor}=="0bb4", ATTRS{idProduct}=="0c02", MODE="0666"
SUBSYSTEM=="usb", ATTRS{idVendor}=="18d1", ATTRS{idProduct}=="2d01", MODE="0666"
SUBSYSTEM=="usb", ATTRS{idVendor}=="0bb4", ATTRS{idProduct}=="0c03", MODE="0666"
SUBSYSTEM=="usb", ATTRS{idVendor}=="0bb4", ATTRS{idProduct}=="0c02", MODE="0666"
ØRestart the udev device to update the configuration
sudo restart udev
ØRestart the Ubuntu PC
ØAndroid device connect PC.
Terminal: lsusb
Phone Environment
| Items | 2.3 | 4.0 & above |
| Factory data reset | Settings > Privacy > Factory data reset | Settings > Backup & reset > Factory data reset |
| SD card | Need insert empty SD card | Same as GB |
| Account | No Google account or other account add | Same as GB |
| Date & time | Settings > Date & time | Same as GB |
| Accessibility | 1.Install CtsDelegatingAccessibilityService.apk | android 4.4 此项不用设置 1.Install CtsDelegatingAccessibilityService.apk |
| 2. enable Settings > Accessibility > (1)Accessibility >(2)Delegating Accessibility Service | 2.enable Settings > Accessibility > Delegating Accessibility Service | |
| Administration | 1.Install CtsDeviceAdmin.apk | 1.Install CtsDeviceAdmin.apk |
| 2. Settings > security > Select device administrators | 2. Settings > Security> Device administrators | |
| CTS Media | 1.Download CTS media files. (CTS official ) | Same as GB |
| 2. Run copy_media.sh [720x480|1280x720|1920x1080|all] [-s serial]. If no resolution is specified, the default maximum resolution of 480x360 is assumed. | ||
| 3.command:. copy_media.sh all | ||
| Tag log | None | ICS eng load, disable the Tag log in EM |
| Screen Timeout / Sleep | Settings > Display > Screen Timeout should be set to “30 minutes” or the largest value | Settings > Display > Sleep should be set to “30 minutes” or the largest value |
| Screen lock | Settings > Security > Set up Screen lock > None | Settings > Security > Screen lock>None |
| USB debugging | Settings > Applications > Development > USB debugging | Settings > Developer options > USB debugging |
| Stay Awake | Settings > Applications > Development > Stay awake | Settings > Developer options > Stay awake |
| Allow mock locations | Settings > Applications > Development > allow mock locations | Settings > Developer options > allow mock locations |
| Home screen | Before start CTS, device at the home screen | Same as GB |
| IMEI | 1.Write IMEI | Same as GB |
| 2.Settings > About phone > Status > IMEI information > IMEI | ||
| USB cable | Make sure USB cable is stability | Same as GB |
| SIM card | Insert valid SIM card | Same as GB |
| CMCC TD | Settings > Wireless & networks > Mobile networks > Access Point Names > enable CMNET | Settings > More… > Mobile networks > Access Point Names > enable CMNET |
| Other tasks | While a device is running tests, it must not be used for any other tasks | Same as GB |
| Press key | Do not press any keys on the device while CTS is running | Same as GB |
| System Language | Settings > Language & keyboard > Select Language > English | Settings > Language & input > Language > English |
| Input method | Settings > Language & keyboard > Select input method > Android keyboard | Settings > Language & input > Default > English (US) Android keyboard |
| Wi-Fi | Make sure the AP is usability and connectedly. | Same as GB |
| Data connection | Settings > SIM management > Enable Data Connection | Same as GB |
| Phone number | 1.Setup “My phone number “ | 1.Setup “My phone number “ |
| 2. Settings > About phone > Status > My phone number | 2. The phone number will display on the testResult.xml | |
| 3. The phone number will display on the testResult.xml |
常见Fail FAQ
1.testVideoSnapshot
ID: FAQ08014
2.
How to Summit CTS Issue
ID: FAQ08013
3.testCopyPixelsToBuffer
ID: FAQ07432
4.testPermissionDiffCert
ID: FAQ06193
5.android.permission2.cts.NoReceiveGsmSmsPermissionTest
ID: FAQ08390
CTS Fail: android.permission2.cts.NoReceiveGsmSmsPermissionTest/android.telephony.cts.SmsManagerTest
6.android.media.cts.StreamingMediaPlayerTest
ID: FAQ08989
7.XDPI，YDPI
ID: FAQ09135
8.BuildFingerprint
ID: FAQ09039
9.android.app.cts.SystemFeaturesTest
ID: FAQ08388
CTS Fail: android.app.cts.SystemFeaturesTest#testCameraFeatures
10.android.accessibilityservice.cts.AccessibilityTextTraversalTest
ID: FAQ06620
11.android.mediastress
ID: FAQ10066
12.libcore.java.net.InetAddressTest
ID: FAQ09392
13.android.security.cts.KernelSettingsTest
ID: FAQ09377
14.testFocusAreas
ID: FAQ08312
15.testJpegExif
ID: FAQ07607
16.testGetWithID
ID: FAQ07182
17.DisplayRefreshRateTest
ID: FAQ05826
18.testGetWithId
ID: FAQ04037
19.Root Process
ID: FAQ09841
20.Android adb shell 无法启动
ID: FAQ04112
21. APK 签名
ID: FAQ05362
22.开启或关闭SIP功能
ID: FAQ04013
23.testDigitalZoomPreviewCombinations/testDigitalZoom failed
ID: FAQ14249
cts 命令
run cts --p 包名
run cts --c test名字
run cts --t test名字#选项
Android 4.4 Google issue
CTS Version:
特别注意：
有link的Google issue ，不需要再来申请分析报告，用link 申请waive.
说明：
以下表格已得到Google回复的Google issue 和已提交google等待google 反馈的 Google issue:
|
CTS Test Case
|
Reason
|
| android.view.cts.ViewTest -- testGetLocalVisibleRect fail junit.framework.AssertionFailedError: expected:<100> but was:<150> at android.view.cts.ViewTest.testGetLocalVisibleRect(ViewTest.java:1972) -- testMeasure fail junit.framework.AssertionFailedError: expected:<100> but was:<150> at android.view.cts.ViewTest.testMeasure(ViewTest.java:1372) |
waiting for google feedback.
已提交分析材料给Google,等待google 反馈.
可以申请分析报告.
|
| android.security.cts.AudioPolicyBinderTest
|
waiting for google feedback.
已提交分析材料给Google,等待google 反馈.
可以申请分析报告.
|
| android.webkit.cts.WebViewTest | you can get a waiver. Google 直接waive.无需link/报告. |
| libcore.java.io.SerializationTest#testDeserializeUnserializableField FAIL |
waiting for google feedback.
|
| android.graphics.cts.BitmapFactoryTest | you can get a waiver .
Google 直接waive.无需link/报告.
|
| android.media.cts.EncodeVirtualDisplayWithCompositionTest waive link: |
you can get a waiver .
|
| android.core.tests.libcore.package.tests 12 项Fail
|
Google最新反馈，不能waive,
急于送测的客户申请Temp Patch. 此问题不会有正式Patch.
|
| android.keystore.cts.KeyChainTest #testIsBoundKeyAlgorithm_RequiredAlgorithmsSupported |
you can get a waiver .
可以申请分析报告，
4.4R4和4.4R3的分析报告一样的
如已有4.4R3的，则无需再申请
|
| com.android.cts.verifier.bluetooth /Reliable Write 1. Install "CTS Verifier.apk" . 1. Install "CTS Verifier.apk" . Waive link:
|
you can get a waiver .
|
| android.permission.cts.FileSystemPermissionTest Waive link:
| you can get a waiver . only for [/dev/pvr_sync] fail, other fail can't waive. |
| 部分Chip CTS测试出现多项Not Executed,运行到android.holo 会停止InstallException ( cause: ShellCommandUnresponsiveException) when attempting install /home/chenli/mq/cts/android-cts/tools/./../../android-cts/repository/testcases/CtsHoloTestCases.apk on device 0123456789ABCDEF |
waiting for google feedback.
已提交分析材料给Google,等待google 反馈.
|
CTS Version:
说明：
以下表格不是全部的Google issue,还有一部分Fail未列出
如1. 还在等Google回复的Case 2.与客户手机配置相关的Fail需要具体分析的 ， 若Fail不在下表，建议客户提Eservice 过来跟踪。
|
CTS Test Case
|
Reason
|
| android.app.cts.WallpaperManagerTest #testSuggestDesiredDimensions | this case fail on CTS4.4_r1, it ‘s a Google issue, CTS case version is not consistent. |
| android.os.cts.BuildVersionTest #testReleaseVersion |
this case fail is on CTS4.4_r1, It ‘s a Google issue,CTS case version is not consistent.
|
| In android.core.tests.libcore.package.tests 29TCs | these cases fail on CTS 4.4_r1, It ‘s a Google issue, CTS case version is not consistent. |
| org.apache.harmony.nio_char.tests.java.nio.charset. ASCIICharsetEncoderTest #testInternalState_Flushed | these cases fail on CTS 4.4_r1, It ‘s a Google issue, CTS case version is not consistent. |
| com.android.cts.browser.BrowserBenchTest #testOctane | this case fail on CTS 4.4_r1, It ‘s a Google issue, Google反馈在多browser情况下可以先设置一个作为默认的。 |
| android.keystore.cts.KeyChainTest #testIsBoundKeyAlgorithm_RequiredAlgorithmsSupported | this case fail on CTS4.4_r1, It ‘s a Google issue, Google will waive this case. |
| org.apache.harmony.luni.tests.java.net.URLConnectionTest #test_getOutputStream | this case fail on CTS 4.4_r1, It’s Google server issue, 参考机同样会fail. |
| android.media.cts.MediaCodecCapabilitiesTest #testAvcHigh40 | this case fail on CTS 4.4_r1, It’s Google server issue, 参考机同样会fail. |
| android.media.cts.StreamingMediaPlayerTest #testHLS | this case fail on CTS 4.4_r1, It’s Google server issue, Google反馈可以waive. |
有以下几点需要说明一下：
1.对CTS google issue, 我们只有得到 Google回复才会告知是google issue. 可以waive.
2.CTS issue需要送测厂商主动去和google商谈，MTK从Google拿到回复只是为给客户一个放心的答复，告知客户这个问题是已经有结论的。所以关于Google issue还是需要客户与Google商谈并得到答复为准。
Android 5.0R3 Google issue
CTS Version:
Android 5.0 R3 Compatibility Test Suite (CTS) - ARM
特别注意：
1.有link的Google issue ，不需要再来申请分析报告，用link申请waive.
2.已得到Google回复的： You can get a waiver .
3.已提交google等待google 回复的： Waiting for google feedback.
所有Waiting for Google feedback的Google issue, 均需要客户与Google 确认是否可以waive, MTK 亦在努力与Google 沟通中，一旦拿到Google waiver, 会修改成:You can get a waiver
以下Google issue,不是全部Google issue, 其它fail 需要提供Log 来确认
|
CTS Test Case
|
Status
|
|
[CTS 5.0_r3]
android.widget.cts.PopupWindowTest -- testGetMaxAvailableHeight fail junit.framework.AssertionFailedError at junit.framework.Assert.fail(Assert.java:48) |
Waiting for google feedback.
|
|
[CTS 5.0_r2][CTS 5.1_r1] android.provider.cts.BrowserTest#testAccessSearches
android.database.sqlite.SQLiteConstraintException: UNIQUE constraint failed: bookmarks._id (code 1555) at android.database.DatabaseUtils.readExceptionFromParcel(DatabaseUtils.java:173)
| Waiting for google feedback. https://android-review.googlesource.com/115642/ |
|
[CTS 5.0_r2][CTS 5.1_r1] android.provider.cts.BrowserTest#testAccessHistory
android.database.sqlite.SQLiteConstraintException: UNIQUE constraint failed: bookmarks._id (code 1555) at android.database.DatabaseUtils.readExceptionFromParcel(DatabaseUtils.java:173)
|
Waiting for google feedback.
https://android-review.googlesource.com/115650/ |
|
[CTS 5.0_r2][CTS 5.1_r1]
android.mediastress.cts.HEVCR1080p AacShort PlayerTest#testPlay00 android.mediastress.cts.HEVCR1080pAacRepeated PlayerTest#testPlay00 android.mediastress.cts.HEVCR1080pAacLongPlayerTest#testPlay00 |
[Only For MT6582 L]
You can get a waiver .
用b/21632109申请waive
|
|
[CTS 5.0_r2][CTS 5.1_r1] android.mediastress package have 112 Tcs Fail
|
[Only Happen on 64bit L]
Waiting for google feedback.
分析报告可在MOL搜索下载：
CTS_Mediastress 112_Fail_for_64bit_device.pptx
|
|
[CTS 5.0_r3][CTS 5.1_r2]
android.keystore.cts.KeyChainTest #testIsBoundKeyAlgorithm_RequiredAlgorithmsSupported |
Waiting for google feedback. 分析报告和KK一样，是同样的问题，可以申请分析报告 |
|
[CTS Verifier CTS 5.0_r1]
Projection Offscreen Activity [Steps] 1. Install "CTS Verifier.apk" . 2. Start CTS verifier, go to "Projection Offscreen Activity". 3. no usb cable connected 4.click power key to turn off the screen. 5.waiting for 5s,then turn it back on. [Expected Result] a sound will be played after 5 seconds and the item pass . [Actual Result] a sound is not played after 5 seconds, need to wait about 10s or more time. |
Waiting for google feedback. 可以申请分析报告 |
|
[CTS Verifier 5.0_r2] [CTS Verifier 5.1_r1]
Connectivity Constraints [Steps] 1. Install "CTS Verifier.apk" . 2. Start CTS verifier, go to "Connectivity Constraints“. 3. Tap "START TEST" button. [Expected Result] All items are pass. [Actual Result] "Connectivity Constraints" fail at "Device with no connectivity will still exceute a job with no connectivity constrains". |
Waiting for google feedback. https://android-review.googlesource.com/115320/ |
|
[CTS Verifier 5.0_r2] [CTS Verifier 5.1_r1] Battery Saving Mode Test & High Accuracy Mode Test
[Steps] 1. Install "CTS Verifier.apk" . 2. Start CTS verifier, go to "Battery Saving Mode Test (or "High Accuracy Mode Test "). 3. Testing items follow tips. [Expected Result] All items are pass. [Actual Result] "Battery Saving Mode Test" fail at "network location provider should be not on in locationmanager". "High Accuracy Mode Test" fail at "network location provider should be not on in locationmanager". |
Waiting for google feedback.
分析报告可在MOL搜索下载：
CTS Verifier 5.0_r1_Battery Saving Mode Test High Accuracy Mode Test.pptx
|
Android 5.1R1 Google issue
CTS Version:
特别注意：
1.有link的Google issue ，不需要再来申请分析报告，用link申请waive.
2.已得到Google回复的： You can get a waiver .
3.已提交google等待google 回复的： Waiting for google feedback.
所有Waiting for Google feedback的Google issue, 均需要客户与Google 确认是否可以waive, MTK 亦在努力与Google 沟通中，一旦拿到Google waiver, 会修改成:You can get a waiver
以下Google issue,不是全部Google issue, 其它fail 需要提供Log 来确认
|
CTS Test Case
|
Status
|
| [CTS 5.1_r2] android.media.cts.VideoEncoderTest 多项fail 【注意】需要提Eservice 过来，用MTK RD修改过的测试包来确认此项是否为Google issue. 若MTk工程师分析为Google issue, 请参看Status状况 |
Waiting for google feedback.
对于急于送测的客户，此项有修改方案来规避google issue，前提是客戶可以接受263 encoder的能力調整為只能壓QCIF (176x144)
|
|
[CTS 5.1_r1] android.mediastress 多项Fail
【注意】需要提eservice过来确认 是否为
Google issue. 需提供测试报告+ CTS log
|
You can get a waiver .
b/18816147
Google 原文:
5.1 builds running CTS 5.1_r1 a waiver will be granted based on b/18816147 |
|
[CTS 5.1_r2]
android.security.cts.ServicePermissionsTest
#testDumpProtected
junit.framework.AssertionFailedError: dump() for backup produced a single line which didn't reference a permission; it may be leaking sensitive data. at junit.framework.Assert.fail(Assert.java:50)
|
You can get a waiver .
b/19422232
【注意】b/19422232为google内部的bug编号, 客户直接提供给google 即可得到waive.
|
| [CTS 5.1_r2] android.webkit.cts.WebSettingsTest #testUserAgentString_default
junit.framework.ComparisonFailure: expected:<LMY47D[]> but was:<LMY47D[; wv]> at junit.framework.Assert.assertEquals(Assert.java:85)
|
You can get a waiver .
Google 原文: Waiver will be granted.
贵司内置了GMS包，这个是GMS issue，是因为GMS包与原生chromium lib有冲突。原则上google是有要求GMS和native browser不共用
|
|
[CTS 5.1_r2] android.keystore.cts.KeyChainTest
#testIsBoundKeyAlgorithm_RequiredAlgorithmsSupported
|
Waiting for google feedback.
可以申请分析报告
|
|
[CTS Verifier 5.1_r2]Screen pinning Test
无虚拟按键，只有实体键，此项fail
|
此google issue ，Google 不同意waive,
MTK已有解决方案， 可提eservice来申请解决方案~
|
|
[CTS Verifier 5.1_r1] Battery Saving Mode Test & High Accuracy Mode Test
[Steps] 1. Install "CTS Verifier.apk" . 2. Start CTS verifier, go to "Battery Saving Mode Test (or "High Accuracy Mode Test "). 3. Testing items follow tips. [Expected Result] All items are pass. [Actual Result] "Battery Saving Mode Test" fail at "network location provider should be not on in locationmanager". "High Accuracy Mode Test" fail at "network location provider should be not on in locationmanager". |
Waiting for google feedback.
和CTS5.0的一样，分析报告可在MOL搜索下载：
CTS Verifier 5.0_r1_Battery Saving Mode Test High Accuracy Mode Test.pptx
|
|
[CTS 5.1_r1]
android.telephony.cts.SubscriptionManagerTest
#testAddOnSubscriptionsChangedListener Test failed to run to completion. Reason: 'Failed to receive adb shell test output within 600000 ms. Test may have timed out, or adb connection to device became unresponsive'. Check device logcat for details |
You can get a waiver .
Google 原文:
Looks like a waiver is necessary for L MR1
Not necessary to provide a waiver link... all TAMs should know about this. It is a known issue.
MTK提交的waiver Link: |
|
|
|
|
[CTS 5.0_r2][CTS 5.1_r2] android.provider.cts.BrowserTest#testAccessSearches
android.database.sqlite.SQLiteConstraintException: UNIQUE constraint failed: bookmarks._id (code 1555) at android.database.DatabaseUtils.readExceptionFromParcel(DatabaseUtils.java:173)
|
Waiting for google feedback.
|
|
CTS 5.0_r2][CTS 5.1_r2] android.provider.cts.BrowserTest#testAccessHistory
android.database.sqlite.SQLiteConstraintException: UNIQUE constraint failed: bookmarks._id (code 1555) at android.database.DatabaseUtils.readExceptionFromParcel(DatabaseUtils.java:173)
|
Waiting for google feedback.
https://android-review.googlesource.com/115650/ |
|
[CTS 5.0_r2][CTS 5.1_r1] android.mediastress package have 112 Tcs Fail
|
[Only Happen on 64bit L]
Waiting for google feedback. 可以申请分析报告 |
|
CTS Verifier 5.0_r2] [CTS Verifier 5.1_r1]
Connectivity Constraints
[Steps] 1. Install "CTS Verifier.apk" . 2. Start CTS verifier, go to "Connectivity Constraints“. 3. Tap "START TEST" button. [Expected Result] All items are pass. [Actual Result] "Connectivity Constraints" fail at "Device with no connectivity will still exceute a job with no connectivity constrains". |
Waiting for google feedback.
https://android-review.googlesource.com/115320/ |
|
[CTS 5.0_r3][CTS 5.1_r2]
android.mediastress.cts.HEVCR1080p AacShort PlayerTest#testPlay00 android.mediastress.cts.HEVCR1080pAacRepeated PlayerTest#testPlay00 android.mediastress.cts.HEVCR1080pAacLongPlayerTest#testPlay00 |
[Only For MT6582 & MT6580 L]
You can get a waiver .
用b/21632109申请waive
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
508
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
