Android S CTS STS 常见的 fail 项

此文章只是一篇总结，针对 MTK 平台近期的 CTS STS 做个简单的整理回顾，后期不断扩展

1. CTS 专区

CtsLocationPrivilegedTestCases android.location.cts.privileged.GnssMeasurementValuesTest#testListenForGnssMeasurements

junit.framework.AssertionFailedError: Failing tests are:
(Test: FAIL) SatellitPvt is supported. The test must find a measurement with SatellitePvt within 75 seconds.
at junit.framework.Assert.fail(Assert.java:50)
at android.location.cts.common.SoftAssert.assertAll(SoftAssert.java:162)
at android.location.cts.privileged.GnssMeasurementValuesTest.testListenForGnssMeasurements(GnssMeasurementValuesTest.java:156)

此项需要特殊的测试条件，需要在 GPS 信号很好的地方或者室内装有 GPS 放大器，复测可以正常 pass 。

==================================================================================

CtsPermissionTestCases
android.permission.cts.ProviderPermissionTest#testManageDocuments

junit.framework.AssertionFailedError: Exactly one package (must be com.google.android.documentsui) can request the MANAGE_DOCUMENTS permission; found package com.android.contacts which must be revoked for security reasons at junit.framework.Assert.fail(Assert.java:50) at android.permission.cts.ProviderPermissionTest.testManageDocuments(ProviderPermissionTest.java:219)

此项是由于 MTK 平台的联系人权限导致的，在清单文件中注释掉 android.permission.MANAGE_DOCUMENTS

=====================================================================================

CtsNetTestCases
android.net.cts.ConnectivityManagerTest#testOpenConnection

java.lang.AssertionError: Cell network not available. Please ensure the device has working mobile data.
at org.junit.Assert.fail(Assert.java:89)
at org.junit.Assert.assertTrue(Assert.java:42)
at org.junit.Assert.assertNotNull(Assert.java:713)
at android.net.cts.util.CtsNetUtils.connectToCell(CtsNetUtils.java:501)
复测log:
java.net.ConnectException: Failed to connect to google-ipv6test.appspot.com/xx.xxx.xxx.xx:xx

需要特殊的 ipv6 网络

=====================================================================================

CtsMediaTestCases
android.media.cts.MediaCodecListTest#testAllComponentInstantiation
android.media.cts.DecodeAccuracyTest#testSurfaceViewLargerHeightDecodeAccuracy[48]
android.media.cts.DecodeAccuracyTest#testSurfaceViewVideoDecodeAccuracy[48]

junit.framework.AssertionFailedError: With the best matched border crop (0.0, 0.0), greatest pixel difference is 188 at (81, 161) which is over the allowed difference 90
at junit.framework.Assert.fail(Assert.java:50)
at junit.framework.Assert.assertTrue(Assert.java:20)
at android.media.cts.DecodeAccuracyTest.validateResult(DecodeAccuracyTest.java:282)
at android.media.cts.DecodeAccuracyTest.runTest(DecodeAccuracyTest.java:251)
at android.media.cts.DecodeAccuracyTest.testSurfaceViewLargerWidthDecodeAccuracy(DecodeAccuracyTest.java:218)

MTK 规避方案： media_codecs_c2.xml
Alias name=“OMX.MTK.VIDEO.DECODER.AVC”
Limit name=“size” min=“64x64” max=“2048x1088” 修改为2048x1088

=====================================================================================

CtsVideoTestCases
android.video.cts.VideoEncoderDecoderTest#testHevcOther0Perf0320x0240

junit.framework.AssertionFailedError: Expected achievable frame rates for c2.mtk.hevc.encoder video/hevc 320x240: [310.0, 681.0]. Measured frame rate: [123.10797174571141, 130.61042655947836].

修改帧率范围根据 log 提示

=====================================================================================

CtsMediaTestCases
android.media.cts.VideoDecoderPerfTest#testMpeg4Goog0Perf0480x0360
android.media.cts.VideoDecoderPerfTest#testMpeg4Goog0Perf1280x0720

arm64-v8a：
junit.framework.AssertionFailedError: Expected achievable frame rates for c2.android.mpeg4.decoder video/mp4v-es 480x360:[43.186956521739134, 95.12608695652175]. Measured frame rate:[215.5960576145203, 215.68285190914435].
armeabi-v7a：
junit.framework.AssertionFailedError: Expected achievable frame rates for c2.android.mpeg4.decoder video/mp4v-es 480x360:[43.186956521739134, 95.12608695652175]. Measured frame rate:[261.5961776889531, 261.0502049746129].

arm64-v8a：
junit.framework.AssertionFailedError: Expected achievable frame rates for c2.android.mpeg4.decoder video/mp4v-es 480x360: [43.186956521739134, 95.12608695652175].Measured frame rate: [215.5960576145203, 215.68285190914435].
armeabi-v7a：
junit.framework.AssertionFailedError: Expected achievable frame rates for c2.android.mpeg4.decoder video/mp4v-es 480x360: [43.186956521739134, 95.12608695652175].Measured frame rate: [261.5961776889531, 261.0502049746129].

与上面的类似

=====================================================================================

CtsLocationFineTestCases
android.location.cts.fine.LocationManagerFineTest#testAddProviderRequestListener

expected not to be: null at android.location.cts.fine.LocationManagerFineTest.testAddProviderRequestListener(LocationManagerFineTest.java:859)

google 8月份的安全补丁导致的，使用最新的 daily build tool (12_r6) 可以 pass

=====================================================================================

CtsSecurityTestCases
android.security.cts.CertificateTest#testNoRemovedCertificates

junit.framework.AssertionFailedError: Missing CA certificates expected:<[]> but was:<[证书sha256]>

CTS 有测项禁止OEM沒有告知 Google 就自己增减预置证书，原因是删除了过期的根证书

=====================================================================================

CtsGraphicsTestCases
android.graphics.drawable.cts.AdaptiveIconMaskTest#testDeviceConfig_iconMask_useRoundIcon

java.lang.AssertionError: expected:<false> but was:<true>
at org.junit.Assert.fail(Assert.java:89)
at org.junit.Assert.failNotEquals(Assert.java:835)
at org.junit.Assert.assertEquals(Assert.java:120)
at org.junit.Assert.assertEquals(Assert.java:146)
at android.graphics.drawable.cts.AdaptiveIconMaskTest.testDeviceConfig_iconMask_useRoundIcon(AdaptiveIconMaskTest.java:141)

桌面图标修改圆角

=====================================================================================

CtsIntentSignatureTestCases
android.signature.cts.intent.IntentTest#shouldNotFindUnexpectedIntents

java.lang.AssertionError: [Package: com.google.android.tts Invalid Intent: [android.intent.action.ACTION_PACKAGE_CHANGED]]

gms 包更新到 22 年 1 月份导致的，回退 SpeechServicesByGoogle apk 到 12 月份

=====================================================================================

CtsMediaProviderTranscodeTests
android.mediaprovidertranscode.cts.TranscodeTest#testSameTranscodedFile_ContentResolver
android.mediaprovidertranscode.cts.TranscodeTest#testOriginalCallingUid_modernAppPassLegacyAppUid
android.mediaprovidertranscode.cts.TranscodeTest#testExtraMediaCapabilitiesHevcUnspecifiedFalse_ContentResolver
android.mediaprovidertranscode.cts.TranscodeTest#testGetOriginalMediaFormatFileDescriptor_onSuccess_returnsOriginalFileDescriptor
android.mediaprovidertranscode.cts.TranscodeTest#testTranscodingCacheClear
android.mediaprovidertranscode.cts.TranscodeTest#testTranscodedCacheReuse_FilePath

FAILURE: android.system.ErrnoException: read failed: EFAULT (Bad address)
关键log：
TranscodeHelper: Transcoding wait end: /storage/emulated/0/DCIM/Camera/TranscodeTestHEVC_585499525975.mp4. Uid: 10103. Timeout: true. Success: false session: {id: 1, status: STATUS_RUNNING, result: RESULT_NONE, progress: 38}
Timeout: true

【Ago平台】
请申请patch:ALPS06359285
【非ago平台】
You can get a waiver bug id:225819430
Google的回复:
Our MediaProvider owner has merged the change into the internal branch and the Google built MediaProvider module will be made available in the future.
The waiver could be granted only if you are shipping a device that’s preloading Google built MediaProvider.
CTS_12_R3 工具waiver的，R4 工具并不waiver，最后申请 waiver，google 也同意了

=====================================================================================

CtsWifiTestCases
android.net.wifi.cts.WifiManagerTest#testTetheredBridgedApWifiForcedChannel
android.net.wifi.cts.WifiManagerTest#testTetheredBridgedAp

junit.framework.AssertionFailedError: SoftAp state and info on bridged AP mode are mismatch!!! shouldFallbackSingleApMode = false, isEnabled = true

测试之前请关闭wifi 扫描
Please turn off Settings->Location->Location services->Wi-Fi scanning.

=====================================================================================

CtsPermissionTestCases
android.permission.cts.NearbyDevicesPermissionTest#testRequestBluetoothPermissionNeverForLocation31_GrantNearby
android.permission.cts.NearbyDevicesRenouncePermissionTest#scanRenouncingInMiddleOfChainNotesBluetoothButNotLocation
android.permission.cts.NearbyDevicesRenouncePermissionTest#scanRenouncingLocationNotesBluetoothButNotLocation

FAILURE: expected: FILTERED but was : FULL

方法:
1.将手机恢复出厂设置
2.不连wifi,打开蓝牙

=====================================================================================

CtsTelephonyTestCases
android.telephony.cts.SmsUsageMonitorShortCodeTest#testSmsShortCodeDestination

fail: java.lang.AssertionError: country: cz number: 90902 expected:<4> but was:<0>

google更新服务器脚本，谷歌有一笔更新SMS codes的代码,请合入以下 googlepatch：
https://android-review.googlesource.com/c/platform/frameworks/base/+/1772567/3/core/res/res/xml/sms_short_codes.xml#96

=====================================================================================

需要插白卡测试的cases
CtsOmapiTestCases
CtsSimAppDialogTestCases
CtsCarrierApiTestCases
必须插入Google 定义的UICC SIM卡才可以pass.（插入卡后，重启手机，设为默认SIM卡）

=====================================================================================

2. STS 专区

StsHostTestCases [12_sts-r13]
android.security.sts.CVE_2021_0518#testPocCVE_2021_0518

由于合入2023年4月份的安全补丁导致，You Can get a waiver，bug id :274541381
Google 也会在 sts-14 工具进行修复，sts 工具的更新计划为每月的第一个周一，例如 sts-14 将在4月3日发布。