Android Too many open files, fd泄露

接到一个三方apk在我方手机运行异常的问题.

fatal exception信息如下:

05-13 18:31:34.485 13749 13749 E AndroidRuntime: FATAL EXCEPTION: main
05-13 18:31:34.485 13749 13749 E AndroidRuntime: Process: com.****.****, PID: 13749
05-13 18:31:34.485 13749 13749 E AndroidRuntime: java.lang.RuntimeException: Adding window failed
05-13 18:31:34.485 13749 13749 E AndroidRuntime: 	at android.view.ViewRootImpl.setView(ViewRootImpl.java:809)
05-13 18:31:34.485 13749 13749 E AndroidRuntime: 	at android.view.WindowManagerGlobal.addView(WindowManagerGlobal.java:356)
05-13 18:31:34.485 13749 13749 E AndroidRuntime: 	at android.view.WindowManagerImpl.addView(WindowManagerImpl.java:93)
05-13 18:31:34.485 13749 13749 E AndroidRuntime: 	at android.app.ActivityThread.handleResumeActivity(ActivityThread.java:4136)
05-13 18:31:34.485 13749 13749 E AndroidRuntime: 	at android.app.servertransaction.ResumeActivityItem.execute(ResumeActivityItem.java:51)
05-13 18:31:34.485 13749 13749 E AndroidRuntime: 	at android.app.servertransaction.TransactionExecutor.executeLifecycleState(TransactionExecutor.java:145)
05-13 18:31:34.485 13749 13749 E AndroidRuntime: 	at android.app.servertransaction.TransactionExecutor.execute(TransactionExecutor.java:70)
05-13 18:31:34.485 13749 13749 E AndroidRuntime: 	at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1872)
05-13 18:31:34.485 13749 13749 E AndroidRuntime: 	at android.os.Handler.dispatchMessage(Handler.java:106)
05-13 18:31:34.485 13749 13749 E AndroidRuntime: 	at android.os.Looper.loop(Looper.java:210)
05-13 18:31:34.485 13749 13749 E AndroidRuntime: 	at android.app.ActivityThread.main(ActivityThread.java:7124)
05-13 18:31:34.485 13749 13749 E AndroidRuntime: 	at java.lang.reflect.Method.invoke(Native Method)
05-13 18:31:34.485 13749 13749 E AndroidRuntime: 	at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:493)
05-13 18:31:34.485 13749 13749 E AndroidRuntime: 	at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:898)
05-13 18:31:34.485 13749 13749 E AndroidRuntime: Caused by: android.os.DeadObjectException: Transaction failed on small parcel; remote process probably died
05-13 18:31:34.485 13749 13749 E AndroidRuntime: 	at android.os.BinderProxy.transactNative(Native Method)
05-13 18:31:34.485 13749 13749 E AndroidRuntime: 	at android.os.BinderProxy.transact(BinderProxy.java:467)
05-13 18:31:34.485 13749 13749 E AndroidRuntime: 	at android.view.IWindowSession$Stub$Proxy.addToDisplay(IWindowSession.java:825)
05-13 18:31:34.485 13749 13749 E AndroidRuntime: 	at android.view.ViewRootImpl.setView(ViewRootImpl.java:797)
05-13 18:31:34.485 13749 13749 E AndroidRuntime: 	... 13 more

fatal exception中 添加窗口失败,看上去堆栈信息没有和三方apk有关的信息.

但是在报错之前,发现大量篇幅的Too many open files

05-13 18:30:02.923 13749 13864 I Adreno  : DequeueBuffer: dequeueBuffer failed
05-13 18:30:02.925 13749 13864 I Adreno  : DequeueBuffer: dequeueBuffer failed
05-13 18:30:02.925 13749 13864 I Adreno  : DequeueBuffer: dequeueBuffer failed
05-13 18:30:02.925 13749 13864 W OpenGLRenderer: swapBuffers encountered EGL error 12301 on 0x7ba7638700, halting rendering...
05-13 18:30:02.926 13749 14684 E PendingMediaStoreSerializer: File not found while getting output stream for 33746658248_pending_media.json.tmp
05-13 18:30:02.926 13749 14684 E PendingMediaStoreSerializer: java.io.FileNotFoundException: /data/user/0/com.*****.android/files/33746658248_pending_media.json.tmp (Too many open files)
05-13 18:30:02.926 13749 14684 E PendingMediaStoreSerializer: 	at java.io.FileOutputStream.open0(Native Method)
05-13 18:30:02.926 13749 14684 E PendingMediaStoreSerializer: 	at java.io.FileOutputStream.open(FileOutputStream.java:308)
05-13 18:30:02.926 13749 14684 E PendingMediaStoreSerializer: 	at java.io.FileOutputStream.<init>(FileOutputStream.java:238)
05-13 18:30:02.926 13749 14684 E PendingMediaStoreSerializer: 	at android.app.ContextImpl.openFileOutput(ContextImpl.java:581)
05-13 18:30:02.926 13749 14684 E PendingMediaStoreSerializer: 	at android.content.ContextWrapper.openFileOutput(ContextWrapper.java:208)
05-13 18:30:02.926 13749 14684 E PendingMediaStoreSerializer: 	at com.******.pendingmedia.store.PendingMediaStoreSerializer.A02(:109)
05-13 18:30:02.926 13749 14684 E PendingMediaStoreSerializer: 	at X.1dt.run(:8)
05-13 18:30:02.926 13749 14684 E PendingMediaStoreSerializer: 	at com.******.pendingmedia.model.PendingMedia.A0Q(:4)
05-13 18:30:02.926 13749 14684 E PendingMediaStoreSerializer: 	at X.1LL.A01(:71)
05-13 18:30:02.926 13749 14684 E PendingMediaStoreSerializer: 	at X.1LO.run(:39)
05-13 18:30:02.926 13749 14684 E PendingMediaStoreSerializer: 	at X.0Ms.run(:2)
05-13 18:30:02.926 13749 14684 E PendingMediaStoreSerializer: 	at X.0Mr.run(:31)
05-13 18:30:02.926 13749 14684 E PendingMediaStoreSerializer: 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1167)
05-13 18:30:02.926 13749 14684 E PendingMediaStoreSerializer: 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
05-13 18:30:02.926 13749 14684 E PendingMediaStoreSerializer: 	at java.lang.Thread.run(Thread.java:764)
05-13 18:30:02.926 13749 14684 E PendingMediaStoreSerializer: 	at X.0dQ.run(:5)

怀疑三方apk逻辑问题,导致死循环 打开文件导致次问题. 最后的crash只是压死骆驼的最后一根稻草. 真正的原因是前面的 文件相关操作.

并且发现Adreno  : DequeueBuffer: dequeueBuffer failed

更详细的判断如下:

1./proc/$pid/limits查看Max open files

# cat proc/8183/limits                                                                                                             
Limit                     Soft Limit           Hard Limit           Units     
Max cpu time              unlimited            unlimited            seconds   
Max file size             unlimited            unlimited            bytes     
Max data size             unlimited            unlimited            bytes     
Max stack size            unlimited            unlimited            bytes     
Max core file size        unlimited            unlimited            bytes     
Max resident set          unlimited            unlimited            bytes     
Max processes             21613                21613                processes 
Max open files            32768                32768                files     
Max locked memory         67108864             67108864             bytes     
Max address space         unlimited            unlimited            bytes     
Max file locks            unlimited            unlimited            locks     
Max pending signals       21613                21613                signals   
Max msgqueue size         819200               819200               bytes     
Max nice priority         40                   40                   
Max realtime priority     0                    0                    
Max realtime timeout      unlimited            unlimited            us

当一个进程打开的文件数超过最大限制,就会抛出各种异常. crash堆栈信息并不够准确

2.ls -la /proc/$pid/fd    查看进程在做什么

lr-x------ 1 u0_a168 u0_a168 64 2020-05-15 15:31 9997 -> /storage/8482-18E5/DCIM/100MEDIA/IMAG0010.jpg
lr-x------ 1 u0_a168 u0_a168 64 2020-05-15 15:31 9998 -> /storage/8482-18E5/DCIM/100MEDIA/IMAG0010.jpg
lr-x------ 1 u0_a168 u0_a168 64 2020-05-15 15:31 9999 -> /storage/8482-18E5/DCIM/100MEDIA/IMAG0010.jpg

这里发现应用fd 都指向同一个文件

3.借助tombstone文件

一般搜索anon_inode就可以了

fd 107: anon_inode:[eventpoll]             epoll创建的fd

    fd 103: /storage/8482-18E5/DCIM/100MEDIA/IMAG0001.jpg
    fd 104: /storage/8482-18E5/DCIM/100MEDIA/IMAG0001.jpg
    fd 105: /storage/8482-18E5/DCIM/100MEDIA/IMAG0001.jpg
    fd 106: anon_inode:[eventfd]
    fd 107: anon_inode:[eventpoll]
.........
    fd 32766: /storage/8482-18E5/DCIM/100MEDIA/IMAG0001.jpg
    fd 32767: /storage/8482-18E5/DCIM/100MEDIA/IMAG0001.jpg

从tombstone中看到  fd确实被耗尽.

尾注:

"Too many open files"
"Could not allocate JNI Env"
"Could not allocate dup blob fd"
"Could not read input channel file descriptors from parcel"
"pthread_create"
"InputChannel is not initialized"
"Could not open input channel pair"
当你看到上面几种crash的堆栈之后，就需要往fd泄露的方向上去思考了