crash工具分析dma设备内存踩踏(二)

最新推荐文章于 2024-02-17 18:07:20 发布

lunhui2016

最新推荐文章于 2024-02-17 18:07:20 发布

阅读量519

点赞数 2

分类专栏： linux Linux debug 文章标签： linux

本文为博主原创文章，未经博主允许不得转载。

本文链接：https://blog.csdn.net/lunhui2016/article/details/133824317

版权

linux 同时被 2 个专栏收录

14 篇文章 2 订阅

订阅专栏

Linux debug

7 篇文章 0 订阅

订阅专栏

背景介绍

背景介绍，看以下链接，由于上一次抓到现场，只有log，无法进行进一步定位和分析，因此安排重新测试复现，本次复现后，抓到log和kdump，结合kdump和vmlinux文件，通过crash工具分析该问题

crash工具分析dma设备内存踩踏(一)

问题log分析

抓到现场的关键log如下所示：

[76108.773766] Unable to handle kernel paging request at virtual address 1800180018021
[76108.773768] Mem abort info:
[76108.773771]   Exception class = DABT (current EL), IL = 32 bits
[76108.773772]   SET = 0, FnV = 0
[76108.773774]   EA = 0, S1PTW = 0
[76108.773775] Data abort info:
[76108.773777]   ISV = 0, ISS = 0x00000004
[76108.773778]   CM = 0, WnR = 0
[76108.773781] [0001800180018021] address between user and kernel address ranges
[76108.773785] Internal error: Oops: 96000004 [#1] PREEMPT SMP
[76108.773791] Modules linked in: bcmdhd hci_uart bluetooth ecdh_generic crypto_engine pvrsrvkm
[76108.773808] CPU: 1 PID: 1690 Comm: cameraTest Tainted: G           O    4.14.61-00010-gd1c8911-dirty #1
...
[76108.773812] task: ffff8000d99edc00 task.stack: ffff00000da20000
[76108.773822] PC is at skb_release_data+0x68/0x18c
[76108.773825] LR is at __kfree_skb+0x24/0x94
[76108.773827] pc : [<ffff000008b6af20>] lr : [<ffff000008b6a6f0>] pstate: 40400145
[76108.773829] sp : ffff00000800ba20
[76108.773830] x29: ffff00000800ba40 x28: ffff000009452bd0 
[76108.773835] x27: ffff00000942a4e0 x26: 000000000000002e 
[76108.773839] x25: ffff80005fa0888e x24: ffff800175f5f000 
[76108.773843] x23: 000000000000003c x22: ffff80005fa090b0 
[76108.773847] x21: 0000000000000000 x20: ffff80005fa09080 
[76108.773851] x19: ffff8000179c2300 x18: 0000ecbcf449c000 
[76108.773855] x17: 000000000000a759 x16: 0000000002986e63 
[76108.773859] x15: ffffffffffffffff x14: ffff000000000000 
[76108.773863] x13: ffffffffffffffff x12: 0000000000000028 
[76108.773867] x11: ffff00000800bb10 x10: 0000000000000880 
[76108.773871] x9 : ffff80005fa08800 x8 : 8001800180018001 
[76108.773875] x7 : ffff000008cf4d04 x6 : 0000000000000000 
[76108.773879] x5 : 0000000000000000 x4 : 0000000000000000 
[76108.773883] x3 : 0000000000000002 x2 : ffff000008cf4d3c 
[76108.773887] x1 : 0000000000000001 x0 : ffff8000179c2300 
[76108.773892] 
[76108.773892] X0: 0xffff8000179c2280:
[76108.773893] 2280  ffffffff 00000000 ffffffff ffffffff 0a227eb0 ffff0000 00000000 00000000
[76108.773906] 22a0  00000000 00000000 0926d5c6 ffff0000 00000000 00000000 00000000 00000000
[76108.773917] 22c0  00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[76108.773929] 22e0  00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[76108.773940] 2300  00000000 00000000 00000000 00000000 f13f2b5a 177738e9 00000000 00000000
[76108.773951] 2320  75f5f000 ffff8001 00000000 00000000 00000000 00000000 00000000 00000000
[76108.773963] 2340  00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[76108.773974] 2360  00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[76108.773988] 
[76108.773988] X9: 0xffff80005fa08780:
[76108.773989] 8780  80008000 80008000 80008000 80008000 80008000 80008000 80008000 80008000
[76108.774001] 87a0  80008000 80008000 80008000 80008000 80008000 80008000 80008000 80008000
[76108.774013] 87c0  80008000 80008000 80008000 80008000 80008000 80008000 80008000 80008000
[76108.774025] 87e0  80008000 80008000 80008000 80008000 80008000 80008000 80008000 80008000
[76108.774036] 8800  80008000 80008000 80008000 80008000 80008000 80008000 80008000 80008000
[76108.774048] 8820  80008000 80008000 80008000 80008000 80008000 80008000 80008000 80008000
[76108.774060] 8840  80008000 80008000 80008000 80008000 80008000 80008000 80008000 80008000
[76108.774072] 8860  80008000 80008000 80008000 80008000 80008000 80008000 80008000 80008000
[76108.774086] 
[76108.774086] X19: 0xffff8000179c2280:
[76108.774087] 2280  ffffffff 00000000 ffffffff ffffffff 0a227eb0 ffff0000 00000000 00000000
[76108.774099] 22a0  00000000 00000000 0926d5c6 ffff0000 00000000 00000000 00000000 00000000
[76108.774111] 22c0  00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[76108.774122] 22e0  00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[76108.774133] 2300  00000000 00000000 00000000 00000000 f13f2b5a 177738e9 00000000 00000000
[76108.774145] 2320  75f5f000 ffff8001 00000000 00000000 00000000 00000000 00000000 00000000
[76108.774156] 2340  00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[76108.774167] 2360  00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[76108.774180] 
[76108.774180] X20: 0xffff80005fa09000:
[76108.774181] 9000  80018001 80018001 80018001 80018001 80018001 80018001 80018001 80018001
[76108.774192] 9020  80018001 80018001 80018001 80018001 80018001 80018001 80018001 80018001
[76108.774204] 9040  80018001 80018001 80018001 80018001 80018001 80018001 80018001 80018001
[76108.774216] 9060  80018001 80018001 80018001 80018001 80018001 80018001 80018001 80018001
[76108.774227] 9080  80018001 80018001 80018001 80018001 80018001 80018001 80018001 80018001
[76108.774239] 90a0  80018001 80018001 80018001 80018001 80018001 80018001 80018001 80018001
[76108.774251] 90c0  80018001 80018001 80018001 80018001 80018001 80018001 80018001 80018001
[76108.774262] 90e0  80018001 80018001 80018001 80018001 80018001 80018001 80018001 80018001
[76108.774275] 
[76108.774275] X22: 0xffff80005fa09030:
[76108.774276] 9030  80018001 80018001 80018001 80018001 80018001 80018001 80018001 80018001
[76108.774288] 9050  80018001 80018001 80018001 80018001 80018001 80018001 80018001 80018001
[76108.774299] 9070  80018001 80018001 80018001 80018001 80018001 80018001 80018001 80018001
[76108.774311] 9090  80018001 80018001 80018001 80018001 80018001 80018001 80018001 80018001
[76108.774323] 90b0  80018001 80018001 80018001 80018001 80018001 80018001 80018001 80018001
[76108.774334] 90d0  80018001 80018001 80018001 80018001 80018001 80018001 80018001 80018001
[76108.774346] 90f0  80018001 80018001 80018001 80018001 80018001 80018001 80018001 80018001
[76108.774358] 9110  80018001 80018001 80018001 80018001 80018001 80018001 80018001 80018001
[76108.774370] 
[76108.774370] X24: 0xffff800175f5ef80:
[76108.774372] ef80  00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[76108.774383] efa0  00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[76108.774395] efc0  00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[76108.774406] efe0  00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[76108.774417] f000  6e616c77 00000030 00000000 00000000 00000000 00000000 77ec7bd0 ffff8001
[76108.774429] f020  00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[76108.774440] f040  00000000 00000000 00000003 00000000 52833050 ffff8001 77c3b050 ffff8001
[76108.774452] f060  76076800 ffff8001 76076800 ffff8001 75f5f070 ffff8001 75f5f070 ffff8001
[76108.774464] 
[76108.774464] X25: 0xffff80005fa0880e:
[76108.774465] 880c  80008000 80008000 80008000 80008000 80008000 80008000 80008000 80008000
[76108.774477] 882c  80008000 80008000 80008000 80008000 80008000 80008000 80008000 80008000
[76108.774489] 884c  80008000 80008000 80008000 80008000 80008000 80008000 80008000 80008000
[76108.774501] 886c  80008000 80008000 80008000 80008000 80008000 ffffffff c73cffff fb6aa686
[76108.774512] 888c  01009899 3b000000 04000000 00000000 00000000 00000000 00000000 00000000
[76108.774524] 88ac  00000000 00000000 00000000 00000000 80008000 80008000 80008000 80008000
[76108.774535] 88cc  80008000 80008000 80008000 80008000 80008000 80008000 80008000 80008000
[76108.774547] 88ec  80008000 80008000 80008000 80008000 80008000 80008000 80008000 80008000
[76108.774558] 890c  80008000 80008000 80008000 80008000 80008000 80008000 80008000 80008000
[76108.774572] 
[76108.774574] Process cameraTest (pid: 1690, stack limit = 0xffff00000da20000)
[76108.774576] Call trace:
[76108.774579] Exception stack(0xffff00000800b8e0 to 0xffff00000800ba20)
[76108.774583] b8e0: ffff8000179c2300 0000000000000001 ffff000008cf4d3c 0000000000000002
[76108.774586] b900: 0000000000000000 0000000000000000 0000000000000000 ffff000008cf4d04
[76108.774589] b920: 8001800180018001 ffff80005fa08800 0000000000000880 ffff00000800bb10
[76108.774593] b940: 0000000000000028 ffffffffffffffff ffff000000000000 ffffffffffffffff
[76108.774596] b960: 0000000002986e63 000000000000a759 0000ecbcf449c000 ffff8000179c2300
[76108.774599] b980: ffff80005fa09080 0000000000000000 ffff80005fa090b0 000000000000003c
[76108.774602] b9a0: ffff800175f5f000 ffff80005fa0888e 000000000000002e ffff00000942a4e0
[76108.774605] b9c0: ffff000009452bd0 ffff00000800ba40 ffff000008b6a6f0 ffff00000800ba20
[76108.774609] b9e0: ffff000008b6af20 0000000040400145 ffff00000800ba50 ffff000008156ae0
[76108.774612] ba00: 0000ffffffffffff ffff00000942a4e0 ffff00000800ba40 ffff000008b6af20
[76108.774616] [<ffff000008b6af20>] skb_release_data+0x68/0x18c
[76108.774619] [<ffff000008b6a6f0>] __kfree_skb+0x24/0x94
[76108.774622] [<ffff000008b6ac5c>] consume_skb+0x1a8/0x1ec
[76108.774628] [<ffff000008cf539c>] packet_rcv+0x4c/0x3a4
[76108.774633] [<ffff000008b8a938>] __netif_receive_skb_core+0x954/0xabc
[76108.774637] [<ffff000008b8355c>] netif_receive_skb_internal+0x228/0x2e0
[76108.774640] [<ffff000008b83314>] netif_receive_skb+0x194/0x1b4
[76108.774689] [<ffff00000106d420>] dhd_rx_frame+0xe8c/0xf90 [bcmdhd]
[76108.774740] [<ffff00000117b474>] dhd_napi_poll+0x184/0x208 [bcmdhd]
[76108.774744] [<ffff000008b8b8dc>] net_rx_action+0x10c/0x4f8
[76108.774749] [<ffff000008081d38>] __do_softirq+0x348/0x5c8
[76108.774753] [<ffff0000080e5af0>] irq_exit+0xec/0xf8
[76108.774757] [<ffff000008092a40>] handle_IPI+0x1c8/0x314
[76108.774760] [<ffff00000808181c>] gic_handle_irq+0xa0/0xb8
[76108.774762] Exception stack(0xffff00000da234f0 to 0xffff00000da23630)
[76108.774765] 34e0:                                   ffff000008170950 0000000000000001
[76108.774768] 3500: ffff0000084e0360 0000000000000000 0000000000000001 0000000000000080
[76108.774771] 3520: 0000000000000000 ffff0000084e1d44 0000000000000000 0000000000000000
[76108.774775] 3540: ffffffffffffffff 0000000000000000 0000000000000000 ffff00000b6a0c70
[76108.774778] 3560: 34203d207865646e 0000000000000001 ffff000008e35234 0000000000000041
[76108.774781] 3580: 0000ecbcf449c000 ffff00000941a018 ffff00000a1c8000 ffff00000a1c6000
[76108.774784] 35a0: 0000000000000140 ffff000008170278 0000000000000000 0000000000000038
[76108.774788] 35c0: ffff00000944de80 0000000000000001 ffff000009476760 ffff00000da236c0
[76108.774791] 35e0: ffff000008170950 ffff00000da23630 ffff000008170954 0000000060400145
[76108.774794] 3600: ffff00000a1c8000 ffff00000941a018 0000ffffffffffff ffff000008153478
[76108.774796] 3620: ffff00000da236c0 ffff000008170954
[76108.774800] [<ffff000008083234>] el1_irq+0xb4/0x12c
[76108.774805] [<ffff000008170954>] console_unlock+0x600/0x774
[76108.774808] [<ffff000008170278>] vprintk_emit+0x2c4/0x304
[76108.774813] [<ffff00000875d75c>] dev_vprintk_emit+0x194/0x1d4
[76108.774816] [<ffff00000875d7f8>] dev_printk_emit+0x5c/0x68
[76108.774819] [<ffff00000875a998>] _dev_info+0x9c/0xd8
[76108.774824] [<ffff000008a02cf4>] sdrv_dummy_buf_init+0xa0/0x128
[76108.774829] [<ffff0000089dce60>] __vb2_queue_alloc+0x374/0x494
[76108.774832] [<ffff0000089dc140>] vb2_core_reqbufs+0x274/0x440
[76108.774835] [<ffff0000089e1be4>] vb2_ioctl_reqbufs+0x6c/0x94
[76108.774840] [<ffff0000089c6974>] v4l_reqbufs+0x48/0x58
[76108.774843] [<ffff0000089c43a4>] __video_do_ioctl+0x130/0x26c
[76108.774845] [<ffff0000089c3d80>] video_usercopy+0x310/0x7e8
[76108.774848] [<ffff0000089c426c>] video_ioctl2+0x14/0x1c
[76108.774851] [<ffff0000089c3540>] v4l2_ioctl+0xa0/0xc8
[76108.774856] [<ffff0000082f5024>] do_vfs_ioctl+0x5e0/0x8e0
[76108.774859] [<ffff0000082f5490>] SyS_ioctl+0x88/0x94
[76108.774862] Exception stack(0xffff00000da23ec0 to 0xffff00000da24000)
[76108.774865] 3ec0: 0000000000000007 00000000c0145608 0000f62f87dfebf0 0000bce56bd9ada0
[76108.774868] 3ee0: 0000bce56bd9adbb 0000f62f86224048 7373656363757320 21796c6c75667373
[76108.774872] 3f00: 000000000000001d 0000f62f87dfeba8 0000f62f87dfeb70 0000f62f87dfeba8
[76108.774875] 3f20: 0000f62f87dfebf0 0000000000000000 0000000000000010 0000f62f8725940a
[76108.774878] 3f40: 0000bce56bda2810 0000f62f871fa308 0000f62f85394000 0000f62f87dff020
[76108.774881] 3f60: 0000f62f84656000 0000f62f87dfec44 0000f62f87dff020 00000000000004c0
[76108.774885] 3f80: 000000000000024c 0000f62f87dfed50 0000f62f87dff020 0000f62f87ec5020
[76108.774888] 3fa0: 0000000000000000 0000f62f87dfebe0 0000f62f871fa390 0000f62f87dfeaf0
[76108.774891] 3fc0: 0000f62f8723c888 00000000a0000000 0000000000000007 000000000000001d
[76108.774894] 3fe0: 0000000000000000 0000000000000000 0000000000000000 0000000000000000
[76108.774897] [<ffff000008083ac0>] el0_svc_naked+0x34/0x38
[76108.774902] Code: 340002c8 aa1f03f5 9100c296 f94002c8 (f9401109) 
[76108.774913] SMP: stopping secondary CPUs
[76108.774921] ---[ end trace 0c998bce55eae5b7 ]---

通过x20/x22寄存器保存的地址的数据规律来看，都是填充了0x80018001，基本可以确认跟第一次抓到的问题是同一个问题了，踩踏的数据规律是一致的。

结合现场log堆栈信息，使用crash工具反编译最后一个函数skb_release_data()如下：

crash> dis skb_release_data
0xffff000008b6aeb8 <skb_release_data>:  stp     x22, x21, [sp, #-48]!
0xffff000008b6aebc <skb_release_data+4>:        stp     x20, x19, [sp, #16]
0xffff000008b6aec0 <skb_release_data+8>:        stp     x29, x30, [sp, #32]
0xffff000008b6aec4 <skb_release_data+12>:       add     x29, sp, #0x20
0xffff000008b6aec8 <skb_release_data+16>:       ldr     x9, [x0, #208]
0xffff000008b6aecc <skb_release_data+20>:       ldr     w10, [x0, #204] //skb buffer中拿到指向skb_shared_info这块buffer的指针
0xffff000008b6aed0 <skb_release_data+24>:       ldrb    w8, [x0, #142]
0xffff000008b6aed4 <skb_release_data+28>:       mov     x19, x0
0xffff000008b6aed8 <skb_release_data+32>:       add     x20, x9, x10
0xffff000008b6aedc <skb_release_data+36>:       tbz     w8, #0, 0xffff000008b6af0c <skb_release_data+84>
0xffff000008b6aee0 <skb_release_data+40>:       tst     w8, #0x2
0xffff000008b6aee4 <skb_release_data+44>:       mov     w8, #0x10001                    // #65537
0xffff000008b6aee8 <skb_release_data+48>:       csinc   w8, w8, wzr, ne  // ne = any
0xffff000008b6aeec <skb_release_data+52>:       add     x9, x20, #0x24
0xffff000008b6aef0 <skb_release_data+56>:       prfm    pstl1strm, [x9]
0xffff000008b6aef4 <skb_release_data+60>:       ldxr    w10, [x9]
0xffff000008b6aef8 <skb_release_data+64>:       sub     w10, w10, w8
0xffff000008b6aefc <skb_release_data+68>:       stlxr   w11, w10, [x9]
0xffff000008b6af00 <skb_release_data+72>:       cbnz    w11, 0xffff000008b6aef4 <skb_release_data+60>
0xffff000008b6af04 <skb_release_data+76>:       dmb     ish
0xffff000008b6af08 <skb_release_data+80>:       cbnz    w10, 0xffff000008b6b014 <skb_release_data+348>
0xffff000008b6af0c <skb_release_data+84>:       ldrb    w8, [x20, #2]
0xffff000008b6af10 <skb_release_data+88>:       cbz     w8, 0xffff000008b6af68 <skb_release_data+176>
0xffff000008b6af14 <skb_release_data+92>:       mov     x21, xzr
0xffff000008b6af18 <skb_release_data+96>:       add     x22, x20, #0x30
0xffff000008b6af1c <skb_release_data+100>:      ldr     x8, [x22]            //x22 = 0xffff80005fa090b0
0xffff000008b6af20 <skb_release_data+104>:      ldr     x9, [x8, #32]       //panic， x8=8001800180018001
...

从汇编分析，是X22寄存器保存的指针指向的buffer被填充成0x80001, X0和X19保存着skb_release_data(struct sk_buff *skb)的参数，源码如下所示：

static void skb_release_data(struct sk_buff *skb)
{
	struct skb_shared_info *shinfo = skb_shinfo(skb);
	int i;

	if (skb->cloned &&
	    atomic_sub_return(skb->nohdr ? (1 << SKB_DATAREF_SHIFT) + 1 : 1,
			      &shinfo->dataref))
		return;

	for (i = 0; i < shinfo->nr_frags; i++)
		__skb_frag_unref(&shinfo->frags[i]);

	if (shinfo->frag_list)
		kfree_skb_list(shinfo->frag_list);

	skb_zcopy_clear(skb, true);
	skb_free_head(skb);
}

skb_shinfo(skb)就是从skb buffer中拿到指向skb_shared_info这块buffer的指针，如下所示：

#ifdef NET_SKBUFF_DATA_USES_OFFSET
static inline unsigned char *skb_end_pointer(const struct sk_buff *skb)
{
	return skb->head + skb->end;
}

static inline unsigned int skb_end_offset(const struct sk_buff *skb)
{
	return skb->end;
}
#else
static inline unsigned char *skb_end_pointer(const struct sk_buff *skb)
{
	return skb->end;
}

static inline unsigned int skb_end_offset(const struct sk_buff *skb)
{
	return skb->end - skb->head;
}
#endif

/* Internal */
#define skb_shinfo(SKB)	((struct skb_shared_info *)(skb_end_pointer(SKB)))

用crash工具查看这个skb buffer的数据如下：

crash> struct sk_buff -x ffff8000179c2300
struct sk_buff {
  ...
  end = 0x880,            //offset = 204
  head = 0xffff80005fa08800 "",  //offset = 208
...
}

用sk_buffer这块内存的数据可以算出x20 = 0xffff80005fa08800 + 0x880 = 0xffff80005fa09080， X22 = X20 + 0x30 = 0xffff80005fa090b0，因此可以分析，x20保存了shinfo这个指针的值，x22是保存了&shinfo->frags[i]的值，用crash工具查看shinfo这块buffer的内容如下：

crash> rd 0xffff80005fa09080 0x10
ffff80005fa09080:  8001800180018001 8001800180018001   ................
ffff80005fa09090:  8001800180018001 8001800180018001   ................
ffff80005fa090a0:  8001800180018001 8001800180018001   ................
ffff80005fa090b0:  8001800180018001 8001800180018001   ................
ffff80005fa090c0:  8001800180018001 8001800180018001   ................
ffff80005fa090d0:  8001800180018001 8001800180018001   ................
ffff80005fa090e0:  8001800180018001 8001800180018001   ................
ffff80005fa090f0:  8001800180018001 8001800180018001   ................

crash> vtop 0xffff80005fa09080
VIRTUAL           PHYSICAL        
ffff80005fa09080  9fa09080        

PAGE DIRECTORY: ffff00000a278000
   PGD: ffff00000a278800 => 1bfff7803
   PUD: ffff80017fff7008 => e8000080000f11
   PMD: ffff8000400007e8 => 1400000b34ffff8c
vtop: read error: kernel virtual address: ffff800af4fff000  type: "page table"

crash> struct skb_shared_info 0xffff80005fa09080
struct skb_shared_info {
  _unused = 32769,
  nr_frags = 1 '\001',
  tx_flags = 128 '\200',
  gso_size = 32769,            //这个是0x8001
  gso_segs = 32769,
  frag_list = 0x8001800180018001,
  hwtstamps = {
    hwtstamp = -9222949817947160575
  },
  gso_type = 2147581953,
  tskey = 2147581953,            //这个是0x80018001
  ip6_frag_id = 2147581953,
  dataref = {
    counter = -2147385343
  },
  destructor_arg = 0x8001800180018001,
  frags = {{
      page = {
        p = 0x8001800180018001
      },
      page_offset = 2147581953,
      size = 2147581953
    }, {
      page = {
        p = 0x8001800180018001
      },
      page_offset = 2147581953,
      size = 2147581953
    }, {
      page = {
        p = 0x8001800180018001
      },
      page_offset = 2147581953,
      size = 2147581953
    }, {
      page = {
        p = 0x8001800180018001
      },
      page_offset = 2147581953,
      ...

在skb_release_data(struct sk_buff *skb)函数中，会调用__skb_frag_unref()去释放相关page的引用计数(skb_shared_inf.frags[]数组保存了特定格式(page、page_offset、length)的数据)

/**
 * skb_frag_page - retrieve the page referred to by a paged fragment
 * @frag: the paged fragment
 *
 * Returns the &struct page associated with @frag.
 */
static inline struct page *skb_frag_page(const skb_frag_t *frag)
{
	return frag->page.p;
}

/**
 * __skb_frag_unref - release a reference on a paged fragment.
 * @frag: the paged fragment
 *
 * Releases a reference on the paged fragment @frag.
 */
static inline void __skb_frag_unref(skb_frag_t *frag)
{
	put_page(skb_frag_page(frag));
}

结合源码，可以看出，是struct skb_shared_info *shinfo这块buffer数据被破坏了。用crash工具查看这块物理内存对应的page信息：

kmem -p > mem.info
...
ffff7e00017e8000  9fa00000                0        0 12 fffc00000008000 head
ffff7e00017e8040  9fa01000 dead0000ffffffff        4  0 fffc00000000000
ffff7e00017e8080  9fa02000 dead000000000400        5  0 fffc00000000000
ffff7e00017e80c0  9fa03000 dead000000000400        5  0 fffc00000000000
ffff7e00017e8100  9fa04000 dead000000000400        4  0 fffc00000000000
ffff7e00017e8140  9fa05000 dead000000000400        4  0 fffc00000000000
ffff7e00017e8180  9fa06000 dead000000000400        4  0 fffc00000000000
ffff7e00017e81c0  9fa07000 dead000000000400        4  0 fffc00000000000
ffff7e00017e8200  9fa08000                0        0  7 fffc00000008000 head
ffff7e00017e8240  9fa09000 dead0000ffffffff        4  0 fffc00000000000
ffff7e00017e8280  9fa0a000 dead000000000400        4  0 fffc00000000000
ffff7e00017e82c0  9fa0b000 dead000000000400        4  0 fffc00000000000
ffff7e00017e8300  9fa0c000 dead000000000400        4  0 fffc00000000000
ffff7e00017e8340  9fa0d000 dead000000000400        4  0 fffc00000000000
ffff7e00017e8380  9fa0e000 dead000000000400        4  0 fffc00000000000
ffff7e00017e83c0  9fa0f000 dead000000000400        4  0 fffc00000000000
ffff7e00017e8400  9fa10000                0        4  0 fffc00000000000
...

可以看出，0x9fa00000和0x9fa08000这两个是一个Compound page(复合页，多个page聚集在一起)，从log看0x9fa00000是cam8之前使用的，抓一下这块的memory数据如下：

ffff80005fa08700:  8000800080008000 8000800080008000   ................
...
ffff80005fa08c80:  8000800080008000 8000800080008000   ................
ffff80005fa08c90:  8000800080008000 8000800080008000   ................
ffff80005fa08ca0:  8001800180018001 8001800180018001   ................
...
ffff80005fa09220:  8001800180018001 8001800180018001   ................
ffff80005fa09230:  8001800180018001 8001800180018001   ................
...
ffff80005fab2980:  7034c0347034c034 7034c0347034c034   4.4p4.4p4.4p4.4p
...
ffff80005fae0fe0:  7034c0347034c034 7034c0347034c034   4.4p4.4p4.4p4.4p
ffff80005fae0ff0:  7034c0347034c034 7034c0347034c034   4.4p4.4p4.4p4.4p

可以看到数据分布的规律：0x80008000、0x80018001、0x7034c034(软件填充的蓝屏数据)，且size刚好满足720 * 640中的一行(720)的数据，从log看0x9fa00000是camera之前使用的，已经释放了，基本可以说明是camera驱动分配的内存可能出现user after free导致。

进行以下实验证实该问题：

camera多申请几个buffer来进行使用流转，这些buffer用完后，不去free，人为的把这些buffer进行memset(buf, 0xa5a5, size);，再去check这些用完的buffer数据是否发生变化即可。