本文阐述vxWorks平台下的交换机部分设备上电运行之后ping死机调试过程,通过这个分享体现技术人员的不容易,一个现象可能出现的现象千奇百怪,只有不断的探索才能找到真正的问题;有时候真的是众里寻他千百度,那个bug就在程序开始处。一下就是我定位调试的这个过程,我觉得用这种问答式的方法,可以高效的定位。话不多说,分享走起。
问题现象:
AT91+BCM53262+vxWorks平台,部分设备烧写软件之后上电在网络启动之后出现死机现;而烧录UT-62208F系列软件到CPU板在UT62208F设备上运行正常;
测试1:
用光示波器测设备运行的信号,出现死机之前DDR未获取到信号;
猜测1:
DDR驱动问题
调试1:
移植UT-62208F的DDR软件驱动到UT-62424F平台
1)移植rominit函数到UT-62424F
2)移植sysint函数到UT-62424F
结果1:
一样的死机现象
调试1.1:
修改UT-62208F的内存分区与UT-62424F一样
1)修改config.h文件
#define RAM_LOW_ADRS 0x20004000 //0x20006000 /* VxWorks image entry point */
#define RAM_HIGH_ADRS 0x23000000 //0x20e00000 /* RAM address for ROM boot */
等内存段相关宏。
2)修改Makefile
RAM_LOW_ADRS = 20004000 # RAM text/data address
RAM_HIGH_ADRS = 23000000 # RAM text/data address
注意:config.h中的RAM_LOW_ADRS、RAM_HIGH_ADRS必须保持一致;
结果1.1:
一样的死机现象
测试2:
在异常的CPU板子上面,uboot起来之后通过网络加载vxWorks镜像时候加载过程中也出现死机。
猜测2:
网卡设备驱动问题
调试2:
1)先从uboot入手,清UT-62424F的启动流程与UT-62208F一致
sysClkConnect( );
|
sysHwInit2( );
|
2)移植UT-62208F的网卡设备驱动到UT-62424F平台
62408F:网络驱动
at91EndStart-->board_eth_init-->macb_eth_initialize-->at91_macb_init=macb_init
问题1:
EMAC的中断是如何传递到中断控制器的(EMAC中断之后如何知道传递到那个中断向量)
问题2:
master_clock是如何产生,在UT-62208F和UT-62424F中具体是多少?
UT-62424F时钟相关寄存器:
PIT_MR:
PIV=11331 PITEN=1 PITIEN=1
含义:
period = 11332
Period Interval Timer Enabled
The bit PITS in PIT_SR asserts interrupt.
CKGR_MOR
MOSCXTEN=1 MOSCXTBY=0 MOSCRCEN=1 MOSCXTST=0 KEY=0 MOSCSEL=1 CFDEN=0
含义:
The Main Crystal Oscillator is enabled
The Main On-Chip RC Oscillator is enabled
Main Crystal Oscillator Start-up Time = 0
Should be written at value 0x37. Writing any other value in this field aborts the write operation.
The Main Crystal Oscillator is selected
The Clock Failure Detector is disabled
使用芯片外部主时钟,主时钟启动时间为0;
CKGR_MCFR
MAINF=5860 MAINFRDY=1
含义:
Main Clock = 32768*5860/16=12001280 =12M
CKGR_PLLAR
DIVA=1 PLLACOUNT=17 OUTA=0 MULA=176
计算:PLLACK= Main Clock *(MULA+1) =12M* 177 = 2124M
PMC_MCKR
CSS=2 PRES=2 MDIV=1 PLLADIV2=1
计算:Master Clock= PLLACK/PRES/MIV = 2124M / 4/2/2 = 132M
含义:
PLLACK/PLLADIV2 is selected
Selected clock divided by 4
Master Clock is Prescaler Output Clock divided by 2; SysClk DDR is equal to 2 x MCK. DDRCK is equal to MCK
PLLA clock frequency is divided by 1
PMC_SR
MOSCXTS=1 LOCKA=1 MCKRDY=1 LOCKU=0 OSCSELS=0 PCKRDY0=0 PCKRDY1=0 MOSCSELS=1 MOSCRCS=1 CFDEV=0 CFDS=0 FOS=0
含义:
Main XTAL oscillator is stabilized
PLLA is locked
Master Clock is ready
UPLL Clock is not ready
Internal slow clock RC oscillator is selected
Programmable Clock x is not ready
Selection is done
Main on-chip RC oscillator is stabilized
SCKCR
RCEN=0 OSC32EN=1 OSC32BYP=0 OSCSEL=1
含义:
使用32,768Hz slow clock
DBGU_BRGR
CD=45
UT-62208F时钟相关寄存器:
PIT_MR
PIV=82559 PITEN=1 PITIEN=1
CKGR_MOR
MOSCXTEN=1 MOSCXTBY=0 MOSCRCEN=1 MOSCXTST=0 KEY=0 MOSCSEL=1 CFDEN=0
含义:
The Main Crystal Oscillator is enabled
The Main On-Chip RC Oscillator is enabled
Main Crystal Oscillator Start-up Time = 0
Should be written at value 0x37. Writing any other value in this field aborts the write operation.
The Main Crystal Oscillator is selected
The Clock Failure Detector is disabled
使用芯片外部主时钟,主时钟启动时间为0;
CKGR_MCFR
MAINF=6372 MAINFRDY=1
Main Clock = 32000*6372 /16=12744000= 12.7M
CKGR_PLLAR
DIVA=1 PLLACOUNT=17 OUTA=0 MULA=66
计算:PLLACK= Main Clock *(MULA+1) =12M*67=804M
PMC_MCKR
CSS=2 PRES=1 MDIV=2 PLLADIV2=0
计算:Master Clock= PLLACK/PRES/MIV = 804 / 2/4 = 100.5M
PMC_SR
MOSCXTS=1 LOCKA=1 MCKRDY=1 LOCKU=0 OSCSELS=0 PCKRDY0=0 PCKRDY1=0 MOSCSELS=1 MOSCRCS=1 CFDEV=0 CFDS=0 FOS=0
Main XTAL oscillator is stabilized
PLLA is locked
Master Clock is ready
UPLL Clock is not ready
Internal slow clock RC oscillator is selected
Programmable Clock x is not ready
Selection is done
Main on-chip RC oscillator is stabilized
SCKCR
RCEN=1 OSC32EN=0 OSC32BYP=0 OSCSEL=0
含义:
使用内部32KHz RC oscillator
DBGU_BRGR: CD=54
方案1:修改UT-62424F的时钟相关寄存器与UT-62208F一致;
替换UT-62208F的Bootstrap的lowlevel_init.S中”PMC Init“代码到UT-62424F中
PMC_MCKR:PMC Master Clock Register
寄存器的值:CSS=2 PRES=1 MDIV=2 PLLADIV2=0
/*----------------------------------------------------------------------------
* PMC Init Step 1.
* ----------------------------------------------------------------------------
* Setup MCKR
* ----------------------------------------------------------------------------
*/
/* -Master Clock Controller register PMC_MCKR */
ldr r1,=PMC_MCKR
/* Selected clock divided by 2 */
ldr r0,=0x01112
str r0,[r1]
/* Reading the PMC Status to detect when the Master clock is ready */
ldr r2,=PMC_SR
mov r4,#8
MCKRDY_Loop00:
ldr r3,[r2]
and r3,r4,r3
cmp r3,#8
bne MCKRDY_Loop00
/* -Master Clock Controller register PMC_MCKR */
ldr r1,=PMC_MCKR
/* Master Clock is Prescaler Output Clock divided by 4. */
ldr r0,=0x01212
str r0,[r1]
/* Reading the PMC Status to detect when the Master clock is ready */
ldr r2,=PMC_SR
mov r4,#8
MCKRDY_Loop006:
ldr r3,[r2]
and r3,r4,r3
cmp r3,#8
bne MCKRDY_Loop006
/* -Master Clock Controller register PMC_MCKR */
ldr r1,=PMC_MCKR
/* PLLA clock frequency is divided by 1. */
ldr r0,=0x00212
str r0,[r1]
/* Reading the PMC Status to detect when the Master clock is ready */
ldr r2,=PMC_SR
mov r4,#8
MCKRDY_Loop007:
ldr r3,[r2]
and r3,r4,r3
cmp r3,#8
bne MCKRDY_Loop007
CKGR_PLLAR:PMC Clock Generator PLLA Registe
寄存器的值:DIVA=1 PLLACOUNT=17 OUTA=0 MULA=66
/*----------------------------------------------------------------------------
* PMC Init Step 2.
* ----------------------------------------------------------------------------
* Setup PLLA
* ----------------------------------------------------------------------------
*/
/* PLLA Register */
ldr r1,=CKGR_PLLAR
/* Divider is bypassed */
ldr r0,=0x20421101
str r0, [r1]
/* PLL Charge Pump Current Register */
ldr r1, =PMC_PLLICPR
mov r3,#0
str r3,[r1]
/* Reading the PMC Status register to detect when the PLLA is locked */
ldr r2,=PMC_SR
mov r4, #2
MOSCS_Loop1:
ldr r3, [r2]
and r3, r4, r3
cmp r3, #2
bne MOSCS_Loop1
结果1:
乱码,autoboot网络加载死机
方案2:修改UT-62424F的DDR配置与UT-62208F一致;
替换UT-62208F的Bootstrap的lowlevel_init.S中”memory control configuration“代码到UT-62424F中
结果2:
乱码,但是autoboot网络加载成功
调试2.1:计算MCK值
PLLACK= Main Clock *(MULA+1) =12M*67=804M
Master Clock= PLLACK/PRES/MIV = 804 / 2/4 = 100.5M
修改bootrom:at91.h---> #define AT91_MCK_RATE 88000000
修改为:define AT91_MCK_RATE 100000000
结果2.1:乱码,但是autoboot网络加载成功
方案3:修改UT-62424F的DDR配置与UT-62208F一致,时钟相关寄存器保持原来的值
PMC_MCKR:PMC Master Clock Register
寄存器的值:CSS=2 PRES=2 MDIV=1 PLLADIV2=1
/*----------------------------------------------------------------------------
* PMC Init Step 1.
* ----------------------------------------------------------------------------
* Setup MCKR
* ----------------------------------------------------------------------------
*/
/* -Master Clock Controller register PMC_MCKR */
ldr r1,=PMC_MCKR
/* Selected clock divided by 2 */
//ldr r0,=0x01112
ldr r0,=0x01122/*zhoulinhua@UTEK,2018-03-08*/
str r0,[r1]
/* Reading the PMC Status to detect when the Master clock is ready */
ldr r2,=PMC_SR
mov r4,#8
MCKRDY_Loop00:
ldr r3,[r2]
and r3,r4,r3
cmp r3,#8
bne MCKRDY_Loop00
/* -Master Clock Controller register PMC_MCKR */
ldr r1,=PMC_MCKR
/* Master Clock is Prescaler Output Clock divided by 4. */
//ldr r0,=0x01212
ldr r0,=0x01122/*zhoulinhua@UTEK,2018-03-08*/
str r0,[r1]
/* Reading the PMC Status to detect when the Master clock is ready */
ldr r2,=PMC_SR
mov r4,#8
MCKRDY_Loop006:
ldr r3,[r2]
and r3,r4,r3
cmp r3,#8
bne MCKRDY_Loop006
/* -Master Clock Controller register PMC_MCKR */
ldr r1,=PMC_MCKR
/* PLLA clock frequency is divided by 1. */
//ldr r0,=0x00212
/* PLLA clock frequency is divided by 2. */
ldr r0,=0x01212
str r0,[r1]
/* Reading the PMC Status to detect when the Master clock is ready */
ldr r2,=PMC_SR
mov r4,#8
MCKRDY_Loop007:
ldr r3,[r2]
and r3,r4,r3
cmp r3,#8
bne MCKRDY_Loop007
CKGR_PLLAR:PMC Clock Generator PLLA Registe
寄存器的值:DIVA=1 PLLACOUNT=17 OUTA=0 MULA=176
/*----------------------------------------------------------------------------
* PMC Init Step 2.
* ----------------------------------------------------------------------------
* Setup PLLA
* ----------------------------------------------------------------------------
*/
/* PLLA Register */
ldr r1,=CKGR_PLLAR
/* Divider is bypassed */
//ldr r0,=0x20421101
ldr r0,=0x20b01101
str r0, [r1]
/* PLL Charge Pump Current Register */
ldr r1, =PMC_PLLICPR
mov r3,#0
str r3,[r1]
/* Reading the PMC Status register to detect when the PLLA is locked */
ldr r2,=PMC_SR
mov r4, #2
MOSCS_Loop1:
ldr r3, [r2]
and r3, r4, r3
cmp r3, #2
bne MOSCS_Loop1
bootrom:at91.h---> #define AT91_MCK_RATE 88000000不改变;
结果3:
打印正常网络加载正常,调试ok!
总结,ping时候部分机器死机,是因为ping的时候有数据交换,频繁的读写DDR2,而DDR2的时序没有严格根据原厂的推荐做法一致导致DDR2驱动有问题。