自动化测试与持续集成方案--Android crash 收集

之前看到有人在论坛里面问monkey的执行,我决定不按照前言中的顺序写了，大家可以各取所需。本篇文章与Monkey有关，但又不局限于此。我曾经尝试用Monkey来做crash的收集。

1. Monkey的执行
2. Log的分析
3. Crash的采集
4. 思考和讨论

Monkey的执行

---

当我刚开始接触移动互联网的时候，到处找关于Monkey的文档看，每篇文章都说Monkey很简单，就一条命令，然后就是对各种参数的说明。
后面等我实践的时候，发现用好Monkey不是件容易的事情。如果仅仅是执行这条命令，当然很容易。一般情况下，跑Monkey的同时，获取其Logcat。对Logcat的日志进行分析能发现很多问题。
请看我的实现：

<span class="c">#encoding:utf-8</span>
<span class="kn">import</span> <span class="nn">os</span>
<span class="kn">import</span> <span class="nn">time</span>
<span class="kn">import</span> <span class="nn">re</span>

<span class="c">###############################################################</span>
<span class="c">#########                  Monkey测试                  #########</span>
<span class="c">#########                作者：Anderson 糖果            #########</span>
<span class="c">#########                版本  V1.0.1                  #########</span>
<span class="c">#########                时间：2015.04.25               #########</span>
<span class="c">################################################################</span>

<span class="n">packageName</span><span class="o">=</span><span class="s">"com.XXXX.app"</span>
<span class="n">logdir</span><span class="o">=</span><span class="s">r"d:</span><span class="err">\</span><span class="s">jenkins"</span>
<span class="n">remote_path</span><span class="o">=</span><span class="s">r"</span><span class="se">\\</span><span class="s">10.21.101.100</span><span class="err">\</span><span class="s">build</span><span class="err">\</span><span class="s">android"</span>

<span class="n">os</span><span class="o">.</span><span class="n">system</span><span class="p">(</span><span class="s">'adb shell cat /system/build.prop >D:</span><span class="err">\</span><span class="s">jenkins</span><span class="err">\</span><span class="s">phone.text'</span><span class="p">)</span>

<span class="n">f</span> <span class="o">=</span> <span class="s">r"D:</span><span class="err">\</span><span class="s">jenkins</span><span class="err">\</span><span class="s">phone.text"</span> 

<span class="k">def</span> <span class="nf">getcmd</span><span class="p">(</span><span class="n">cmd</span><span class="p">):</span>
    <span class="n">f</span> <span class="o">=</span> <span class="nb">open</span><span class="p">(</span><span class="n">cmd</span><span class="p">,</span><span class="s">"r"</span><span class="p">)</span> 
    <span class="n">lines</span> <span class="o">=</span> <span class="n">f</span><span class="o">.</span><span class="n">readlines</span><span class="p">()</span>
    <span class="k">for</span> <span class="n">line</span> <span class="ow">in</span> <span class="n">lines</span><span class="p">:</span>

        <span class="n">line</span><span class="o">=</span><span class="n">line</span><span class="o">.</span><span class="n">split</span><span class="p">(</span><span class="s">'='</span><span class="p">)</span>
        <span class="k">if</span> <span class="p">(</span><span class="n">line</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">==</span><span class="s">'ro.build.version.release'</span><span class="p">):</span>
            <span class="n">version</span> <span class="o">=</span> <span class="n">line</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span>
        <span class="k">if</span> <span class="p">(</span><span class="n">line</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">==</span><span class="s">'ro.product.model'</span><span class="p">):</span>
            <span class="n">model</span> <span class="o">=</span> <span class="n">line</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span>
        <span class="k">if</span> <span class="p">(</span><span class="n">line</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">==</span><span class="s">'ro.product.brand'</span><span class="p">):</span>
            <span class="n">brand</span> <span class="o">=</span> <span class="n">line</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span>
    <span class="k">return</span> <span class="n">version</span><span class="p">,</span><span class="n">model</span><span class="p">,</span><span class="n">brand</span>


<span class="n">version</span><span class="p">,</span><span class="n">model</span><span class="p">,</span><span class="n">brand</span><span class="o">=</span><span class="n">getcmd</span><span class="p">(</span><span class="n">f</span><span class="p">)</span>

<span class="k">print</span> <span class="n">version</span><span class="p">,</span><span class="n">model</span><span class="p">,</span><span class="n">brand</span>
<span class="n">os</span><span class="o">.</span><span class="n">remove</span><span class="p">(</span><span class="n">f</span><span class="p">)</span>

<span class="c">#print "使用Logcat清空Phone中log"</span>
<span class="n">os</span><span class="o">.</span><span class="n">popen</span><span class="p">(</span><span class="s">"adb logcat -c"</span><span class="p">)</span>

<span class="c">#print"暂停2秒..."</span>
<span class="k">print</span> <span class="s">"wait"</span>
<span class="n">time</span><span class="o">.</span><span class="n">sleep</span><span class="p">(</span><span class="mi">2</span><span class="p">)</span>

<span class="n">now1</span> <span class="o">=</span> <span class="n">time</span><span class="o">.</span><span class="n">strftime</span><span class="p">(</span><span class="s">'</span><span class="si">%</span><span class="s">Y-</span><span class="si">%</span><span class="s">m-</span><span class="si">%</span><span class="s">d-</span><span class="si">%</span><span class="s">H_</span><span class="si">%</span><span class="s">M_</span><span class="si">%</span><span class="s">S'</span><span class="p">,</span> <span class="n">time</span><span class="o">.</span><span class="n">localtime</span><span class="p">(</span><span class="n">time</span><span class="o">.</span><span class="n">time</span><span class="p">()))</span>

<span class="c">#print"开始执行Monkey命令"</span>
<span class="n">monkeylogname</span><span class="o">=</span><span class="n">logdir</span><span class="o">+</span><span class="s">"</span><span class="se">\\</span><span class="s">"</span><span class="o">+</span><span class="n">now1</span><span class="o">+</span><span class="s">"monkey.log"</span>
<span class="k">print</span> <span class="n">monkeylogname</span>
<span class="n">cmd</span><span class="o">=</span><span class="s">"adb shell monkey -p com.XXXX -s 500 --ignore-timeouts --monitor-native-crashes -v -v 10000 >></span><span class="si">%</span><span class="s">s"</span> <span class="o">%</span><span class="p">(</span><span class="n">monkeylogname</span><span class="p">)</span>
<span class="n">os</span><span class="o">.</span><span class="n">popen</span><span class="p">(</span><span class="n">cmd</span><span class="p">)</span>   

<span class="c">#print"手机截屏"</span>
<span class="n">os</span><span class="o">.</span><span class="n">popen</span><span class="p">(</span><span class="s">"adb shell screencap -p /sdcard/monkey_run.png"</span><span class="p">)</span>

<span class="c">#print"拷贝截屏图片至电脑"</span>
<span class="n">cmd1</span><span class="o">=</span><span class="s">"adb pull /sdcard/monkey_run.png </span><span class="si">%</span><span class="s">s"</span> <span class="o">%</span><span class="p">(</span><span class="n">logdir</span><span class="p">)</span>
<span class="n">os</span><span class="o">.</span><span class="n">popen</span><span class="p">(</span><span class="n">cmd1</span><span class="p">)</span>
<span class="k">print</span> <span class="s">"gai ming"</span>
<span class="n">oldname</span><span class="o">=</span><span class="n">logdir</span><span class="o">+</span><span class="s">"</span><span class="se">\\</span><span class="s">"</span><span class="o">+</span><span class="s">r"monkey_run.png"</span>
<span class="k">if</span> <span class="p">(</span><span class="n">os</span><span class="o">.</span><span class="n">path</span><span class="o">.</span><span class="n">exists</span><span class="p">(</span><span class="n">oldname</span><span class="p">)):</span>
    <span class="k">print</span> <span class="s">"file is exist"</span>
<span class="k">else</span><span class="p">:</span>
    <span class="k">print</span> <span class="s">"file isn't exist"</span>
<span class="n">newname</span><span class="o">=</span><span class="n">logdir</span><span class="o">+</span><span class="s">"</span><span class="se">\\</span><span class="s">"</span><span class="o">+</span><span class="n">now1</span><span class="o">+</span><span class="s">r"monkey.png"</span>
<span class="n">os</span><span class="o">.</span><span class="n">rename</span><span class="p">(</span><span class="n">oldname</span><span class="p">,</span> <span class="n">newname</span><span class="p">)</span>

<span class="c">#print"使用Logcat导出日志"</span>

<span class="n">logcatname</span><span class="o">=</span><span class="n">logdir</span><span class="o">+</span><span class="s">"</span><span class="se">\\</span><span class="s">"</span><span class="o">+</span><span class="n">now1</span><span class="o">+</span><span class="s">r"logcat.log"</span>
<span class="n">cmd2</span><span class="o">=</span><span class="s">"adb logcat -d ></span><span class="si">%</span><span class="s">s"</span> <span class="o">%</span><span class="p">(</span><span class="n">logcatname</span><span class="p">)</span>
<span class="n">os</span><span class="o">.</span><span class="n">popen</span><span class="p">(</span><span class="n">cmd2</span><span class="p">)</span>

<span class="c">#print"导出traces文件"</span>

<span class="n">tracesname</span><span class="o">=</span><span class="n">logdir</span><span class="o">+</span><span class="s">"</span><span class="se">\\</span><span class="s">"</span><span class="o">+</span><span class="n">now1</span><span class="o">+</span><span class="s">r"traces.log"</span>
<span class="n">cmd3</span><span class="o">=</span><span class="s">"adb shell cat /data/anr/traces.txt></span><span class="si">%</span><span class="s">s"</span> <span class="o">%</span><span class="p">(</span><span class="n">tracesname</span><span class="p">)</span>
<span class="n">os</span><span class="o">.</span><span class="n">popen</span><span class="p">(</span><span class="n">cmd3</span><span class="p">)</span>

Log的分析

---

当时在网上找文档时，都是说在Monkey的log里面找“FC”，“ANR”。我另劈蹊径，对Logcat日志进行解析。

<span class="c">######################</span>
<span class="c">#获取error</span>
<span class="c">######################</span>

<span class="n">NullPointer</span><span class="o">=</span><span class="s">"java.lang.NullPointerException"</span>
<span class="n">IllegalState</span><span class="o">=</span><span class="s">"java.lang.IllegalStateException"</span>
<span class="n">IllegalArgument</span><span class="o">=</span><span class="s">"java.lang.IllegalArgumentException"</span>
<span class="n">ArrayIndexOutOfBounds</span><span class="o">=</span><span class="s">"java.lang.ArrayIndexOutOfBoundsException"</span>
<span class="n">RuntimeException</span><span class="o">=</span><span class="s">"java.lang.RuntimeException"</span>
<span class="n">SecurityException</span><span class="o">=</span><span class="s">"java.lang.SecurityException"</span>

<span class="k">def</span> <span class="nf">geterror</span><span class="p">():</span>
    <span class="n">f</span> <span class="o">=</span> <span class="nb">open</span><span class="p">(</span><span class="n">logcatname</span><span class="p">,</span><span class="s">"r"</span><span class="p">)</span> 
    <span class="n">lines</span> <span class="o">=</span> <span class="n">f</span><span class="o">.</span><span class="n">readlines</span><span class="p">()</span>
    <span class="n">errfile</span><span class="o">=</span><span class="s">"</span><span class="si">%</span><span class="s">s</span><span class="err">\</span><span class="s">error.log"</span> <span class="o">%</span><span class="p">(</span><span class="n">remote_path</span><span class="p">)</span>
    <span class="k">if</span> <span class="p">(</span><span class="n">os</span><span class="o">.</span><span class="n">path</span><span class="o">.</span><span class="n">exists</span><span class="p">(</span><span class="n">errfile</span><span class="p">)):</span>
        <span class="n">os</span><span class="o">.</span><span class="n">remove</span><span class="p">(</span><span class="n">errfile</span><span class="p">)</span>
    <span class="n">fr</span> <span class="o">=</span> <span class="nb">open</span><span class="p">(</span><span class="n">errfile</span><span class="p">,</span><span class="s">"a"</span><span class="p">)</span>
    <span class="n">fr</span><span class="o">.</span><span class="n">write</span><span class="p">(</span><span class="n">version</span><span class="p">)</span>
    <span class="n">fr</span><span class="o">.</span><span class="n">write</span><span class="p">(</span><span class="s">"</span><span class="se">\n</span><span class="s">"</span><span class="p">)</span>
    <span class="n">fr</span><span class="o">.</span><span class="n">write</span><span class="p">(</span><span class="n">model</span><span class="p">)</span>
    <span class="n">fr</span><span class="o">.</span><span class="n">write</span><span class="p">(</span><span class="s">"</span><span class="se">\n</span><span class="s">"</span><span class="p">)</span>
    <span class="n">fr</span><span class="o">.</span><span class="n">write</span><span class="p">(</span><span class="n">brand</span><span class="p">)</span>
    <span class="n">fr</span><span class="o">.</span><span class="n">write</span><span class="p">(</span><span class="s">"</span><span class="se">\n</span><span class="s">"</span><span class="p">)</span>
    <span class="n">fr</span><span class="o">.</span><span class="n">write</span><span class="p">(</span><span class="n">now1</span><span class="p">)</span>
    <span class="n">fr</span><span class="o">.</span><span class="n">write</span><span class="p">(</span><span class="s">"</span><span class="se">\n</span><span class="s">"</span><span class="p">)</span>

    <span class="n">count</span><span class="o">=</span><span class="mi">0</span>
    <span class="k">for</span> <span class="n">line</span> <span class="ow">in</span> <span class="n">lines</span><span class="p">:</span>
    <span class="k">if</span> <span class="p">(</span> <span class="n">re</span><span class="o">.</span><span class="n">findall</span><span class="p">(</span><span class="n">NullPointer</span><span class="p">,</span><span class="n">line</span><span class="p">)</span> <span class="ow">or</span> <span class="n">re</span><span class="o">.</span><span class="n">findall</span><span class="p">(</span><span class="n">IllegalState</span><span class="p">,</span><span class="n">line</span><span class="p">)</span> <span class="ow">or</span> <span class="n">re</span><span class="o">.</span><span class="n">findall</span><span class="p">(</span><span class="n">IllegalArgument</span><span class="p">,</span><span class="n">line</span><span class="p">)</span> <span class="ow">or</span> <span class="n">re</span><span class="o">.</span><span class="n">findall</span><span class="p">(</span><span class="n">ArrayIndexOutOfBounds</span><span class="p">,</span><span class="n">line</span><span class="p">)</span> <span class="ow">or</span> <span class="n">re</span><span class="o">.</span><span class="n">findall</span><span class="p">(</span><span class="n">RuntimeException</span><span class="p">,</span><span class="n">line</span><span class="p">)</span> <span class="ow">or</span> <span class="n">re</span><span class="o">.</span><span class="n">findall</span><span class="p">(</span><span class="n">SecurityException</span><span class="p">,</span><span class="n">line</span><span class="p">)</span> <span class="p">):</span>
            <span class="n">a</span><span class="o">=</span><span class="n">lines</span><span class="o">.</span><span class="n">index</span><span class="p">(</span><span class="n">line</span><span class="p">)</span>
            <span class="n">count</span> <span class="o">+=</span><span class="mi">1</span>
            <span class="k">for</span> <span class="n">var</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="n">a</span><span class="p">,</span><span class="n">a</span><span class="o">+</span><span class="mi">22</span><span class="p">):</span>
                <span class="k">print</span> <span class="n">lines</span><span class="p">[</span><span class="n">var</span><span class="p">]</span>
                <span class="n">fr</span><span class="o">.</span><span class="n">write</span><span class="p">(</span><span class="n">lines</span><span class="p">[</span><span class="n">var</span><span class="p">])</span>
            <span class="n">fr</span><span class="o">.</span><span class="n">write</span><span class="p">(</span><span class="s">"</span><span class="se">\n</span><span class="s">"</span><span class="p">)</span>
    <span class="n">f</span><span class="o">.</span><span class="n">close</span><span class="p">()</span>
    <span class="n">fr</span><span class="o">.</span><span class="n">close</span><span class="p">()</span>
    <span class="k">return</span> <span class="n">count</span>


<span class="n">number</span><span class="o">=</span><span class="n">geterror</span><span class="p">()</span>
<span class="k">print</span> <span class="n">number</span>

看到的结果是这样子的：

所有的crash都在这个log里面展现，一目了然，而且放到了共享目录，开发可以直接访问。其实这里面还统计了出现闪退的次数，为了统计次数，没有去重。（这个脚本还是写得比较粗糙，但是能达到要求了,有兴趣的可以在上面继续优化。）

Crash的采集

---

这种方式是能采集到一些闪退，但这远远不够。因为客户那边出现闪退，你是没有现场的。有没有一种方法来监控这些闪退。不管是测试这边还是客户那边，只要有crash,就会向服务器发请求，将crash的具体信息记录在数据库里面。
例如：

后面有跟这个系统的作者有交流，他是重写了线程监控UncaughtExceptionHandler。他说可以把接口开放给我，我目前还没开始研究。等有了结果，再来分享。

思考和讨论

---

1. IOS上的crash 如何监控?
2. 跑Monkey时，如何屏幕解锁?有登录咋办? 在登录以后跑，它退出了咋办?
3. 跑Monkey时，怎么避免它将wifi或网络关掉?
4. 开发如何面对这些你捕获的crash log? (我们的开发有一大堆理由说看不懂，解不了)
5. 有没有更好的crash捕获机制?你们是如何做的?

回答：

1. ios 的 crash， 论坛里有文章了。
2. 跑 Monkey 时，如何屏幕解锁?有登录咋办? 在登录以后跑，它退出了咋办? —— monkey 做不了逻辑的东西，这个和 monkey 违背。
3. 跑Monkey时，怎么避免它将wifi或网络关掉? 所以我们用 串口 跑 monkey 或者把 com.android.settings 加入黑名单
4. 开发如何面对这些你捕获的crash log? (我们的开发有一大堆理由说看不懂，解不了) —— 太弱，无解
5. 有，裁剪自己的 log 系统。然后使用 elk 来做日志分析。