Python 动态抓取 Android 进程内存信息 数据可视化

1-简介

• 如果我们想要测试我们的某些应用在安卓系统上的性能使用情况, 我们可以通过adb桥去查询, 但是如果每次都去手动查询, 自己生成报表,那无疑是非常繁重的工作
• 而python就是实现性能自动化测试脚本的脚本语言甚至数据分析、可以方便的实现我们的需求
• 环境依赖: Mac OS、 PyCharm、adb命令1.0.41版本、小米5手机

2-准备

用数据线连接电脑和手机, 并打开手机的开发者usb调试,当在电脑端输入 adb devices
命令能后查看到如下类似输出证明连接成功

[mac@macdeMacBook-Pro ~]$adb devices
List of devices attached
6eafb6ef	device

打开网易云,然后查看网易云后台进程

[mac@macdeMacBook-Pro ~]$adb shell ps | grep music*
u0_a140   13763 763   2220068 318184 futex_wait 0000000000 S com.netease.cloudmusic
u0_a140   13901 763   1955592 148572 SyS_epoll_ 0000000000 S com.netease.cloudmusic:videoplay
u0_a140   13939 763   1945240 152800 SyS_epoll_ 0000000000 S com.netease.cloudmusic:play

主要是对网易云app的两个进程 com.netease.cloudmusic com.netease.cloudmusic:play 进行内存分析,当然也可以选择其他进程

2- ADB 工具类 的 使用

• 由于要和android设备之间进行交互, 而adb命令就是常用手段之一
• 那么我们就可以创建子进程去执行本地的adb命令
• 创建AdbUtil.py文件编写下面代码

import subprocess
import re
def runShellCmd(cmds: str, input=None, catchResult=None) -> str:
    cmds = "shell " + cmds
    return runCmd(cmds, input, catchResult)


def runCmd(cmds: str, input=None, catchResult=None) -> str:
	 """
    :param cmds:            adb 命令
    :param input:           自定义标准输入
    :param catchResult:     自定义去从标准输出中抓取数据返回的函数
    :return:                
    """
    
    cmds = 'adb ' + cmds
    # p = subprocess.Popen(cmds, stdin=subprocess.PIPE, stderr=subprocess.PIPE,
    #                     creationflags=subprocess.CREATE_NEW_PROCESS_GROUP)

    sin = subprocess.PIPE
    if input is not None and type('') != type(input):
        sin = input
        input = None
	
	# stdin、stdout、stderr 分别指定标准输入.标注输出、标准错误输出的位置
	# 这里设置为管道,就可以用变量接收
    p = subprocess.Popen(cmds, stdin=sin, stdout=subprocess.PIPE, stderr=subprocess.PIPE, shell=True)

    out_data, err_data = p.communicate(input)

    # 生成的结果被b''包裹着， 在mac下
    out_data = str(out_data).strip("b'").strip("'")
    err_data = str(err_data).strip("b'").strip("'")

    ret = p.returncode  # 进程执行状态码，0正常
    if catchResult is None:
        if out_data != '':
            return out_data
        else:
            return err_data

    content = ''
    if out_data is not None:
        content += out_data

    if err_data is not None and err_data != '':
        content += err_data
        msg = ">>>指令{0},执行过程出现错误: {1}".format(cmds, err_data.strip())
        print(msg)
    return catchResult(content)

3-抓取内存信息

核心代码:

• 为了方便观察,抓20次即可
• 执行 catch_mem() 方法

def getProcessMem(process_name: str = ""):
    """
         抓取指定进程名称的内存信息
    :param process_name:
    :return:
    """
    mem_info = AdbUtil.runShellCmd(f"dumpsys meminfo {process_name}").replace(r"\r\n", "\n").replace(r"\n", "\n")

	# todo 自定义生成的内存信息文件保存目录: 
 	process_mem_dir = Constant.MEM_ALL

    if not os.path.exists(process_mem_dir):
        os.makedirs(process_mem_dir)
	# 当前抓取时间
    now_time = time.strftime('%Y-%m-%d_%H%M%S', time.localtime())
    # 直接把shell命令结果写入文件即可
    with open(os.path.join(process_mem_dir, r'mem_' + now_time + r'.txt'), 'w') as f:
        f.write(mem_info)
        f.close()
        del f


def catch_mem() -> NoReturn:
    count = 20
    while count:
        count -= 1
        print('>>>------------抓取内存数据------------')
        getProcessMem()  # 抓取总体的内存信息
        time.sleep(5)
    print('>>>------------抓取内存数据------------ 完成')

效果最终生成内存信息文件目录如下:

具体内容为:

• 它代表着这一时刻, 安卓系统每个进程的内存占用情况
• 接下来主要对 com.netease.cloudmusic 和 com.netease.cloudmusic:play 两个进程数据的抓取出来

Applications Memory Usage (in Kilobytes):
Uptime: 6668394 Realtime: 189563989

Total PSS by process:
    290,590K: com.netease.cloudmusic (pid 13763 / activities)
    205,009K: system (pid 1448)
    120,151K: com.android.systemui (pid 1774 / activities)
    116,014K: com.miui.home (pid 2713 / activities)
     86,907K: surfaceflinger (pid 581)
     81,680K: com.netease.cloudmusic:play (pid 13939)
     60,001K: com.netease.cloudmusic:videoplay (pid 13901)
     56,121K: com.android.mms (pid 26214)
     55,870K: com.android.quicksearchbox (pid 13349)
     50,151K: com.baidu.input_mi (pid 5107)
     44,296K: com.kingroot.kinguser:service (pid 13994)
     42,446K: clatd-rmnet_data6 (pid 28499)
     38,950K: com.android.phone (pid 1939)
     35,180K: com.miui.securitycenter.remote (pid 2937)
     33,103K: com.miui.yellowpage (pid 26158)
     31,965K: com.android.fileexplorer (pid 17523 / activities)
     30,751K: com.miui.gallery (pid 16605 / activities)
     28,547K: tpsd (pid 15014)
     26,680K: com.miui.whetstone (pid 2629)
     24,413K: com.miui.analytics (pid 13643)
     23,982K: com.android.settings (pid 15352 / activities)
     23,568K: com.miui.cleanmaster:cmsdk (pid 16917)
     21,652K: android.process.media (pid 13721)
     21,092K: app_process (pid 9387)
     20,332K: com.miui.powerkeeper:service (pid 3034)
     19,673K: com.lbe.security.miui (pid 3310)
     19,295K: mm-qcamera-daemon (pid 897)
     18,247K: logd (pid 455)
     18,142K: kgod (pid 9341)
     16,511K: com.xiaomi.metoknlp (pid 2864)
     15,993K: com.miui.daemon (pid 2588)
     15,938K: com.miui.systemAdSolution (pid 13601)
     15,522K: com.android.calendar (pid 18319)
     15,204K: com.xiaomi.xmsf (pid 2503)
     13,407K: android.process.acore (pid 26129)
     13,336K: com.xiaomi.finddevice (pid 2609)
     13,177K: .dataservices (pid 2564)
     13,000K: com.android.nfc (pid 2603)
     12,828K: com.miui.cloudservice (pid 17967)
     12,382K: com.miui.personalassistant (pid 26518)
     12,229K: com.android.mms (pid 19865)
     10,614K: zygote (pid 763)
     10,493K: com.miui.sysbase (pid 2999)
     10,012K: audioserver (pid 769)
      9,913K: com.mipay.wallet (pid 26499)
      9,556K: com.miui.weather2:pushservice (pid 15654)
      9,307K: com.android.providers.calendar (pid 18339)
      8,905K: rild (pid 847)
      8,843K: com.android.settings:remote (pid 15602)
      8,092K: com.android.deskclock (pid 25934)
      6,758K: com.miui.contentcatcher (pid 2558)
      6,668K: com.xiaomi.simactivate.service (pid 26243)
      6,573K: rild (pid 938)
      6,115K: zygote64 (pid 761)
      5,654K: com.miui.core (pid 2749)
      5,634K: media.extractor (pid 843)
      5,367K: cameraserver (pid 792)
      5,314K: ims_rtp_daemon (pid 956)
      5,262K: cnd (pid 754)
      4,766K: com.xiaomi.android.dm.service (pid 25404)
      4,577K: imscmservice (pid 957)
      4,519K: com.miui.wmsvc (pid 4101)
      4,519K: mm-pp-dpps (pid 599)
      4,481K: mtd (pid 906)
      4,413K: android.ext.services (pid 15250)
      4,402K: netmgrd (pid 962)
      4,352K: com.qualcomm.qcrilmsgtunnel (pid 1928)
      4,202K: com.android.printspooler (pid 14814)
      4,199K: shelld (pid 764)
      4,191K: mediaserver (pid 845)
      4,095K: qseeproxydaemon (pid 896)
      4,053K: qseer2fdaemon (pid 911)
      4,041K: qpaydaemon (pid 905)
      4,015K: qvop-daemon (pid 900)
      3,814K: com.miui.audioeffect (pid 25836)
      3,734K: com.qualcomm.qti.services.secureui:sui_service (pid 2645)
      3,675K: com.qualcomm.timeservice (pid 19888)
      3,637K: com.xiaomi.micloud.sdk (pid 26995)
      3,595K: media.codec (pid 802)
      3,230K: netd (pid 846)
      2,858K: thermal-engine (pid 755)
      2,816K: vsimd (pid 907)
      2,666K: wpa_supplicant (pid 25086)
      2,516K: imsdatadaemon (pid 921)
      2,063K: audiod (pid 903)
      2,043K: usf_proximity (pid 904)
      2,036K: slim_daemon (pid 1150)
      1,793K: lowi-server (pid 1149)
      1,783K: /init (pid 1)
      1,559K: mcd (pid 2455)
      1,533K: tcpdump (pid 5096)
      1,525K: vold (pid 466)
      1,496K: drmserver (pid 795)
      1,437K: 260 (pid 7108)
      1,434K: mediadrmserver (pid 805)
      1,225K: keystore (pid 799)
      1,148K: cnss_diag (pid 5095)
      1,143K: cnss-daemon (pid 889)
      1,099K: ipacm (pid 928)
      1,074K: loc_launcher (pid 890)
      1,052K: qti (pid 951)
      1,048K: fingerprintd (pid 899)
      1,046K: sensors.qcom (pid 662)
      1,023K: time_daemon (pid 902)
        999K: dpmd (pid 920)
        995K: imsqmidaemon (pid 757)
        976K: installd (pid 798)
        967K: qcom-system-daemon (pid 752)
        948K: ueventd (pid 423)
        910K: adbd (pid 10336)
        902K: pm-service (pid 575)
        853K: qseecomd (pid 618)
        829K: gatekeeperd (pid 912)
        789K: displayfeature (pid 600)
        747K: perfd (pid 475)
        746K: seempd (pid 21744)
        733K: lmkd (pid 579)
        719K: fdpp (pid 766)
        719K: mlipayd (pid 908)
        715K: pm-proxy (pid 598)
        703K: servicemanager (pid 580)
        646K: fidoca (pid 909)
        635K: dumpsys (pid 28666)
        631K: mqsasd (pid 767)
        612K: rmt_storage (pid 567)
        576K: healthd (pid 564)
        569K: adsprpcd (pid 756)
        559K: tftp_server (pid 569)
        551K: otad (pid 765)
        547K: sh (pid 24482)
        541K: qseecomd (pid 577)
        535K: sh (pid 14103)
        534K: dpmd (pid 850)
        521K: port-bridge (pid 967)
        511K: sh (pid 9365)
        503K: sh (pid 14935)
        494K: ipacm-diag (pid 917)
        488K: subsystem_ramdump (pid 6469)
        480K: hvdcp_opti (pid 759)
        456K: gpu_monitor (pid 6624)
        336K: debuggerd64 (pid 465)
        317K: debuggerd64:signaller (pid 468)
        309K: debuggerd (pid 464)
        307K: 61:0 (pid 7101)
        258K: 61:0 (pid 8197)
        257K: debuggerd:signaller (pid 469)
        243K: 41:10139 (pid 14099)
        243K: 41:10139 (pid 14929)
        173K: 41:0 (pid 14102)
        173K: 41:0 (pid 14934)
        153K: 41:0 (pid 7866)
        151K: 260 (pid 7115)
        104K: watchdogd (pid 14504)
        100K: watchdogd (pid 7584)
         94K: ku.sud (pid 7576)
         92K: ktools (pid 14439)
         67K: ku.sud (pid 7574)

Total PSS by OOM adjustment:
    415,302K: Native
         86,907K: surfaceflinger (pid 581)
         42,446K: clatd-rmnet_data6 (pid 28499)
         28,547K: tpsd (pid 15014)
         21,092K: app_process (pid 9387)
         19,295K: mm-qcamera-daemon (pid 897)
         18,247K: logd (pid 455)
         18,142K: kgod (pid 9341)
         10,614K: zygote (pid 763)
         10,012K: audioserver (pid 769)
          8,905K: rild (pid 847)
          6,573K: rild (pid 938)
          6,115K: zygote64 (pid 761)
          5,634K: media.extractor (pid 843)
          5,367K: cameraserver (pid 792)
          5,314K: ims_rtp_daemon (pid 956)
          5,262K: cnd (pid 754)
          4,577K: imscmservice (pid 957)
          4,519K: mm-pp-dpps (pid 599)
          4,481K: mtd (pid 906)
          4,402K: netmgrd (pid 962)
          4,199K: shelld (pid 764)
          4,191K: mediaserver (pid 845)
          4,095K: qseeproxydaemon (pid 896)
          4,053K: qseer2fdaemon (pid 911)
          4,041K: qpaydaemon (pid 905)
          4,015K: qvop-daemon (pid 900)
          3,595K: media.codec (pid 802)
          3,230K: netd (pid 846)
          2,858K: thermal-engine (pid 755)
          2,816K: vsimd (pid 907)
          2,666K: wpa_supplicant (pid 25086)
          2,516K: imsdatadaemon (pid 921)
          2,063K: audiod (pid 903)
          2,043K: usf_proximity (pid 904)
          2,036K: slim_daemon (pid 1150)
          1,793K: lowi-server (pid 1149)
          1,783K: /init (pid 1)
          1,559K: mcd (pid 2455)
          1,533K: tcpdump (pid 5096)
          1,525K: vold (pid 466)
          1,496K: drmserver (pid 795)
          1,437K: 260 (pid 7108)
          1,434K: mediadrmserver (pid 805)
          1,225K: keystore (pid 799)
          1,148K: cnss_diag (pid 5095)
          1,143K: cnss-daemon (pid 889)
          1,099K: ipacm (pid 928)
          1,074K: loc_launcher (pid 890)
          1,052K: qti (pid 951)
          1,048K: fingerprintd (pid 899)
          1,046K: sensors.qcom (pid 662)
          1,023K: time_daemon (pid 902)
            999K: dpmd (pid 920)
            995K: imsqmidaemon (pid 757)
            976K: installd (pid 798)
            967K: qcom-system-daemon (pid 752)
            948K: ueventd (pid 423)
            910K: adbd (pid 10336)
            902K: pm-service (pid 575)
            853K: qseecomd (pid 618)
            829K: gatekeeperd (pid 912)
            789K: displayfeature (pid 600)
            747K: perfd (pid 475)
            746K: seempd (pid 21744)
            733K: lmkd (pid 579)
            719K: fdpp (pid 766)
            719K: mlipayd (pid 908)
            715K: pm-proxy (pid 598)
            703K: servicemanager (pid 580)
            646K: fidoca (pid 909)
            635K: dumpsys (pid 28666)
            631K: mqsasd (pid 767)
            612K: rmt_storage (pid 567)
            576K: healthd (pid 564)
            569K: adsprpcd (pid 756)
            559K: tftp_server (pid 569)
            551K: otad (pid 765)
            547K: sh (pid 24482)
            541K: qseecomd (pid 577)
            535K: sh (pid 14103)
            534K: dpmd (pid 850)
            521K: port-bridge (pid 967)
            511K: sh (pid 9365)
            503K: sh (pid 14935)
            494K: ipacm-diag (pid 917)
            488K: subsystem_ramdump (pid 6469)
            480K: hvdcp_opti (pid 759)
            456K: gpu_monitor (pid 6624)
            336K: debuggerd64 (pid 465)
            317K: debuggerd64:signaller (pid 468)
            309K: debuggerd (pid 464)
            307K: 61:0 (pid 7101)
            258K: 61:0 (pid 8197)
            257K: debuggerd:signaller (pid 469)
            243K: 41:10139 (pid 14099)
            243K: 41:10139 (pid 14929)
            173K: 41:0 (pid 14102)
            173K: 41:0 (pid 14934)
            153K: 41:0 (pid 7866)
            151K: 260 (pid 7115)
            104K: watchdogd (pid 14504)
            100K: watchdogd (pid 7584)
             94K: ku.sud (pid 7576)
             92K: ktools (pid 14439)
             67K: ku.sud (pid 7574)
    476,344K: Persistent
        205,009K: system (pid 1448)
        120,151K: com.android.systemui (pid 1774 / activities)
         38,950K: com.android.phone (pid 1939)
         26,680K: com.miui.whetstone (pid 2629)
         15,993K: com.miui.daemon (pid 2588)
         15,204K: com.xiaomi.xmsf (pid 2503)
         13,336K: com.xiaomi.finddevice (pid 2609)
         13,177K: .dataservices (pid 2564)
         13,000K: com.android.nfc (pid 2603)
          6,758K: com.miui.contentcatcher (pid 2558)
          4,352K: com.qualcomm.qcrilmsgtunnel (pid 1928)
          3,734K: com.qualcomm.qti.services.secureui:sui_service (pid 2645)
    325,770K: Foreground
        290,590K: com.netease.cloudmusic (pid 13763 / activities)
         35,180K: com.miui.securitycenter.remote (pid 2937)
    332,747K: Visible
         81,680K: com.netease.cloudmusic:play (pid 13939)
         60,001K: com.netease.cloudmusic:videoplay (pid 13901)
         50,151K: com.baidu.input_mi (pid 5107)
         44,296K: com.kingroot.kinguser:service (pid 13994)
         24,413K: com.miui.analytics (pid 13643)
         20,332K: com.miui.powerkeeper:service (pid 3034)
         16,511K: com.xiaomi.metoknlp (pid 2864)
         15,938K: com.miui.systemAdSolution (pid 13601)
         10,493K: com.miui.sysbase (pid 2999)
          4,519K: com.miui.wmsvc (pid 4101)
          4,413K: android.ext.services (pid 15250)
    135,687K: Perceptible
        116,014K: com.miui.home (pid 2713 / activities)
         19,673K: com.lbe.security.miui (pid 3310)
      8,843K: A Services
          8,843K: com.android.settings:remote (pid 15602)
     41,628K: B Services
         21,652K: android.process.media (pid 13721)
          9,556K: com.miui.weather2:pushservice (pid 15654)
          5,654K: com.miui.core (pid 2749)
          4,766K: com.xiaomi.android.dm.service (pid 25404)
    371,036K: Cached
         56,121K: com.android.mms (pid 26214)
         55,870K: com.android.quicksearchbox (pid 13349)
         33,103K: com.miui.yellowpage (pid 26158)
         31,965K: com.android.fileexplorer (pid 17523 / activities)
         30,751K: com.miui.gallery (pid 16605 / activities)
         23,982K: com.android.settings (pid 15352 / activities)
         23,568K: com.miui.cleanmaster:cmsdk (pid 16917)
         15,522K: com.android.calendar (pid 18319)
         13,407K: android.process.acore (pid 26129)
         12,828K: com.miui.cloudservice (pid 17967)
         12,382K: com.miui.personalassistant (pid 26518)
         12,229K: com.android.mms (pid 19865)
          9,913K: com.mipay.wallet (pid 26499)
          9,307K: com.android.providers.calendar (pid 18339)
          8,092K: com.android.deskclock (pid 25934)
          6,668K: com.xiaomi.simactivate.service (pid 26243)
          4,202K: com.android.printspooler (pid 14814)
          3,814K: com.miui.audioeffect (pid 25836)
          3,675K: com.qualcomm.timeservice (pid 19888)
          3,637K: com.xiaomi.micloud.sdk (pid 26995)

Total PSS by category:
    472,527K: Dalvik
    436,483K: Native
    192,622K: .dex mmap
    151,645K: .so mmap
    129,297K: .oat mmap
    117,536K: EGL mtrack
     92,916K: GL mtrack
     91,727K: .art mmap
     69,264K: Unknown
     64,578K: Dalvik Other
     57,817K: .apk mmap
     57,442K: Gfx dev
     54,476K: Other mmap
     32,094K: Stack
     13,882K: Ashmem
      2,783K: .ttf mmap
      2,323K: Other dev
        125K: .jar mmap
          0K: Cursor
          0K: Other mtrack

Total RAM: 3,801,352K (status normal)
 Free RAM: 2,166,428K (  371,036K cached pss + 1,707,492K cached kernel +    87,900K free)
 Used RAM: 2,173,785K (1,736,321K used pss +   437,464K kernel)
 Lost RAM:  -555,544K
     ZRAM:    17,996K physical used for    73,556K in swap (2,306,044K total swap)
   Tuning: 256 (large 512), oom   322,560K, restore limit   107,520K (high-end-gfx)

3-数据分析

• 遍历所有内存信息文件,统计出每一时刻网易云的两个进程占用内存的大小, 同时也要合并两个进程在每一时刻进程共同占用的内存大小, 最终用matplotlib画图库的pylab进行绘图, 将数据进行可视化展示

3-1 数据收集

• 为了方便将抓取出的数据进行存储,定义一个Cache对象 和 DataResult类对抓取的结果进程存储
• 编写Cache文件, 编写下面代码

from typing import NoReturn

# 结果集对象
dataResult = DataResult()

# 需要统计的进程列表
ProcessList = ["com.netease.cloudmusic", "com.netease.cloudmusic:play"]


class DataResult:
    """	其实是一个多级字典,存放每个进程的内存趋势变化
    	结构大致如下: 
        比如
        {
            "com.netease.cloudmusic": {
                mem: [],		# 绘图所需的 y 坐标集合 (内存大小)
                x_list:[]		# 绘图所需的 x 坐标集合 (时间)
                x_start:[]		# 时间基准点(这个里面就存一个时,但是为了统一操作还是定义成集合)
            },
            "com.netease.cloudmusic:play": {
                mem: [],
                x_list:[]
                x_start:[]
            }
			# 其他进程同上
			....
        }
    """
    process_dict: dict = {}

    def __init__(self) -> None:
        pass

    def isContainProcessKey(self, process_name, key) -> bool:
        process: dict = self.process_dict.get(process_name)
        if process is None or process.get(key) is None:
            return False

        value_list: list = process.get(key)
        return len(value_list) != 0

    def put(self, process_name, key, value) -> NoReturn:
        if self.process_dict.get(process_name) is None:
            self.process_dict[process_name] = {}

        if self.process_dict[process_name].get(key) is None:
            self.process_dict[process_name][key] = []

        self.process_dict[process_name][key].append(value)

    def isContaionProcessValue(self, process_name, key, value) -> bool:
        """
             判断process_dict.process_name.key  是否包含 value
        :param process_name:
        :param key:
        :param value:
        :return:
        """
        process: dict = self.process_dict.get(process_name)
        if process is None or process.get(key) is None:
            return False

        if value in self.process_dict[process_name][key]:
            return True

        return False

    def get(self, process_name, key, default=None) -> list:
        process: dict = self.process_dict.get(process_name)
        if process is None or process.get(key) is None:
            return default

        return process[key]

    def __str__(self) -> str:
        return str(self.process_dict)

3-2 核心代码:

• 主方法是 mem_all_trend()

import project.auto_test.entity.Cache as Cache
import project.auto_test.utils.Constant as Constant
import os
import re
import time
from matplotlib import pylab as pl
import numpy as np

# 抓取进程内存数据的正则表达式
# 示例:    290,590K: com.netease.cloudmusic (pid 13763 / activities)
mem_info_cp = re.compile(r"\s*([\d,]*)\s*[Kk][Bb]?:\s*(\S*)\s*\(pid\s*\d+(\s*/\s*\S*)?\)")


file_name_time_cp = re.compile(r"\d{4}-\d{1,2}-\d{1,2}_\d+")

def convertFileName2Time(file_name: str):
    """
        转成时间戳
    :param file_name:
    :return:
    """
    logTime = file_name_time_cp.search(file_name).group()
    timeArray = time.strptime(logTime, "%Y-%m-%d_%H%M%S")
    return time.mktime(timeArray)


def computeFormMemAll(file_path):
	# 过滤掉重复的进程数据,因为内存信息文件里可能存在重复的进程数据
    visit_set = set()
    with open(file_path, 'r') as f:
    	# 遍历该内存信息文件的每一行
        for line in f:
            ret = mem_info_cp.search(line)
            if not ret:
                continue
			# 获得进程名
            process_name = ret.group(2)
            # 获得占用的内存大小
            mem_size = ret.group(1).replace(",", "")
            # 如果该进程未被访问过, 并且是在我们要观察的进程范围内,则把结果加入Cache中
            if process_name not in visit_set and process_name in Cache.ProcessList:
                visit_set.add(process_name)
                # 添加纵坐标
                Cache.dataResult.put(process_name, "mem", float(mem_size) / 1024)
               
                # 添加横坐标
                # 横坐标的时间不直接使用,以第一个内存采集的时间未基准点作时间差
                if Cache.dataResult.get(process_name, "x_start") is None:
                    Cache.dataResult.put(process_name, "x_start", convertFileName2Time(file_path))
                    Cache.dataResult.put(process_name, "x_list", 0)
                else:
                    value_lit = Cache.dataResult.get(process_name, "x_start")
                    # 计算时间差
                    del_time = convertFileName2Time(file_path) - value_lit[0]
                    Cache.dataResult.put(process_name, "x_list", del_time)
        f.close()

def mem_all_trend():
	# todo Constant.MEM_ALL就是上图表示的内存信息文件目录,里面存放每一时刻的内存信息文件
	mem_all_file_list = os.listdir(Constant.MEM_ALL)
	mem_all_file_list.sort() # 以防获取的文件列表是乱序的

	# 1-遍历每一个内存信息文件,将抓取出的结果存放到 Cache的dataResult类中
    for file_path in mem_all_file_list:
       # 文件绝对路径
       file_path = os.path.join(Constant.MEM_ALL, file_path)
       computeFormMemAll(file_path)

	# 2- 遍历 Cache的dataResult类的每个进程数据画一条趋势变化图,而且还要合并每个进程的数据再画一条趋势变化图

	# 2-1 汇总每个进程的数据,合并出横坐标和纵坐标
	# 横坐标的时间一致不需额外处理, 更复杂的情况是时间不一致这时需要将他们进行混合
	
	# 遍历每个横坐标的时间,计算出总的y坐标(内存总大小)
	 process_name_key = Cache.ProcessList[0]
	 all_x_list =  Cache.dataResult.get(process_name_key, "mem")
	 all_y_list = []
	 # 遍历每个 时间x坐标
	 for x in all_x_list:
        # 去每一个进程找是否存在该x坐标,如果存在合并内存大小
        y = 0
        for process_name_key in Cache.dataResult.process_dict:
            mem_list = Cache.dataResult.get(process_name_key, "mem")
            x_list = Cache.dataResult.get(process_name_key, "x_list")
            index = x_list.index(x)
            if index >= 0:
                y += mem_list[index]

        all_y_list.append(y)
	

	# 3-绘图
	line_color = ['b', 'g', 'r', 'c', 'm', 'y', 'k']
	count = 0

    title = ""
    result_dict = Cache.dataResult.process_dict.copy()
    result_dict["all"] = {}
    result_dict["all"]["mem"] = all_y_list
    result_dict["all"]["x_list"] = all_x_list

	for key in result_dict:
        mem_list = result_dict.get(key).get("mem")
        x_list = result_dict.get(key).get("x_list")
        # 计算内存最大值
        max_mem_value = max(mem_list)
        # 计算内存平均值
        avg_mem_value = np.mean(mem_list)

        if title:
            if count % 3 == 0 and count != 0:
                title += '\n'
            else:
                title += ' | '
        title += '{0}_mean={1:.2f}{3},{0}_max={2:.2f}{3}\n'.format(key, avg_mem_value, max_mem_value, "MB")

        pl.plot(x_list, mem_list, 'o{0}-'.format(line_color[count]), label=key) 

        count += 1
        if count >= 7:
            count = 0

    pl.title(title)
    pl.legend(loc='best')
    pl.xlabel('Time(s)')
    pl.ylabel('Size({0})'.format("MB"))
    pl.show(block=False)

执行后生成的结果如下:

打赏

如果觉得文章有用,你可鼓励下作者