第三次作业

1) 博客开头给出自己的基本信息，格式建议如下：

• 2017*****7108；
• 姓名：李佳傲
• gitee仓库地址：https://gitee.com/LJA8520/projects
• def process_file(dst):
• try:
  f1=open(dst,"r")
  except IOError as s:
  print (s)
  return None
  try:
  bvffer=f1.read()
  except:
  print ("Read File Error!")
  return None
  f1.close()
  return bvffer

  2) 程序分析，对程序中的四个函数做个简要说明，要求附上每一段代码和对应的说明。

   

  process_file(dst):   将.txt文件作为参数dst传入函数，打开，读取，关闭操作，返还该.txt文件内容的字符串

  def process_buffer(bvffer):

  if bvffer:

  word_freq = {}

  bvffer=bvffer.lower()

  for x in '~!@#$%^&*()_+/*-+\][':

  bvffer=bvffer.replace(x, " ")

  words=bvffer.strip().split()

  for word in words:

  word_freq[word]=word_freq.get(word,0)+1

  return word_freq

   

  process_buffer(bvffer):  bvffer参数类型为字符串

  对所传参数进行小写，去个标点，按照空格切分为每个单词，保存到wordList列表中

  遍历wordList列表，以{“单词”：“次数”}的形式存入word_freq字典中

  返还word_freq字典

   

  ef output_result(word_freq):

  if word_freq:

  sorted_word_freq = sorted(word_freq.items(), key=lambda v: v[1], reverse=True)

  for item in sorted_word_freq[:10]:

  print (item)

   

  def main():

  dst = "Gone_with_the_wind.txt"

  bvffer = process_file(dst)

  word_freq = process_buffer(bvffer)

  output_result(word_freq)

  
  

  if __name__ == "__main__":

  import cProfile

  import pstats

  cProfile.run("main()", "result")

  p = pstats.Stats("result")

  p.strip_dirs().sort_stats("call").print_stats()

  p.strip_dirs().sort_stats("cumulative").print_stats()

   

  导入argparse库，将python运行指令的参数存入 dst中，依次执行函数

   

  output_result(word_freq):传入字典word_freq

  对word_freq进行词频排序，输出了top10单词

   

  3) 性能分析结果的及改进。

  • 指出执行次数最多的代码，执行时间最长的代码。

   

  ncalls tottime percall cumtime percall filename:lineno(function)
  422364 0.078 0.000 0.078 0.000 {method 'get' of 'dict' objects}
  31580 0.004 0.000 0.004 0.000 word_freq.py:27(<lambda>)
  974 0.000 0.000 0.000 0.000 {method '__getitem__' of 'dict' objects}
  220 0.000 0.000 0.000 0.000 {built-in method builtins.isinstance}
  100 0.030 0.000 0.030 0.000 {method 'replace' of 'str' objects}
  100 0.000 0.000 0.000 0.000 {method 'encode' of 'str' objects}
  63 0.000 0.000 0.000 0.000 parse.py:664(__missing__)
  57 0.000 0.000 0.000 0.000 {built-in method builtins.chr}
  40 0.000 0.000 0.001 0.000 pydevd_utils.py:102(quote_smart)
  40 0.000 0.000 0.000 0.000 {method 'rstrip' of 'bytes' objects}
  40 0.000 0.000 0.001 0.000 parse.py:670(quote)
  40 0.000 0.000 0.000 0.000 {method 'join' of 'str' objects}
  40 0.000 0.000 0.001 0.000 parse.py:751(<listcomp>)
  40 0.000 0.000 0.001 0.000 parse.py:731(quote_from_bytes)
  20 0.000 0.000 0.000 0.000 pydevd_utils.py:79(is_string)
  20 0.000 0.000 0.003 0.000 pydevd.py:1670(write)
  20 0.000 0.000 0.005 0.000 pydevd_io.py:26(write)
  20 0.000 0.000 0.000 0.000 queue.py:206(_put)
  20 0.000 0.000 0.000 0.000 {method 'append' of 'collections.deque' objects}
  20 0.000 0.000 0.000 0.000 pydevd_constants.py:504(get_global_debugger)
  20 0.000 0.000 0.000 0.000 pydevd_constants.py:489(get_protocol)
  20 0.000 0.000 0.000 0.000 {built-in method builtins.len}
  20 0.000 0.000 0.000 0.000 queue.py:115(put)
  20 0.000 0.000 0.000 0.000 {method 'acquire' of '_thread.lock' objects}
  20 0.000 0.000 0.000 0.000 threading.py:332(notify)
  20 0.002 0.000 0.002 0.000 {method 'write' of '_io.TextIOWrapper' objects}
  20 0.000 0.000 0.002 0.000 pydevd_net_command_factory_xml.py:127(make_io_message)
  20 0.000 0.000 0.001 0.000 pydevd_net_command.py:25(__init__)
  20 0.000 0.000 0.000 0.000 pydevd_utils.py:88(to_string)
  20 0.000 0.000 0.000 0.000 threading.py:240(__exit__)
  20 0.000 0.000 0.000 0.000 {method '__enter__' of '_thread.lock' objects}
  20 0.000 0.000 0.000 0.000 threading.py:252(_is_owned)
  20 0.000 0.000 0.000 0.000 pydevd_xml.py:19(make_valid_xml_value)
  20 0.000 0.000 0.000 0.000 threading.py:237(__enter__)
  20 0.000 0.000 0.000 0.000 {method '__exit__' of '_thread.lock' objects}
  20 0.000 0.000 0.001 0.000 pydevd_comm.py:343(add_command)
  10 0.000 0.000 0.006 0.001 {built-in method builtins.print}
  6 0.000 0.000 0.000 0.000 {method 'format' of 'str' objects}
  2 0.000 0.000 0.000 0.000 futures.py:24(__del__)
  1 0.000 0.000 0.000 0.000 {method 'remove' of 'collections.deque' objects}
  1 0.000 0.000 0.000 0.000 parse.py:656(__init__)
  1 0.003 0.003 0.453 0.453 <string>:1(<module>)
  1 0.000 0.000 0.000 0.000 {method 'disable' of '_lsprof.Profiler' objects}
  1 0.000 0.000 0.079 0.079 word_freq.py:25(output_result)
  1 0.034 0.034 0.034 0.034 {method 'split' of 'str' objects}
  1 0.000 0.000 0.021 0.021 word_freq.py:2(process_file)
  1 0.011 0.011 0.450 0.450 word_freq.py:30(main)
  1 0.000 0.000 0.000 0.000 {method 'release' of '_thread.lock' objects}
  1 0.020 0.020 0.020 0.020 {method 'read' of '_io.TextIOWrapper' objects}
  1 0.000 0.000 0.000 0.000 {built-in method io.open}
  1 0.021 0.021 0.021 0.021 {method 'lower' of 'str' objects}
  1 0.002 0.002 0.002 0.002 {method 'strip' of 'str' objects}
  1 0.000 0.000 0.000 0.000 {method 'items' of 'dict' objects}
  1 0.000 0.000 0.000 0.000 {method 'union' of 'frozenset' objects}
  1 0.069 0.069 0.074 0.074 {built-in method builtins.sorted}
  1 0.000 0.000 0.000 0.000 {method 'close' of '_io.TextIOWrapper' objects}
  1 0.000 0.000 0.000 0.000 {built-in method _locale._getdefaultlocale}
  1 0.000 0.000 0.000 0.000 _bootlocale.py:11(getpreferredencoding)
  1 0.174 0.174 0.339 0.339 word_freq.py:15(process_buffer)
  1 0.000 0.000 0.454 0.454 {built-in method builtins.exec}

   

  可以发现 程序中运行次数最多的代码是

   

  给出改进优化的方法以及你的改进代码

  • 　　去掉time.sleep() 方法

  4) 程序运行命令、运行结果截图以及改进后的程序运行命令及结果截图 。

   

  修改前运行结果截图

   

  ncalls tottime percall cumtime percall filename:lineno(function)
  1 0.000 0.000 0.448 0.448 {built-in method builtins.exec}
  1 0.003 0.003 0.448 0.448 <string>:1(<module>)
  1 0.008 0.008 0.445 0.445 word_freq.py:30(main)
  1 0.171 0.171 0.341 0.341 word_freq.py:15(process_buffer)
  422364 0.080 0.000 0.080 0.000 {method 'get' of 'dict' objects}
  1 0.000 0.000 0.075 0.075 word_freq.py:25(output_result)
  1 0.065 0.065 0.069 0.069 {built-in method builtins.sorted}
  1 0.033 0.033 0.033 0.033 {method 'split' of 'str' objects}
  100 0.032 0.000 0.032 0.000 {method 'replace' of 'str' objects}
  1 0.022 0.022 0.022 0.022 {method 'lower' of 'str' objects}
  1 0.000 0.000 0.021 0.021 word_freq.py:2(process_file)
  1 0.020 0.020 0.020 0.020 {method 'read' of '_io.TextIOWrapper' objects}
  10 0.000 0.000 0.006 0.001 {built-in method builtins.print}
  20 0.000 0.000 0.006 0.000 pydevd_io.py:26(write)
  31580 0.004 0.000 0.004 0.000 word_freq.py:27(<lambda>)
  20 0.000 0.000 0.003 0.000 pydevd.py:1670(write)
  1 0.003 0.003 0.003 0.003 {method 'strip' of 'str' objects}
  20 0.002 0.000 0.002 0.000 {method 'write' of '_io.TextIOWrapper' objects}
  20 0.000 0.000 0.002 0.000 pydevd_net_command_factory_xml.py:127(make_io_message)
  40 0.000 0.000 0.001 0.000 pydevd_utils.py:102(quote_smart)
  40 0.000 0.000 0.001 0.000 parse.py:670(quote)
  20 0.000 0.000 0.001 0.000 pydevd_net_command.py:25(__init__)
  40 0.000 0.000 0.001 0.000 parse.py:731(quote_from_bytes)
  40 0.000 0.000 0.001 0.000 parse.py:751(<listcomp>)
  20 0.000 0.000 0.001 0.000 pydevd_comm.py:343(add_command)
  20 0.000 0.000 0.000 0.000 queue.py:115(put)
  974 0.000 0.000 0.000 0.000 {method '__getitem__' of 'dict' objects}
  1 0.000 0.000 0.000 0.000 {built-in method io.open}
  63 0.000 0.000 0.000 0.000 parse.py:664(__missing__)
  20 0.000 0.000 0.000 0.000 threading.py:332(notify)
  1 0.000 0.000 0.000 0.000 {method 'close' of '_io.TextIOWrapper' objects}
  20 0.000 0.000 0.000 0.000 pydevd_utils.py:88(to_string)
  20 0.000 0.000 0.000 0.000 pydevd_xml.py:19(make_valid_xml_value)
  40 0.000 0.000 0.000 0.000 {method 'rstrip' of 'bytes' objects}
  100 0.000 0.000 0.000 0.000 {method 'encode' of 'str' objects}
  220 0.000 0.000 0.000 0.000 {built-in method builtins.isinstance}
  57 0.000 0.000 0.000 0.000 {built-in method builtins.chr}
  20 0.000 0.000 0.000 0.000 threading.py:240(__exit__)
  20 0.000 0.000 0.000 0.000 queue.py:206(_put)
  20 0.000 0.000 0.000 0.000 threading.py:252(_is_owned)
  20 0.000 0.000 0.000 0.000 threading.py:237(__enter__)
  40 0.000 0.000 0.000 0.000 {method 'join' of 'str' objects}
  20 0.000 0.000 0.000 0.000 pydevd_utils.py:79(is_string)
  6 0.000 0.000 0.000 0.000 {method 'format' of 'str' objects}
  20 0.000 0.000 0.000 0.000 pydevd_constants.py:504(get_global_debugger)
  20 0.000 0.000 0.000 0.000 {method '__enter__' of '_thread.lock' objects}
  20 0.000 0.000 0.000 0.000 {method 'acquire' of '_thread.lock' objects}
  1 0.000 0.000 0.000 0.000 _bootlocale.py:11(getpreferredencoding)
  2 0.000 0.000 0.000 0.000 futures.py:24(__del__)
  20 0.000 0.000 0.000 0.000 pydevd_constants.py:489(get_protocol)
  1 0.000 0.000 0.000 0.000 parse.py:656(__init__)
  1 0.000 0.000 0.000 0.000 {built-in method _locale._getdefaultlocale}
  20 0.000 0.000 0.000 0.000 {built-in method builtins.len}
  20 0.000 0.000 0.000 0.000 {method 'append' of 'collections.deque' objects}
  20 0.000 0.000 0.000 0.000 {method '__exit__' of '_thread.lock' objects}
  1 0.000 0.000 0.000 0.000 {method 'union' of 'frozenset' objects}
  1 0.000 0.000 0.000 0.000 {method 'release' of '_thread.lock' objects}
  1 0.000 0.000 0.000 0.000 {method 'remove' of 'collections.deque' objects}
  1 0.000 0.000 0.000 0.000 {method 'items' of 'dict' objects}
  1 0.000 0.000 0.000 0.000 {method 'disable' of '_lsprof.Profiler' objects}

   

  修改后运行结果截图

   

  ncalls tottime percall cumtime percall filename:lineno(function)
  1 0.000 0.000 0.439 0.439 {built-in method builtins.exec}
  1 0.004 0.004 0.439 0.439 <string>:1(<module>)
  1 0.009 0.009 0.435 0.435 word_freq.py:30(main)
  1 0.168 0.168 0.329 0.329 word_freq.py:15(process_buffer)
  1 0.000 0.000 0.078 0.078 word_freq.py:25(output_result)
  422364 0.074 0.000 0.074 0.000 {method 'get' of 'dict' objects}
  1 0.065 0.065 0.070 0.070 {built-in method builtins.sorted}
  1 0.034 0.034 0.034 0.034 {method 'split' of 'str' objects}
  100 0.029 0.000 0.029 0.000 {method 'replace' of 'str' objects}
  1 0.022 0.022 0.022 0.022 {method 'lower' of 'str' objects}
  1 0.000 0.000 0.020 0.020 word_freq.py:2(process_file)
  1 0.019 0.019 0.019 0.019 {method 'read' of '_io.TextIOWrapper' objects}
  10 0.000 0.000 0.008 0.001 {built-in method builtins.print}
  20 0.000 0.000 0.008 0.000 pydevd_io.py:26(write)
  31580 0.004 0.000 0.004 0.000 word_freq.py:27(<lambda>)
  20 0.000 0.000 0.004 0.000 pydevd.py:1670(write)
  20 0.003 0.000 0.003 0.000 {method 'write' of '_io.TextIOWrapper' objects}
  20 0.001 0.000 0.003 0.000 pydevd_net_command_factory_xml.py:127(make_io_message)
  1 0.003 0.003 0.003 0.003 {method 'strip' of 'str' objects}
  40 0.000 0.000 0.002 0.000 pydevd_utils.py:102(quote_smart)
  40 0.000 0.000 0.002 0.000 parse.py:670(quote)
  20 0.000 0.000 0.002 0.000 pydevd_net_command.py:25(__init__)
  40 0.000 0.000 0.001 0.000 parse.py:731(quote_from_bytes)
  40 0.000 0.000 0.001 0.000 parse.py:751(<listcomp>)
  20 0.000 0.000 0.001 0.000 pydevd_comm.py:343(add_command)
  20 0.000 0.000 0.000 0.000 queue.py:115(put)
  974 0.000 0.000 0.000 0.000 {method '__getitem__' of 'dict' objects}
  1 0.000 0.000 0.000 0.000 {built-in method io.open}
  20 0.000 0.000 0.000 0.000 threading.py:332(notify)
  63 0.000 0.000 0.000 0.000 parse.py:664(__missing__)
  20 0.000 0.000 0.000 0.000 pydevd_xml.py:19(make_valid_xml_value)
  20 0.000 0.000 0.000 0.000 pydevd_utils.py:88(to_string)
  100 0.000 0.000 0.000 0.000 {method 'encode' of 'str' objects}
  20 0.000 0.000 0.000 0.000 queue.py:206(_put)
  1 0.000 0.000 0.000 0.000 {method 'close' of '_io.TextIOWrapper' objects}
  220 0.000 0.000 0.000 0.000 {built-in method builtins.isinstance}
  40 0.000 0.000 0.000 0.000 {method 'rstrip' of 'bytes' objects}
  20 0.000 0.000 0.000 0.000 {method 'append' of 'collections.deque' objects}
  40 0.000 0.000 0.000 0.000 {method 'join' of 'str' objects}
  20 0.000 0.000 0.000 0.000 pydevd_constants.py:504(get_global_debugger)
  20 0.000 0.000 0.000 0.000 threading.py:237(__enter__)
  20 0.000 0.000 0.000 0.000 threading.py:252(_is_owned)
  20 0.000 0.000 0.000 0.000 threading.py:240(__exit__)
  20 0.000 0.000 0.000 0.000 {built-in method builtins.len}
  57 0.000 0.000 0.000 0.000 {built-in method builtins.chr}
  20 0.000 0.000 0.000 0.000 pydevd_utils.py:79(is_string)
  20 0.000 0.000 0.000 0.000 {method 'acquire' of '_thread.lock' objects}
  6 0.000 0.000 0.000 0.000 {method 'format' of 'str' objects}
  20 0.000 0.000 0.000 0.000 {method '__enter__' of '_thread.lock' objects}
  1 0.000 0.000 0.000 0.000 _bootlocale.py:11(getpreferredencoding)
  1 0.000 0.000 0.000 0.000 parse.py:656(__init__)
  20 0.000 0.000 0.000 0.000 pydevd_constants.py:489(get_protocol)
  2 0.000 0.000 0.000 0.000 futures.py:24(__del__)
  1 0.000 0.000 0.000 0.000 {built-in method _locale._getdefaultlocale}
  20 0.000 0.000 0.000 0.000 {method '__exit__' of '_thread.lock' objects}
  1 0.000 0.000 0.000 0.000 {method 'union' of 'frozenset' objects}
  1 0.000 0.000 0.000 0.000 {method 'release' of '_thread.lock' objects}
  1 0.000 0.000 0.000 0.000 {method 'items' of 'dict' objects}
  1 0.000 0.000 0.000 0.000 {method 'disable' of '_lsprof.Profiler' objects}
  1 0.000 0.000 0.000 0.000 {method 'remove' of 'collections.deque' objects}

   

  5) 给出你对此次任务的总结与反思。

  学习了使用python命令执行py文件方式以及argparse库的使用方法，怎么能运行的更快

转载于:https://www.cnblogs.com/LJA8520/p/10610647.html