实战：使用多线程处理并发操作

Anthony_路人甲

于 2024-07-06 12:54:29 发布

阅读量443

点赞数 17

分类专栏： selenium 工具类 pytest 文章标签： python selenium 自动化 pytest

本文链接：https://blog.csdn.net/wl18271672781/article/details/140227670

版权

工具类同时被 3 个专栏收录

23 篇文章 0 订阅

订阅专栏

pytest

22 篇文章 0 订阅

订阅专栏

selenium

10 篇文章 0 订阅

订阅专栏

目的：UI登录后，对应的详情接口及以后的是否关单等接口操作逻辑，做并发处理。

1. 并发之前10详情的执行时间

大概半小时（30min）

效果：

5并发后的，同样10详情的处理时间：约 10 分钟，节省 50% 以上时间

2024-06-03 18:17:07,917 - my_logger - INFO - 获取的异常详情页的result为：Terminated
2024-06-03 18:17:08,918 - my_logger - INFO - 轮询结果进行中。。。。。。
2024-06-03 18:17:08,918 - my_logger - INFO - 仍存在异常数据，获取的异常详情页的except_value_result为：[[{'values': ['0323vx0l7bd03', '2', '700000', '2024-06-03T04:11:42+08:00[Asia/Shanghai]', '0323vx0l7bd03', '1', '700000', '2024-06-03T03:03:42.221+08:00[Asia/Shanghai]']}, {'values': ['0323vx9f7b402', '2', '550000', '2024-06-03T04:14:49+08:00[Asia/Shanghai]', '0323vx9f7b402', '1', '550000', '2024-06-03T03:09:17.429+08:00[Asia/Shanghai]']}], [{'values': ['0323vuyc7b207', '2', '100000', '2024-06-03T03:24:51+08:00[Asia/Shanghai]', '0323vuyc7b207', '1', '100000', '2024-06-03T02:19:12.522+08:00[Asia/Shanghai]']}], [{'values': ['0323vtbt7b305', '2', '250000', '2024-06-03T02:49:01+08:00[Asia/Shanghai]', '0323vtbt7b305', '1', '250000', '2024-06-03T01:44:05.463+08:00[Asia/Shanghai]']}, {'values': ['0323vth67b401', '2', '450000', '2024-06-03T02:54:01+08:00[Asia/Shanghai]', '0323vth67b401', '15', '450000', '2024-06-03T01:47:10+08:00[Asia/Shanghai]']}], [{'values': ['0323vlkn7b607', '2', '40000', '2024-06-03T00:05:12+08:00[Asia/Shanghai]', '0323vlkn7b607', '15', '40000', '2024-06-02T22:56:29+08:00[Asia/Shanghai]']}, {'values': ['0323vll57be00', '2', '300000', '2024-06-03T00:05:12+08:00[Asia/Shanghai]', '0323vll57be00', '1', '300000', '2024-06-02T22:56:46.478+08:00[Asia/Shanghai]']}, {'values': ['0323vmdr7b004', '2', '50000', '2024-06-03T00:19:55+08:00[Asia/Shanghai]', '0323vmdr7b004', '1', '50000', '2024-06-02T23:13:57.509+08:00[Asia/Shanghai]']}], [{'values': ['0323vjti7b105', '2', '100000', '2024-06-02T23:24:29+08:00[Asia/Shanghai]', '0323vjti7b105', '15', '100000', '2024-06-02T22:18:59+08:00[Asia/Shanghai]']}, {'values': ['0323vjx37b600', '2', '500000', '2024-06-02T23:24:29+08:00[Asia/Shanghai]', '0323vjx37b600', '15', '500000', '2024-06-02T22:20:43+08:00[Asia/Shanghai]']}], [{'values': ['0323vi4c7a403', '2', '19500', '2024-06-02T22:50:28+08:00[Asia/Shanghai]', '0323vi4c7a403', '1', '19500', '2024-06-02T21:41:57.914+08:00[Asia/Shanghai]']}, {'values': ['0323vi5d7b703', '2', '2000000', '2024-06-02T22:50:28+08:00[Asia/Shanghai]', '0323vi5d7b703', '1', '2000000', '2024-06-02T21:43:24.002+08:00[Asia/Shanghai]']}], [{'values': ['0323ve7h7a401', '2', '200000', '2024-06-02T21:24:34+08:00[Asia/Shanghai]', '0323ve7h7a401', '1', '200000', '2024-06-02T20:18:19.049+08:00[Asia/Shanghai]']}, {'values': ['0323vem27b605', '2', '600000', '2024-06-02T21:34:38+08:00[Asia/Shanghai]', '0323vem27b605', '1', '600000', '2024-06-02T20:26:31.098+08:00[Asia/Shanghai]']}], [{'values': ['0323v8st7a402', '2', '20000', '2024-06-02T19:29:01+08:00[Asia/Shanghai]', '0323v8st7a402', '15', '20000', '2024-06-02T18:20:33+08:00[Asia/Shanghai]']}, {'values': ['0323v8x07ba05', '2', '10000000', '2024-06-02T19:29:01+08:00[Asia/Shanghai]', '0323v8x07ba05', '15', '10000000', '2024-06-02T18:23:04+08:00[Asia/Shanghai]']}], [{'values': ['0323v7797be01', '2', '50000', '2024-06-02T18:52:01+08:00[Asia/Shanghai]', '0323v7797be01', '15', '50000', '2024-06-02T17:46:01+08:00[Asia/Shanghai]']}]]
2024-06-03 18:17:08,918 - my_logger - INFO - >>>>>>>>>>不可关闭的异常详情页，url为：https://console.sofa.antdigital.com/fsm/exception/detail/2220
2024-06-03 18:17:08,920 - my_logger - DEBUG - ================================最终不可关闭的异常详情url为：['https://console.sofa.antdigital.com/fsm/exception/detail/2277', 'https://console.sofa.antdigital.com/fsm/exception/detail/2244', 'https://console.sofa.antdigital.com/fsm/exception/detail/2220']
2024-06-03 18:17:08,920 - my_logger - DEBUG - ================================最终可闭的异常详情url为：['https://console.sofa.antdigital.com/fsm/exception/detail/2261', 'https://console.sofa.antdigital.com/fsm/exception/detail/2252', 'https://console.sofa.antdigital.com/fsm/exception/detail/2226', 'https://console.sofa.antdigital.com/fsm/exception/detail/2238', 'https://console.sofa.antdigital.com/fsm/exception/detail/2224', 'https://console.sofa.antdigital.com/fsm/exception/detail/2237', 'https://console.sofa.antdigital.com/fsm/exception/detail/2264']
2024-06-03 18:17:11,018 - my_logger - DEBUG - ========统计运行时长：Function main_run took 662.9302668571472 seconds to run.

2. 使用多线程能力

logger.info(f"获取的异常列表为：{issue_ids}")

    # 创建 ThreadPoolExecutor 实例，同时处理多个 issue_id
    with concurrent.futures.ThreadPoolExecutor() as executor:
        # 使用 executor.map 并发处理 issue_ids
        executor.map(process_issue, issue_ids)

碰到错误：

AttributeError: module 'concurrent' has no attribute 'futures'

原因：

python 版本问题

2024-06-03 17:59:25,383 - my_logger - INFO - 获取的异常列表为：[2277, 2252, 2261, 2238, 2244, 2226, 2237, 2220, 2224, 2264]
Traceback (most recent call last):
  File "C:\Users\Administrator\PycharmProjects\pythonProject\zisun_tool.py", line 619, in <module>
    main_run()
  File "C:\Users\Administrator\PycharmProjects\pythonProject\decorator.py", line 10, in wrapper
    result = func(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Administrator\PycharmProjects\pythonProject\zisun_tool.py", line 603, in main_run
    with concurrent.futures.ThreadPoolExecutor() as executor:
         ^^^^^^^^^^^^^^^^^^
AttributeError: module 'concurrent' has no attribute 'futures'

解决方案：

使用新多线程池的模块（ThreadPool）

 # 创建 ThreadPool 实例（指定线程数量）
    pool = ThreadPool(5)  # 在括号内指定线程数量

    # 利用 map 函数并行处理 issue_ids
    pool.map(process_issue, issue_ids)

    # 关闭线程池
    pool.close()
    pool.join()

欢迎关注公众号，与Joker一起探索测试之道。