python polling轮询函数

很多时候使用python写自动化比如查询get或者post请求的时候，都是使用如下的语法

import requests
import time

def get200Status(url):
    startTime = time.time()
    while True:
        resp = requests.get('http://tools.2345.com/frame/black/mobile/18428385555')
        if resp.status_code == 200:
            return
        else:
            pass
        time.sleep(5)
        endTime = time.time()
        if endTime - startTime > 100:
            raise Exception("在100s内查询结果失败")

基本都是通过sleep,尤其是在做接口测试的时候。比如创建一个服务，创建的时候需要等待期状态返回为可用状态，此时才会去操作它，如果使用上面的方法似乎也可以，但是不够。这里可以使用 polling，关于其功能：

Polling is a powerful python utility used to wait for a function to return a certain expected condition. Some possible uses cases include:

Wait for API response to return with code 200
Wait for a file to exist (or not exist)
Wait for a thread lock on a resource to expire

进入到polling函数中

def poll(target, step, args=(), kwargs=None, timeout=None, max_tries=None, check_success=is_truthy,
         step_function=step_constant, ignore_exceptions=(), poll_forever=False, collect_values=None,
         log=logging.NOTSET, log_error=logging.NOTSET):
    """Poll by calling a target function until a certain condition is met.

    必须至少指定要调用的目标函数target，以及每次函数调用之间的基本等待时间

    :param step: 定义等待的时间量（以秒为单位

    :param args: Arguments to be passed to the target function

    :type kwargs: dict
    :param kwargs: Keyword arguments to be passed to the target function

    :param timeout: The target function will be called until the time elapsed is greater than the maximum timeout
        (in seconds). NOTE timeout == 0 or timeout == None is equivalent to setting poll_forever=True.
        NOTE that the actual execution time of the function *can* exceed the time specified in the timeout.
        For instance, if the target function takes 10 seconds to execute and the timeout is 21 seconds, the polling
        function will take a total of 30 seconds (two iterations of the target --20s which is less than the timeout--21s,
        and a final iteration).

    :param max_tries: Maximum number of times the target function will be called before failing

    :param check_success: A callback function that accepts the return value of the target function. It should
        return true if you want the polling function to stop and return this value. It should return false if you want it
        to continue executing. The default is a callback that tests for truthiness (anything not False, 0, or empty
        collection).

    :param step_function: A callback function that accepts each iteration's "step." By default, this is constant,
        but you can also pass a function that will increase or decrease the step. As an example, you can increase the wait
        time between calling the target function by 10 seconds every iteration until the step is 100 seconds--at which
        point it should remain constant at 100 seconds

        >>> def my_step_function(step):
        >>>     step += 10
        >>>     return max(step, 100)

    :type ignore_exceptions: tuple
    :param ignore_exceptions: You can specify a tuple of exceptions that should be caught and ignored on every
        iteration. If the target function raises one of these exceptions, it will be caught and the exception
        instance will be pushed to the queue of values collected during polling. Any other exceptions raised will be
        raised as normal.

    :param poll_forever: If set to true, this function will retry until an exception is raised or the target's
        return value satisfies the check_success function. If this is not set, then a timeout or a max_tries must be set.

    :type collect_values: Queue
    :param collect_values: By default, polling will create a new Queue to store all of the target's return values.
        Optionally, you can specify your own queue to collect these values for access to it outside of function scope.

    :type log: int or str, one of logging._levelNames
    :param log: (optional) By default, return values passed to check_success are not logged. However, if this param is
        set to a log level greater than NOTSET, then the return values passed to check_success will be logged.
        This is done by using the decorator log_value.

    :type log_error: int or str, one of logging._levelNames
    :param log_level: (optional) If ignore_exception has been set, you might want to log the exceptions that are
        ignored. If the log_error level is greater than NOTSET, then any caught exceptions will be logged at that
        level. Note: the logger.exception() function is not used. That would print the stacktrace in the logs. Because
        you are ignoring these exceptions, it seems unlikely that'd you'd want a full stack trace for each exception.
        However, if you do what this, you can retrieve the exceptions using the collect_values parameter.

    :return: Polling will return first value from the target function that meets the condions of the check_success
        callback. By default, this will be the first value that is not None, 0, False, '', or an empty collection.

    Note: a message is written to polling2 logger when poll() is called. This logs a message like so:

        >>> Begin poll(target=<>, step=<>, timeout=<>, max_tries=<>, poll_forever=<>)

    This message should allow a user to work-out how long the poll could take, and thereby detect a hang in real-time
    if the poll takes longer than it should.
    """

如果使用sleep则会一直等待。那看看polling如何奇妙的实现上述功能的。

1、每分钟轮询一次，直到URL返回200个状态代码

import requests
import polling
polling.poll(lambda:requests.get('http://tools.2345.com/frame/black/mobile/18428387755').status_code==200,step=60,poll_forever=True)

比如创建一台ECS云服务器，需要等待创建的服务器从创建中变为可用状态，此时才能去服务器上操作

import requests
polling.poll(
    lambda: requests.get('your.instance.ip').status_code == 200,
    step=60,
    ignore_exceptions=(requests.exceptions.ConnectionError,),
    poll_forever=True)

2、查询文件是否存在

等待一个文件存在，比如查询某次服务活动的日志落地或者数据落地信息

# This call will wait until the file exists, checking every 0.1 seconds and stopping after 3 seconds have elapsed
file_handle = polling.poll(
    lambda: open('/tmp/myfile.txt'),
    ignore_exceptions=(IOError,),
    timeout=3,
    step=0.1)

# Polling will return the value of your polling function, so you can now interact with it
file_handle.close()

3、轮询selenium webdriver元素

from selenium import webdriver
driver = webdriver.Firefox()

driver.get('http://google.com')
search_box = polling.poll(
    lambda: driver.find_element_by_id('search'),
    step=0.5,
    timeout=7)

search_box.send_keys('python polling')

4、使用自定义条件回调函数

import requests

def is_correct_response(response):
    """Check that the response returned 'success'"""
    return response == 'success'

polling.poll(
    lambda: requests.put('http://mysite.com/api/user', data={'username': 'Jill'}),
    check_success=is_correct_response,
    step=1,
    timeout=10)

比如使用查询手机号归属地，如果返回响应码是200就简单的认为是响应OK的

import polling
import logging
import requests

successCode = 200

def is_correct_response(response):
    """Check that the response returned 'success'"""
    print("返回状态码为： ",response.status_code)
    print("返回内容为： ", response.text)
    if response.status_code==successCode:
        return True
    else:
        return False
def query_PhoneNumber_Until_SpecifyStatus(phoneNmber=''):
    polling.poll(lambda:requests.get(f'http://tools.2345.com/frame/black/mobile/{phoneNmber}',),
                 check_success=is_correct_response, step=1, timeout=10,log=logging.DEBUG)

if __name__ == '__main__':
    query_PhoneNumber_Until_SpecifyStatus("18477889900")

那实际中的情况一般是，如果response中确实有返回的信息，比如类似返回了id ,状态码等参数，通过去判断响应的数据是否为预期，从而返回Bool值

参考：
polling2 0.4.4