流式更新-滑动窗口---python实现

最新推荐文章于 2024-08-24 07:00:00 发布

切尔诺贝尔

最新推荐文章于 2024-08-24 07:00:00 发布

阅读量1k

点赞数

分类专栏： python 文章标签：离线更新流式更新 python 滑动窗口

本文链接：https://blog.csdn.net/weixin_40580582/article/details/102526524

版权

python 专栏收录该内容

19 篇文章 0 订阅

订阅专栏

背景
- 需要及时更新数据，且这批数据是需要进行打分计算的，打分规则如下

      {
        "interval": 3, 
        "score": 60
      },
      {
        "interval": 6,# 数据超过六小时未更新则降为50分
        "score": 60
      },
      {
        "interval": 9999999999999999,
        "score": 10
      }

滑动窗口实现逻辑

首先建立一个map,key=分数，value=对应时间

map = {"60":time.time()-interval}

然后每次取值为大于上一个窗口对应时间戳，小于当前窗口时间戳。随后再把当前时间窗口赋值成start窗口，python代码实现如下

#!/usr/bin/env python
# -*- coding:utf-8 -*-
# author:sdot
# datetime:2019/9/30 17:18
# software: PyCharm

"""
滑动窗口算法

"""
# interval 为降级间隔，单位second
import time

score_interval = [
    {
        "interval": 60 * 15,
        "score": 95
    },
    {
        "interval": 60 * 30,
        "score": 90
    },
    {
        "interval": 60 * 60,
        "score": 85
    },
    {
        "interval": 60 * 2,
        "score": 80
    }
]


def CreateSlidingWindows():
    windows = dict()
    pre = int(time.time())
    for item in score_interval:
        # 大于当前分数时间间隔，则为多少分
        windows[item["score"]] = pre
        pre -= item["interval"] * 1000
    return windows


s_windows = CreateSlidingWindows()
e_windows = None

def slid_windows():
    e_windows = CreateSlidingWindows()
    global s_windows
    for sorce, interval in s_windows.items():
        codition = {"gt": interval, "lt": e_windows[sorce]}
        print(sorce,codition)
    s_windows = e_windows


def run():
    while True:
        slid_windows()
        time.sleep(3)


if __name__ == '__main__':
    run()