python3生产kafka消息--动态参数读取文件

最新推荐文章于 2022-10-12 12:01:40 发布

万物可测

最新推荐文章于 2022-10-12 12:01:40 发布

阅读量470

点赞数

分类专栏： python3

本文链接：https://blog.csdn.net/qq_38915739/article/details/108993043

版权

python3 专栏收录该内容

18 篇文章 0 订阅

订阅专栏

log：

通过

import linecache

member = linecache.getline("member_id.TXT", i)
line = member_id.strip('\n')
data["member"] = line

控制每次获取到的文件对应的参数

# !/usr/bin/env python
# -*- encoding: utf-8 -*-
# @Author : yyq

import json
import time
import linecache

from kafka import KafkaProducer
time1 = time.time()
producer = KafkaProducer(bootstrap_servers='192.168.172.123:9091,192.168.172.234:9091')
data = {"recordType":"GAIN","changeType":"SYSTEM"}

for i in range(0, 500000):
    data["recordSourceKeys"] = "57889913280038349"+str(i)
    #根据i读取对应行的参数
    member = linecache.getline("member_id.TXT", i)
    line = member_id.strip('\n')
    data["member"] = line
    point_account = linecache.getline("point_account.TXT", i)
    line2 = point_account.strip('\n')
    data["platAccount"] = line2
    msg = json.dumps(data).encode()
    future = producer.send('event.notification.EA-LOYALTY', msg)
    record_metadata = future.get(timeout=10)
time2 = time.time()
print("总耗时：", time2-time1)

文件格式：txt

数据存放方式：每行一条数据