influxdb-comparisons IOT数据测试 使用记录

生成测试数据

./bulk_data_gen -format influx-bulk -use-case iot -seed 100 -sampling-interval 10s -scale-var 10 -timestamp-start "2022-12-01T00:00:00+80:00" -timestamp-end "2023-01-03T00:00:00+80:00" > iot.txt

• 模拟数据格式：-use-case取值iot、devops等
• 设备数量： scale-var
• 模拟数据起止时间 -timestamp-start “2022-12-01T00:00:00+80:00” -timestamp-end “2023-01-03T00:00:00+80:00” RFC 3339时间格式

生成的数据量

using random seed 100
2023/01/03 21:02:42 Using sampling interval 10s
2023/01/03 21:02:42 Using cardinality of 90
2023/01/03 21:03:03 Written 26517367 points, 61682167 values, took 20.875691 seconds
2023/01/03 21:03:03 bulk_data_gen - main() took 20.882143269s

生成的测试数据 lineprotocol

radiator_valve_room,room_id=4,home_id=0000000000008,radiator=3,sensor_id=0000000000449 opening_level=4.7134109896042569,battery_voltage=2.6384394746188229 1669567860000000000
radiator_valve_room,room_id=4,home_id=0000000000009,radiator=1,sensor_id=0000000000515 opening_level=0.0000000000000000,battery_voltage=2.6167390896804705 1669567860000000000
air_condition_room,room_id=4,home_id=0000000000000,sensor_id=0000000000028 temperature=15.4495740959809140,humidity=43.5387514349416165,battery_voltage=2.6396879862259377 1669567860000000000
light_level_room,room_id=4,home_id=0000000000001,sensor_id=0000000000083 level=9994.7319743336229294,battery_voltage=2.7372084804392189 1669567860000000000
light_level_room,room_id=4,home_id=0000000000002,sensor_id=0000000000127 level=10007.2267689830587187,battery_voltage=2.6417993826213837 1669567860000000000
water_level,home_id=0000000000003,sensor_id=0000000000188 level=4996.9571598958691538,battery_voltage=2.7195733174879151 1669567860000000000
air_condition_room,room_id=4,home_id=0000000000004,sensor_id=0000000000221 temperature=26.3298074605278671,humidity=38.3242305917479413,battery_voltage=2.5785880918193103 1669567860000000000
radiator_valve_room,room_id=5,home_id=0000000000005,radiator=1,sensor_id=0000000000297 opening_level=9.4633341824906836,battery_voltage=2.6977550298026358 1669567860000000000
air_quality_room,room_id=5,home_id=0000000000006,sensor_id=0000000000345 co2_level=300.8629977911269293,co_level=0.0518373191323168,battery_voltage=2.6799883343437774 1669567860000000000
air_condition_room,room_id=4,home_id=0000000000007,sensor_id=0000000000409 temperature=19.9569586151896843,humidity=26.6590854147728180,battery_voltage=2.7074397479067387 1669567860000000000
air_condition_room,room_id=4,home_id=0000000000008,sensor_id=0000000000450 temperature=17.3928266620264793,humidity=39.6829522027461223,battery_voltage=2.6544269187731713 1669567860000000000
window_state_room,room_id=4,home_id=0000000000009,sensor_id=0000000000516,window_id=2 state=0.0000000000000000,battery_voltage=2.7063236442066230 1669567860000000000
air_quality_room,room_id=4,home_id=0000000000000,sensor_id=0000000000029 co2_level=290.8555081831560187,co_level=0.0506118195514378,battery_voltage=2.6377034346557169 1669567860000000000
air_condition_room,room_id=5,home_id=0000000000001,sensor_id=0000000000086 temperature=24.0105269537033976,humidity=30.7422763096272256,battery_voltage=2.6887571525544618 1669567860000000000
window_state_room,room_id=5,home_id=0000000000002,sensor_id=0000000000130,window_id=2 state=1.0000000000000000,battery_voltage=2.6771684190664624 1669567860000000000
door_state,home_id=0000000000003,door_id=0,sensor_id=0000000000191 state=1.0000000000000000,battery_voltage=2.6496625466883801 1669567860000000000
window_state_room,room_id=5,home_id=0000000000004,sensor_id=0000000000224,window_id=1 state=1.0000000000000000,battery_voltage=2.6486184404257598 1669567860000000000
light_level_room,room_id=5,home_id=0000000000005,sensor_id=0000000000300 level=9985.8588173703683424,battery_voltage=2.6528324116163056 1669567860000000000
radiator_valve_room,room_id=6,home_id=0000000000006,radiator=1,sensor_id=0000000000348 opening_level=0.6465701364719936,battery_voltage=2.6432712565203711 1669567860000000000
air_condition_outdoor,home_id=0000000000007,sensor_id=0000000000412 temperature=7.0910940843599892,humidity=72.5612661862510038,battery_voltage=2.7856269166221708 1669567860000000000
window_state_room,room_id=5,home_id=0000000000008,sensor_id=0000000000453,window_id=1 state=1.0000000000000000,battery_voltage=2.6878960293484910 1669567860000000000
air_quality_room,room_id=4,home_id=0000000000009,sensor_id=0000000000519 co2_level=301.4172200746851900,co_level=0.0499902370408330,battery_voltage=2.6908781480875308 1669567860000000000
radiator_valve_room,room_id=5,home_id=0000000000000,radiator=1,sensor_id=0000000000032 opening_level=10.4403234720709790,battery_voltage=2.7498132253130136 1669567860000000000

数据倒数influxdb

./bulk_load_influx -urls http://192.168.3.30:8086 -organization zf -token F55E73_1cBPHqQ87JKdiJmyLQvhQIlmUjf7cFeOtvAAs9_JsRy-l4SJPejRclCW1vYluWpZU441-bHc-C39jIA== -db iot -file ./iot.txt 

或者

cat data/influx.dat | ./bulk_load_influx -urls http://192.168.3.30:8086 -organization zf -token F55E73_1cBPHqQ87JKdiJmyLQvhQIlmUjf7cFeOtvAAs9_JsRy-l4SJPejRclCW1vYluWpZU441-bHc-C39jIA== -db test

数据加载结果

2023/01/03 21:06:08 daemon URLs: [http://192.168.3.30:8086]
2023/01/03 21:06:08 Ingestion rate control is off
2023/01/03 21:06:08 Using InfluxDB API version 2
2023/01/03 21:06:08 SysInfo:
2023/01/03 21:06:08   Current GOMAXPROCS: 12
2023/01/03 21:06:08   Num CPUs: 12
2023/01/03 21:06:08 The following databases already exist in the data store: _tasks, iot, _monitoring
2023/01/03 21:06:08 Database iot [e19a1def4848d626] already exists
2023/01/03 21:06:08 Trend statistics using 30 samples
2023/01/03 21:06:08 Started load with 1 workers
2023/01/03 21:06:08 First statistic report received

2023/01/03 21:10:19 run complete after 5304 batches:
* : min:      NaN/s, mean: 246461.80/s, moving mean: 251701.50/s, moving median:      NaN/s, max:     NaN/s, count:     5304, sum: 61891915.000000 
2023/01/03 21:10:19 [worker 0] backoffs took a total of 0.000000sec of runtime
2023/01/03 21:10:19 loaded 26517367 items in 251.286158sec with 1 workers (mean point rate 105526.572572/sec, mean value rate 245465.836496/s, 17.34MB/sec from stdin)
{"byte_rate_MB_per_sec":17.337019842053024,"daemon_urls":["http://192.168.3.30:8086"],"load_seconds":251.286158109,"mean_point_rate":105526.57257188676,"mean_value_rate":245465.83649563466,"num_batches":5304,"num_items_loaded":26517367,"num_workers":1,"results_report_database":"database_benchmarks","results_report_destination":"","results_report_hostname":"","results_report_tags":null,"stats":{"*":{"Min":5725,"Max":12530,"Mean":11668.913084464548,"Sum":61891915,"Count":5304}},"sys_info":{"GOMAXPROCS":12,"num_cpus":12}}

数据导入influxdb后显示结果

其他链接
https://bbs.csdn.net/topics/603980556
https://www.zhihu.com/tardis/bd/ans/2371000094