Flink之实时统计指定时间段内热门商品的TopN（双11一小时内的热门品牌排行榜）

IT但丁

已于 2023-11-08 17:16:38 修改

阅读量4.8k

点赞数 6

分类专栏： Flink实时篇文章标签：大数据 flink

于 2020-03-14 23:20:06 首次发布

本文链接：https://blog.csdn.net/u010271601/article/details/104868913

版权

文章目录

一、需求说明

统计一定时间段内的，热门商品/品牌TopN

1、以案例驱动理解

数据：

{
   "userId": "u001", "itemId": "p1001", "categoryId": "c11", type: "pv", "timestamp": "2020-03-08 11:11:11"}
{
   "userId": "u002", "itemId": "p1001", "categoryId": "c11", type: "pv", "timestamp": "2020-03-08 11:11:11"}
{
   "userId": "u003", "itemId": "p1001", "categoryId": "c11", type: "pv", "timestamp": "2020-03-08 11:11:11"}
{
   "userId": "u003", "itemId": "p1001", "categoryId": "c11", type: "cart", "timestamp": "2020-03-08 11:11:11"}
{
   "userId": "u011", "itemId": "p2222", "categoryId": "c22", type: "pv", "timestamp": "2020-03-08 11:11:11"}
{
   "userId": "u012", "itemId": "p2222", "categoryId": "c22", type: "pv", "timestamp": "2020-03-08 11:11:11"}
{
   "userId": "u012", "itemId": "p2222", "categoryId": "c22", type: "pv", "timestamp": "2020-03-08 11:12:01"}
{
   "userId": "u001", "itemId": "p1001", "categoryId": "c11", type: "pv", "timestamp": "2020-03-08 11:12:01"}
{
   "userId": "u002", "itemId": "p1001", "categoryId": "c11", type: "pv", "timestamp": "2020-03-08 11:12:01"}
{
   "userId": "u003", "itemId": "p1001", "categoryId": "c11", type: "pv", "timestamp": "2020-03-08 11:12:01"}
{
   "userId": "u003", "itemId": "p1001", "categoryId": "c11", type: "cart", "timestamp": "2020-03-08 11:12:01"}
{
   "userId": "u011", "itemId": "p2222", "categoryId": "c22", type: "pv", "timestamp": "2020-03-08 11:12:01"}
{
   "userId": "u012", "itemId": "p2222", "categoryId": "c22", type: "pv", "timestamp": "2020-03-08 11:12:01"}

{
   "userId": "u011", "itemId": "p2222", "categoryId": "c22", type: "pv", "timestamp": "2020-03-08 11:13:01"}

二、技术点

Flink的EventTime
Flink的滑动窗口（滚动窗口也可以完成，但是生成的结果太突兀，没有平滑性）☆☆☆☆
Flink的定时器 ☆☆☆☆☆

三、代码实现（一）比较捞

使用window.apply( )方法 → 见第5步
窗口触发时，会执行一次apply,相当于对窗口中的全量数据进行计算（全部拿出在计算）
窗口不触发，会把数据缓存在内存中，当窗口特别长时，那么这种apply不太好

1、调用底层的Process（可做类似map的操作），将Json字符串解析成MyBehavior对象

import com.alibaba.fastjson.JSON;
import org.apache.flink.api.java.tuple.Tuple;
import org.apache.flink.streaming.api.TimeCharacteristic;
import org.apache.flink.streaming.api.datastream.DataStreamSource;
import org.apache.flink.streaming.api.datastream.KeyedStream;
import org.apache.flink.streaming.api.datastream.SingleOutputStreamOperator;
import org.apache.flink.streaming.api.datastream.WindowedStream;
import org.apache.flink.streaming.api.environment.StreamExecutionEnvironment;
import org.apache.flink.streaming.api.functions.ProcessFunction;
import org.apache.flink.streaming.