Flink电商用户行为分析之市场营销商业指标分析

---

前言

在进行PV以及UV统计之后，作为电商应用，其市场营销商业指标也是非常重要的组成部分，在此对其进行程序的总结和分析。

一、模块创建与数据准备

在UserBehaviorAnalysis下新建一个maven module作为子项目，命名为MarketAnalysis，在这个模块中我们应该用自定义的测试源来产生测试数据流。

二、APP市场推广统计

2.1.自定义测试源

定义一个源数据样例类MarketingUserBehavior，再定义一个SourceFunction，用于产生用户行为源数据，命名为SimulatedEventSourcce:
代码如下（示例）：

case class MarketingUserBehavior(userId: Long, behavior: String, channel: String,
timestamp: Long)
class SimulatedEventSource extends RichParallelSourceFunction[MarketingUserBehavior]
{
 var running = true
 val channelSet: Seq[String] = Seq("AppStore", "XiaomiStore", "HuaweiStore", "weibo",
"wechat", "tieba")
 val behaviorTypes: Seq[String] = Seq("BROWSE", "CLICK", "PURCHASE", "UNINSTALL")
 val rand: Random = Random
 override def run(ctx: SourceContext[MarketingUserBehavior]): Unit = {
 val maxElements = Long.MaxValue
 var count = 0L
 while (running && count < maxElements) {
 val id = UUID.randomUUID().toString.toLong
 val behaviorType = behaviorTypes(rand.nextInt(behaviorTypes.size))
 val channel = channelSet(rand.nextInt(channelSet.size))
 val ts = System.currentTimeMillis()
 ctx.collectWithTimestamp(MarketingUserBehavior(id, behaviorType, channel, ts),
ts)
 count += 1
 TimeUnit.MILLISECONDS.sleep(5L)
 }
 }

2.2 不分渠道（总量）统计

另 外 定 义 一 个 窗 口 处 理 的 输 出 结 果 样例类 MarketingViewCount，并自定义ProcessWindowFunction 进行处理，代码如下：

case class MarketingCountView(windowStart: Long, windowEnd: Long, channel: String, 
behavior: String, count: Long)
object AppMarketingByChannel {
 def main(args: Array[String]): Unit = {
 val env = StreamExecutionEnvironment.getExecutionEnvironment
 env.setStreamTimeCharacteristic(TimeCharacteristic.EventTime)
 env.setParallelism(1)
 val stream: DataStream[MarketingUserBehavior] = env.addSource(new 
SimulatedEventSource)
 .assignAscendingTimestamps(_.timestamp)
 stream
 .filter(_.behavior != "UNINSTALL")
 .map(data => {
 ((data.channel, data.behavior), 1L)
 })
 .keyBy(_._1)
 .timeWindow(Time.hours(1), Time.seconds(1))
 .process(new MarketingCountByChannel())
 .print(
env.execute(getClass.getSimpleName)
 }
}
class MarketingCountByChannel() extends ProcessWindowFunction[((String, String), 
Long), MarketingViewCount, (String, String), TimeWindow] {
 override def process(key: (String, String),
 context: Context,
 elements: Iterable[((String, String), Long)],
 out: Collector[MarketingViewCount]): Unit = {
 val startTs = context.window.getStart
 val endTs = context.window.getEnd
 val channel = key._1
val behaviorType = key._2
val count = elements.size
 out.collect( MarketingViewCount(formatTs(startTs), formatTs(endTs), channel, 
behaviorType, count) )
 }
 private def formatTs (ts: Long) = {
 val df = new SimpleDateFormat ("yyyy/MM/dd-HH:mm:ss")
 df.format (new Date (ts) )
 }
}

2.3 分渠道（总量）统计

同样我们还可以考察不分渠道的市场推广统计，这样得到的就是所有渠道推广的总量。在 src/main/scala 下创建 AppMarketingStatistics.scala 文件，代码如下：

object AppMarketingStatistics {
 def main(args: Array[String]): Unit = {
 val env = StreamExecutionEnvironment.getExecutionEnvironment
 env.setStreamTimeCharacteristic(TimeCharacteristic.EventTime)
 env.setParallelism(1)
 val stream: DataStream[MarketingUserBehavior] = env.addSource(new 
SimulatedEventSource)
 .assignAscendingTimestamps(_.timestamp)
 stream
 .filter(_.behavior != "UNINSTALL")
 .map(data => {
 ("dummyKey", 1L)
 })
 .keyBy(_._1)
 .timeWindow(Time.hours(1), Time.seconds(1))
 .process(new MarketingCountTotal())
 .print()
 env.execute(getClass.getSimpleName)
 }
}
class MarketingCountTotal() extends ProcessWindowFunction[(String, Long), 
MarketingViewCount, String, TimeWindow]{
 override def process(key: String, context: Context, elements: Iterable[(String, 
Long)], out: Collector[MarketingViewCount]): Unit = {
 val startTs = context.window.getStart
 val endTs = context.window.getEnd
val count = elements.size
 out.collect( MarketingViewCount(formatTs(startTs), formatTs(endTs), "total", 
"total", count) )
 }
 private def formatTs (ts: Long) = {
 val df = new SimpleDateFormat ("yyyy/MM/dd-HH:mm:ss")
 df.format (new Date (ts) )
 }
}

2.4 页面广告分析

电商网站的市场营销商业指标中，除了自身的 APP 推广，还会考虑到页面上的广告投放（包括自己经营的产品和其它网站的广告）。所以广告相关的统计分析，也是市场营销的重要指标。对于广告的统计，最简单也最重要的就是页面广告的点击量，网站往往需要根据广告点击量来制定定价策略和调整推广方式，而且也可以借此收集用户的偏好信息。更加具体的应用是，我们可以根据用户的地理位置进行划分，从而总结出不同省份用户对不同广告的偏好，这样更有助于广告的精准投放。

2.4.1 页面广告点击量统计

接下来我们就进行页面广告按照省份划分的点击量的统计。在 src/main/scala 下创建 AdStatisticsByGeo.scala 文件。同样由于没有现成的数据，我们定义一些测试数据，放在 AdClickLog.csv 中，用来生成用户点击广告行为的事件流。在代码中我们首先定义源数据的样例类 AdClickLog，以及输出统计数据的样例类 CountByProvince。主函数中先以 province 进行 keyBy，然后开一小时的时间窗口，滑动距离为 5 秒，统计窗口内的点击事件数量。具体代码实现如下：

case class AdClickLog(userId: Long, adId: Long, province: String, city: String, 
timestamp: Long)
case class CountByProvince(windowEnd: String, province: String, count: Long)
object AdStatisticsByGeo {
 def main(args: Array[String]): Unit = {
 val env = StreamExecutionEnvironment.getExecutionEnvironment
 env.setStreamTimeCharacteristic(TimeCharacteristic.EventTime)
 env.setParallelism(1)
 val adLogStream: DataStream[AdClickLog] = 
env.readTextFile("YOURPATH\\resources\\AdClickLog.csv")
 .map(data => {
 val dataArray = data.split(",")
 AdClickLog(dataArray(0).toLong, dataArray(1).toLong, dataArray(2), 
dataArray(3), dataArray(4).toLong)
 })
 .assignAscendingTimestamps(_.timestamp * 1000L)
 val adCountStream = adLogStream
 .keyBy(_.province)
 .timeWindow(Time.minutes(60), Time.seconds(5))
 .aggregate(new CountAgg(), new CountResult())
 .print()
 env.execute("ad statistics job")
 }
}
class CountAgg() extends AggregateFunction[AdClickLog, Long, Long]{
 override def add(value: AdClickLog, accumulator: Long): Long = accumulator + 1L
 override def createAccumulator(): Long = 0L
 override def getResult(accumulator: Long): Long = accumulator
 override def merge(a: Long, b: Long): Long = a + b
}
class CountResult() extends WindowFunction[Long, CountByProvince, String, TimeWindow]{
 override def apply(key: String, window: TimeWindow, input: Iterable[Long], out: 
Collector[CountByProvince]): Unit = {
 out.collect(CountByProvince(formatTs(window.getEnd), key, input.iterator.next()))
 }
 private def formatTs (ts: Long) = {
 val df = new SimpleDateFormat ("yyyy/MM/dd-HH:mm:ss")
 df.format (new Date (ts) )
 }
}

总结

既然贴近电商业务，那么市场营销指标就是一个非常重要的评价指标，在这里我们实现了市场推广统计以及页面广告点击量的功能，后者采用了滑动窗口函数实现。