【云星数据---Apache Flink实战系列(精品版)】：Apache Flink批处理API详解与编程实战019--DateSet实用API详解019

filterWith

def filterWith(fun: (T) ⇒ Boolean): DataSet[T]
Applies a predicate fun to each item of the data set,
keeping only those for which the predicate holds

可以使用片函数进行filter操作。

filterWith示例一：全函数

执行程序：

//1.引入增强依赖
import org.apache.flink.api.scala.extensions._

//2.创建DataSet[Point]
case class Point(x: Double, y: Double)
val ds = benv.fromElements(Point(1, 2), Point(3, 4), Point(5, 6))

//3.使用filterWith进行元素过滤
val r=ds.filterWith {
    case Point(x, y) => x > 1 && y <5
}

//4.显示结果
r.collect

执行结果：

res153: Seq[Point] = Buffer(Point(3.0,4.0))

flink web ui中的执行效果：

filterWith示例二：全函数

执行程序：

//1.引入增强依赖
import org.apache.flink.api.scala.extensions._

//2.创建DataSet[Point]
case class Point(x: Double, y: Double)
val ds = benv.fromElements(Point(1, 2), Point(3, 4), Point(5, 6))

//3.使用filterWith进行元素过滤
val r=ds.filterWith {
    case Point(x, _) => x > 1
}

//4.显示结果
r.collect

执行结果：

res154: Seq[Point] = Buffer(Point(3.0,4.0), Point(5.0,6.0))

reduceWith

def reduceWith(fun: (T, T) ⇒ T): DataSet[T]
Applies a reducer fun to the data set

可以使用偏函数进行reduce操作。

执行程序：

//1.引入增强依赖
import org.apache.flink.api.scala.extensions._

//2.创建DataSet[Point]
case class Point(x: Double, y: Double)
val ds = benv.fromElements(Point(1, 2), Point(3, 4), Point(5, 6))


//3.使用reduceWith进行元素的merger
val r=ds.reduceWith {
    case (Point(x1, y1), (Point(x2, y2))) => Point(x1 + x2, y1 + y2)
}

//4.显示结果
r.collect

执行结果：

res159: Seq[Point] = Buffer(Point(9.0,12.0))

flatMapWith

def flatMapWith[R](fun: (T) ⇒ 
TraversableOnce[R])(implicit arg0: TypeInformation[R], arg1: ClassTag[R]): DataSet[R]

Applies a function fun to each item of the dataset, producing a 
collection of items that will be flattened in the resulting data set

可以使用偏函数进行flatMap操作。

执行程序：

//1.引入增强依赖
import org.apache.flink.api.scala.extensions._

//2.创建DataSet[Point]
case class Point(x: Double, y: Double)
val ds = benv.fromElements(Point(1, 2), Point(3, 4), Point(5, 6))

//3.使用reduceWith进行元素的merger
val r0=ds.flatMapWith {
  case Point(x, y) => Seq("x" -> x, "y" -> y)
}

//4.显示结果
r0.collect

执行结果：

res1: Seq[(String, Double)] = Buffer((x,1.0), (y,2.0), (x,3.0), (y,4.0), (x,5.0), (y,6.0))