val datas = env.fromElements((1, "aa", 9), (3, "bb", 7) , (2, "bb", 6),
(2, "aa", 6), (2, "aa", 6), (2, "aa", "6"))
datas.distinct().print()
Picked up _JAVA_OPTIONS:
Exception in thread "main" org.apache.flink.api.common.InvalidProgramException: This type (GenericType<java.lang.Object>) cannot be used as key.
at org.apache.flink.api.common.operators.Keys$ExpressionKeys.<init>(Keys.java:327)
at org.apache.flink.api.common.operators.Keys$ExpressionKeys.<init>(Keys.java:291)
at org.apache.flink.api.common.operators.Keys$ExpressionKeys.<init>(Keys.java:222)
at org.apache.flink.api.java.operators.DistinctOperator.<init>(DistinctOperator.java:59)
at org.apache.flink.api.scala.DataSet.distinct(DataSet.scala:990)
at com.sydney.dream.flinkdemo.FlinkDatasetApi$.distinct(FlinkDatasetApi.scala:50)
at com.sydney.dream.flinkdemo.FlinkDatasetApi$.main(FlinkDatasetApi.scala:43)
at com.sydney.dream.flinkdemo.FlinkDatasetApi.main(FlinkDatasetApi.scala)
给的数据中,tuple的第三个元素的值,是object类型(有数字6,也有字符串“6”),不可以当做key。
正确如下:
val datas = env.fromElements((1, "aa", 9), (3, "bb", 7) , (2, "bb", 6),
(2, "aa", 6), (2, "aa", 6), (2, "aa", 6))
datas.distinct().print()