Pyflink SQL子查询UDF报错

最新推荐文章于 2023-10-26 21:32:32 发布

胖胖胖胖胖虎

最新推荐文章于 2023-10-26 21:32:32 发布

阅读量586

点赞数

本文链接：https://blog.csdn.net/qq_15138049/article/details/114986433

版权

Pyflink 子查询UDF报错：train_and_predict是自定义的UDF，splitStr是自定义的UDTF

t_env.sql_query("""
select A.hotime ,
        A.before_ta ,
        A.before_rssi ,
        A.after_ta ,
        A.after_rssil ,
        A.nb_tath ,
        A.nb_rssith ,
        nbr_rssi ,
        nbr_ta
from (SELECT
    hotime ,
    before_ta ,
    before_rssi ,
    after_ta ,
    after_rssil ,
    nb_tath ,
    nb_rssith ,
    train_and_predict(nb_tath, nb_rssith) predict
FROM
    source) as  A,LATERAL TABLE(splitStr(predict)) as T(nbr_rssi, nbr_ta)
""").insert_into("predict_sink")

Traceback (most recent call last):
  File
"C:/Users/Administrator.XTZ-02012091146/PycharmProjects/pythonProject/kerasTest/UdtfNtPredictPyFlink.py",
line 280, in <module>
    t_env.execute('NT重连预测参数')
  File
"D:\tools\Python3.6.5\lib\site-packages\pyflink\table\table_environment.py",
line 1057, in execute
    return JobExecutionResult(self._j_tenv.execute(job_name))
  File "D:\tools\Python3.6.5\lib\site-packages\py4j\java_gateway.py", line
1286, in __call__
    answer, self.gateway_client, self.target_id, self.name)
  File "D:\tools\Python3.6.5\lib\site-packages\pyflink\util\exceptions.py",
line 147, in deco
    return f(*a, **kw)
  File "D:\tools\Python3.6.5\lib\site-packages\py4j\protocol.py", line 328,
in get_return_value
    format(target_id, ".", name), value)
py4j.protocol.Py4JJavaError: An error occurred while calling o25.execute.
: java.lang.IndexOutOfBoundsException: Index: 7, Size: 7
        at java.util.ArrayList.rangeCheck(ArrayList.java:657)

Flink中文邮件列表提了个topic: http://apache-flink.147419.n8.nabble.com/pyflink-UDTF-td11326.html

t_env.sql_query("""
SELECT
     hotime ,
     before_ta ,
     before_rssi ,
     after_ta ,
     after_rssil ,
     nb_tath ,
     nb_rssith ,
     train_and_predict(nb_tath, nb_rssith) predict
FROM source
""").join_lateral("splitStr(predict) as (nbr_rssi, nbr_ta)")
.select("""
     hotime ,
     before_ta ,
     before_rssi ,
     after_ta ,
     after_rssil ,
     nb_tath ,
     nb_rssith ,
     nbr_rssi ,
     nbr_ta
""").insert_into("predict_sink")