java.lang.IllegalStateException: Dup key found, key=[1.0E8],
value1=[....],
value2=[....]
at org.apache.kylin.dict.lookup.LookupTable.initRow(LookupTable.java:84)
at org.apache.kylin.dict.lookup.LookupTable.init(LookupTable.java:67)
错误原因:
1.model中 data model 》 join conditions 关系配置错误
2.hive表中,和事实表外键
关联的字段在维度表中的key不唯一,将hive表中的数据去重
- insert overwrite table case_seed_dim
- select
- t.case_seed_one,t.case_seed_one_short_descrip,t.case_seed_one_long_descrip,
- t.case_seed_two,t.case_seed_two_short_descrip,t.case_seed_two_long_descrip,
- t.case_seed_three,t.case_seed_three_short_descrip,t.case_seed_three_long_descrip,
- t.case_seed_no,t.case_seed_delete,t.case_seed_valid,t.case_seed_four,
- t.case_seed_four_short_descrip,t.case_seed_four_long_descrip
- from
- ( select case_seed_one, case_seed_one_short_descrip, case_seed_one_long_descrip,
- case_seed_two,case_seed_two_short_descrip, case_seed_two_long_descrip, case_seed_three,
- case_seed_three_short_descrip,case_seed_three_long_descrip,case_seed_no,
- case_seed_delete, case_seed_valid,case_seed_four, case_seed_four_short_descrip,
- case_seed_four_long_descrip,
- row_number() over(distribute by case_seed_no sort by case_seed_one ) as rn
- from case_seed_dim ) t
- where t.rn=1;