spark 存入hbase_spark模块数据存入Hbase数据源报错

执行内容:

FeatureRDDProviderFactory(getHBaseProviderParam()).save(result, getHBaseProviderParam(), phonegjLineName)

报错内容如下:

Exception in thread "main" java.lang.IllegalArgumentException: The simple feature type contains attribute name(s) that are reserved words: ID. You may override this check by putting Boolean.TRUE into the SimpleFeatureType user data under the key 'override.reserved.words' before calling createSchema, or by setting the system property 'override.reserved.words' to 'true', however it may cause errors with some functionality.

at org.locationtech.geomesa.utils.index.ReservedWordCheck$.validateAttributeNames(GeoMesaSchemaValidator.scala:56)

at org.locationtech.geomesa.utils.index.GeoMesaSchemaValidator$.validate(GeoMesaSchemaValidator.scala:30)

at org.locationtech.geomesa.index.geotools.MetadataBackedDataStore.validateNewSchema(MetadataBackedDataStore.scala:57)

at org.locationtech.geomesa.index.geotools.GeoMesaDataStore.validateNewSchema(GeoMesaDataStore.scala:112)

at org.locationtech.geomesa.hbase.data.HBaseDataStore.validateNewSchema(HBaseDataStore.scala:52)

at org.locationtech.geomesa.index.geotools.MetadataBackedDataStore.createSchema(MetadataBackedDataStore.scala:138)

at org.locationtech.geomesa.hbase.data.HBaseDataStore.createSchema(HBaseDataStore.scala:61)

at com.supermap.bdt.rddprovider.hbase.GeoMesaHBaseFeatureRDDProvider.createSchema(GeoMesaHBaseFeatureRDDProvider.scala:294)

at com.supermap.bdt.FeatureRDDProvider$class.save(FeatureRDDProvider.scala:72)

at com.supermap.bdt.rddprovider.hbase.GeoMesaHBaseFeatureRDDProvider.save(GeoMesaHBaseFeatureRDDProvider.scala:35)

at com.cloudoforce.spark.app.analysis.FollowAnalysisApp$.pointToLineGj1(FollowAnalysisApp.scala:760)

at com.cloudoforce.spark.app.analysis.FollowAnalysisApp$.main(FollowAnalysisApp.scala:147)

at com.cloudoforce.spark.app.analysis.FollowAnalysisApp.main(FollowAnalysisApp.scala)

2020-10-28 18:21:56 INFO  SparkContext:54 - Invoking stop() from shutdown hook

其实我在创建SimpleFeatureType已经加入了userdata的设置

public static SimpleFeatureType getFeatureType(String typeName, String geoType) {

SimpleFeatureTypeBuilder builder = new SimpleFeatureTypeBuilder();

builder.setName(typeName);

builder.setCRS(DefaultGeographicCRS.WGS84);//

JSONObject ajson = DatasetSetting.getDvSetting(geoType);

String geo = ajson.getString("geo");

String colName = ajson.getString("colName");

String colType = ajson.getString("colType");

String colLength = ajson.getString("colLength");

//判断空间类型

if(!"table".equals(geo)){

if ("point".equals(geo)) builder.userData("override.reserved.words","true").add("smgeometry", Point.class);

else if ("polyline".equals(geo)) builder.userData("override.reserved.words","true").add("smgeometry", LineString.class);

else if ("polygon".equals(geo)) builder.userData("override.reserved.words","true").add("smgeometry", Polygon.class);

}

String[] colNames = colName.split(",");

String[] colTypes = colType.split(",");

String[] colLengths = colLength.split(",");

for (int i = 0; i < colNames.length; i++) {

String aColName = colNames[i];

String aColType = colTypes[i];

int aColLength = Integer.parseInt(colLengths[i]);

if ("text".equals(aColType)) {

builder.userData("override.reserved.words","true").length(aColLength).add(aColName, String.class);

} else if ("int16".equals(aColType)) {

builder.userData("override.reserved.words","true").length(aColLength).add(aColName, Short.class);

} else if ("int32".equals(aColType)) {

builder.userData("override.reserved.words","true").length(aColLength).add(aColName, Integer.class);

} else if ("int64".equals(aColType)) {

builder.userData("override.reserved.words","true").length(aColLength).add(aColName, Long.class);

} else if ("double".equals(aColType)) {

builder.userData("override.reserved.words","true").minOccurs(0).maxOccurs(2).nillable(false).add(aColName,Double.class);

} else if ("date".equals(aColType)) {

builder.userData("override.reserved.words","true").add(aColName, Timestamp.class);

}

}

// build the type

return builder.buildFeatureType();

}

  • 0
    点赞
  • 0
    收藏
    觉得还不错? 一键收藏
  • 0
    评论
【优质项目推荐】 1、项目代码均经过严格本地测试,运行OK,确保功能稳定后才上传平台。可放心下载并立即投入使用,若遇到任何使用问题,随时欢迎私信反馈与沟通,博主会第一时间回复。 2、项目适用于计算机相关专业(如计科、信息安全、数据科学、人工智能、通信、物联网、自动化、电子信息等)的在校学生、专业教师,或企业员工,小白入门等都适用。 3、该项目不仅具有很高的学习借鉴价值,对于初学者来说,也是入门进阶的绝佳选择;当然也可以直接用于 毕设、课设、期末大作业或项目初期立项演示等。 3、开放创新:如果您有一定基础,且热爱探索钻研,可以在此代码基础上二次开发,进行修改、扩展,创造出属于自己的独特应用。 欢迎下载使用优质资源!欢迎借鉴使用,并欢迎学习交流,共同探索编程的无穷魅力! 基于业务逻辑生成特征变量python实现源码+数据集+超详细注释.zip基于业务逻辑生成特征变量python实现源码+数据集+超详细注释.zip基于业务逻辑生成特征变量python实现源码+数据集+超详细注释.zip基于业务逻辑生成特征变量python实现源码+数据集+超详细注释.zip基于业务逻辑生成特征变量python实现源码+数据集+超详细注释.zip基于业务逻辑生成特征变量python实现源码+数据集+超详细注释.zip基于业务逻辑生成特征变量python实现源码+数据集+超详细注释.zip 基于业务逻辑生成特征变量python实现源码+数据集+超详细注释.zip 基于业务逻辑生成特征变量python实现源码+数据集+超详细注释.zip

“相关推荐”对你有帮助么?

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值