org.apache.spark.sql.catalyst.analysis.NoSuchDatabaseException: Database 'traffic' not found;

最新推荐文章于 2022-04-26 16:26:38 发布
最新推荐文章于 2022-04-26 16:26:38 发布
阅读量5.4k 收藏 3
点赞数 1
分类专栏： spark hive 文章标签： spark hive 写入数据
版权声明：本文为博主原创文章，遵循 CC 4.0 BY-SA 版权协议，转载请附上原文出处链接和本声明。
本文链接：https://blog.csdn.net/zzw_17600691357/article/details/95939559
版权
spark 同时被 2 个专栏收录
11 篇文章 0 订阅
订阅专栏
hive
11 篇文章 0 订阅
订阅专栏

spark向hive中写入数据时，提示找不到已经创建完成的数据库，

org.apache.spark.sql.catalyst.analysis.NoSuchDatabaseException: Database 'traffic' not found;

我的问题出现原因：

没有为spark设置hive的路径

解决方案：

将hive客户端的hive-site.xml放入到spark的conf目录下，spark会自己找到该配置文件，找到hive的所在位置

[root@node4 bin]# ./spark-submit --master spark://node1:7077 --class com.producedate2hive.Data2Hive /root/TrafficProject-1.0-SNAPSHOT-jar-with-dependencies.jar 
2019-07-15 09:58:43 WARN  NativeCodeLoader:62 - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2019-07-15 09:58:44 INFO  SparkContext:54 - Running Spark version 2.3.1
2019-07-15 09:58:44 INFO  SparkContext:54 - Submitted application: traffic2hive
2019-07-15 09:58:44 INFO  SecurityManager:54 - Changing view acls to: root
2019-07-15 09:58:44 INFO  SecurityManager:54 - Changing modify acls to: root
2019-07-15 09:58:44 INFO  SecurityManager:54 - Changing view acls groups to: 
2019-07-15 09:58:44 INFO  SecurityManager:54 - Changing modify acls groups to: 
2019-07-15 09:58:44 INFO  SecurityManager:54 - SecurityManager: authentication disabled; ui acls disabled; users  with view permissions: Set(root); groups with view permissions: Set(); users  with modify permissions: Set(root); groups with modify permissions: Set()
2019-07-15 09:58:44 INFO  Utils:54 - Successfully started service 'sparkDriver' on port 53087.
2019-07-15 09:58:44 INFO  SparkEnv:54 - Registering MapOutputTracker
2019-07-15 09:58:44 INFO  SparkEnv:54 - Registering BlockManagerMaster
2019-07-15 09:58:44 INFO  BlockManagerMasterEndpoint:54 - Using org.apache.spark.storage.DefaultTopologyMapper for getting topology information
2019-07-15 09:58:44 INFO  BlockManagerMasterEndpoint:54 - BlockManagerMasterEndpoint up
2019-07-15 09:58:44 INFO  DiskBlockManager:54 - Created local directory at /tmp/blockmgr-af28f89f-6caa-45b3-93e2-e1531b36d850
2019-07-15 09:58:44 INFO  MemoryStore:54 - MemoryStore started with capacity 366.3 MB
2019-07-15 09:58:44 INFO  SparkEnv:54 - Registering OutputCommitCoordinator
2019-07-15 09:58:44 INFO  log:192 - Logging initialized @1921ms
2019-07-15 09:58:44 INFO  Server:346 - jetty-9.3.z-SNAPSHOT
2019-07-15 09:58:44 INFO  Server:414 - Started @2006ms
2019-07-15 09:58:44 INFO  AbstractConnector:278 - Started ServerConnector@53f83f19{HTTP/1.1,[http/1.1]}{0.0.0.0:4040}
2019-07-15 09:58:44 INFO  Utils:54 - Successfully started service 'SparkUI' on port 4040.
2019-07-15 09:58:44 INFO  ContextHandler:781 - Started o.s.j.s.ServletContextHandler@71b1a49c{/jobs,null,AVAILABLE,@Spark}
2019-07-15 09:58:44 INFO  ContextHandler:781 - Started o.s.j.s.ServletContextHandler@28f8e165{/jobs/json,null,AVAILABLE,@Spark}
2019-07-15 09:58:44 INFO  ContextHandler:781 - Started o.s.j.s.ServletContextHandler@545f80bf{/jobs/job,null,AVAILABLE,@Spark}
2019-07-15 09:58:44 INFO  ContextHandler:781 - Started o.s.j.s.ServletContextHandler@22fa55b2{/jobs/job/json,null,AVAILABLE,@Spark}
2019-07-15 09:58:44 INFO  ContextHandler:781 - Started o.s.j.s.ServletContextHandler@4d666b41{/stages,null,AVAILABLE,@Spark}
2019-07-15 09:58:44 INFO  ContextHandler:781 - Started o.s.j.s.ServletContextHandler@6594402a{/stages/json,null,AVAILABLE,@Spark}
2019-07-15 09:58:44 INFO  ContextHandler:781 - Started o.s.j.s.ServletContextHandler@30f4b1a6{/stages/stage,null,AVAILABLE,@Spark}
2019-07-15 09:58:44 INFO  ContextHandler:781 - Started o.s.j.s.ServletContextHandler@79c3f01f{/stages/stage/json,null,AVAILABLE,@Spark}
2019-07-15 09:58:44 INFO  ContextHandler:781 - Started o.s.j.s.ServletContextHandler@6c2f1700{/stages/pool,null,AVAILABLE,@Spark}
2019-07-15 09:58:44 INFO  ContextHandler:781 - Started o.s.j.s.ServletContextHandler@350b3a17{/stages/pool/json,null,AVAILABLE,@Spark}
2019-07-15 09:58:44 INFO  ContextHandler:781 - Started o.s.j.s.ServletContextHandler@38600b{/storage,null,AVAILABLE,@Spark}
2019-07-15 09:58:44 INFO  ContextHandler:781 - Started o.s.j.s.ServletContextHandler@669d2b1b{/storage/json,null,AVAILABLE,@Spark}
2019-07-15 09:58:44 INFO  ContextHandler:781 - Started o.s.j.s.ServletContextHandler@721eb7df{/storage/rdd,null,AVAILABLE,@Spark}
2019-07-15 09:58:44 INFO  ContextHandler:781 - Started o.s.j.s.ServletContextHandler@1ea9f009{/storage/rdd/json,null,AVAILABLE,@Spark}
2019-07-15 09:58:44 INFO  ContextHandler:781 - Started o.s.j.s.ServletContextHandler@5d52e3ef{/environment,null,AVAILABLE,@Spark}
2019-07-15 09:58:44 INFO  ContextHandler:781 - Started o.s.j.s.ServletContextHandler@5298dead{/environment/json,null,AVAILABLE,@Spark}
2019-07-15 09:58:44 INFO  ContextHandler:781 - Started o.s.j.s.ServletContextHandler@553f3b6e{/executors,null,AVAILABLE,@Spark}
2019-07-15 09:58:44 INFO  ContextHandler:781 - Started o.s.j.s.ServletContextHandler@4c7a078{/executors/json,null,AVAILABLE,@Spark}
2019-07-15 09:58:44 INFO  ContextHandler:781 - Started o.s.j.s.ServletContextHandler@4e406694{/executors/threadDump,null,AVAILABLE,@Spark}
2019-07-15 09:58:44 INFO  ContextHandler:781 - Started o.s.j.s.ServletContextHandler@5ab9b447{/executors/threadDump/json,null,AVAILABLE,@Spark}
2019-07-15 09:58:44 INFO  ContextHandler:781 - Started o.s.j.s.ServletContextHandler@76f10035{/static,null,AVAILABLE,@Spark}
2019-07-15 09:58:44 INFO  ContextHandler:781 - Started o.s.j.s.ServletContextHandler@50b1f030{/,null,AVAILABLE,@Spark}
2019-07-15 09:58:44 INFO  ContextHandler:781 - Started o.s.j.s.ServletContextHandler@4163f1cd{/api,null,AVAILABLE,@Spark}
2019-07-15 09:58:44 INFO  ContextHandler:781 - Started o.s.j.s.ServletContextHandler@23aae55{/jobs/job/kill,null,AVAILABLE,@Spark}
2019-07-15 09:58:44 INFO  ContextHandler:781 - Started o.s.j.s.ServletContextHandler@5f574cc2{/stages/stage/kill,null,AVAILABLE,@Spark}
2019-07-15 09:58:44 INFO  SparkUI:54 - Bound SparkUI to 0.0.0.0, and started at http://node4:4040
2019-07-15 09:58:45 INFO  SparkContext:54 - Added JAR file:/root/TrafficProject-1.0-SNAPSHOT-jar-with-dependencies.jar at spark://node4:53087/jars/TrafficProject-1.0-SNAPSHOT-jar-with-dependencies.jar with timestamp 1563155925020
2019-07-15 09:58:45 INFO  StandaloneAppClient$ClientEndpoint:54 - Connecting to master spark://node1:7077...
2019-07-15 09:58:45 INFO  TransportClientFactory:267 - Successfully created connection to node1/172.16.220.151:7077 after 32 ms (0 ms spent in bootstraps)
2019-07-15 09:58:45 INFO  StandaloneSchedulerBackend:54 - Connected to Spark cluster with app ID app-20190715095845-0001
2019-07-15 09:58:45 INFO  Utils:54 - Successfully started service 'org.apache.spark.network.netty.NettyBlockTransferService' on port 35754.
2019-07-15 09:58:45 INFO  NettyBlockTransferService:54 - Server created on node4:35754
2019-07-15 09:58:45 INFO  BlockManager:54 - Using org.apache.spark.storage.RandomBlockReplicationPolicy for block replication policy
2019-07-15 09:58:45 INFO  StandaloneAppClient$ClientEndpoint:54 - Executor added: app-20190715095845-0001/0 on worker-20190715095545-172.16.220.153-47385 (172.16.220.153:47385) with 2 core(s)
2019-07-15 09:58:45 INFO  StandaloneSchedulerBackend:54 - Granted executor ID app-20190715095845-0001/0 on hostPort 172.16.220.153:47385 with 2 core(s), 1024.0 MB RAM
2019-07-15 09:58:45 INFO  StandaloneAppClient$ClientEndpoint:54 - Executor added: app-20190715095845-0001/1 on worker-20190715095545-172.16.220.152-42429 (172.16.220.152:42429) with 2 core(s)
2019-07-15 09:58:45 INFO  StandaloneSchedulerBackend:54 - Granted executor ID app-20190715095845-0001/1 on hostPort 172.16.220.152:42429 with 2 core(s), 1024.0 MB RAM
2019-07-15 09:58:45 INFO  BlockManagerMaster:54 - Registering BlockManager BlockManagerId(driver, node4, 35754, None)
2019-07-15 09:58:45 INFO  BlockManagerMasterEndpoint:54 - Registering block manager node4:35754 with 366.3 MB RAM, BlockManagerId(driver, node4, 35754, None)
2019-07-15 09:58:45 INFO  BlockManagerMaster:54 - Registered BlockManager BlockManagerId(driver, node4, 35754, None)
2019-07-15 09:58:45 INFO  BlockManager:54 - Initialized BlockManager: BlockManagerId(driver, node4, 35754, None)
2019-07-15 09:58:45 INFO  StandaloneAppClient$ClientEndpoint:54 - Executor updated: app-20190715095845-0001/0 is now RUNNING
2019-07-15 09:58:45 INFO  StandaloneAppClient$ClientEndpoint:54 - Executor updated: app-20190715095845-0001/1 is now RUNNING
2019-07-15 09:58:45 INFO  ContextHandler:781 - Started o.s.j.s.ServletContextHandler@5eb97ced{/metrics/json,null,AVAILABLE,@Spark}
2019-07-15 09:58:46 INFO  EventLoggingListener:54 - Logging events to hdfs://mycluster/spark/log/app-20190715095845-0001.lz4
2019-07-15 09:58:46 INFO  StandaloneSchedulerBackend:54 - SchedulerBackend is ready for scheduling beginning after reached minRegisteredResourcesRatio: 0.0
2019-07-15 09:58:46 INFO  SharedState:54 - Setting hive.metastore.warehouse.dir ('null') to the value of spark.sql.warehouse.dir ('file:/opt/spark-2.3.1/bin/spark-warehouse').
2019-07-15 09:58:46 INFO  SharedState:54 - Warehouse path is 'file:/opt/spark-2.3.1/bin/spark-warehouse'.
2019-07-15 09:58:46 INFO  ContextHandler:781 - Started o.s.j.s.ServletContextHandler@715b886f{/SQL,null,AVAILABLE,@Spark}
2019-07-15 09:58:46 INFO  ContextHandler:781 - Started o.s.j.s.ServletContextHandler@7fb29ca9{/SQL/json,null,AVAILABLE,@Spark}
2019-07-15 09:58:46 INFO  ContextHandler:781 - Started o.s.j.s.ServletContextHandler@73844119{/SQL/execution,null,AVAILABLE,@Spark}
2019-07-15 09:58:46 INFO  ContextHandler:781 - Started o.s.j.s.ServletContextHandler@44f24a20{/SQL/execution/json,null,AVAILABLE,@Spark}
2019-07-15 09:58:46 INFO  ContextHandler:781 - Started o.s.j.s.ServletContextHandler@1687eb01{/static/sql,null,AVAILABLE,@Spark}
2019-07-15 09:58:47 INFO  StateStoreCoordinatorRef:54 - Registered StateStoreCoordinator endpoint
2019-07-15 09:58:47 INFO  HiveUtils:54 - Initializing HiveMetastoreConnection version 1.2.1 using Spark classes.
2019-07-15 09:58:48 INFO  HiveMetaStore:589 - 0: Opening raw store with implemenation class:org.apache.hadoop.hive.metastore.ObjectStore
2019-07-15 09:58:48 INFO  ObjectStore:289 - ObjectStore, initialize called
2019-07-15 09:58:48 INFO  Persistence:77 - Property hive.metastore.integral.jdo.pushdown unknown - will be ignored
2019-07-15 09:58:48 INFO  Persistence:77 - Property datanucleus.cache.level2 unknown - will be ignored
2019-07-15 09:58:49 INFO  CoarseGrainedSchedulerBackend$DriverEndpoint:54 - Registered executor NettyRpcEndpointRef(spark-client://Executor) (172.16.220.153:52220) with ID 0
2019-07-15 09:58:49 INFO  ObjectStore:370 - Setting MetaStore object pin classes with hive.metastore.cache.pinobjtypes="Table,StorageDescriptor,SerDeInfo,Partition,Database,Type,FieldSchema,Order"
2019-07-15 09:58:50 INFO  BlockManagerMasterEndpoint:54 - Registering block manager 172.16.220.153:60391 with 366.3 MB RAM, BlockManagerId(0, 172.16.220.153, 60391, None)
2019-07-15 09:58:50 INFO  CoarseGrainedSchedulerBackend$DriverEndpoint:54 - Registered executor NettyRpcEndpointRef(spark-client://Executor) (172.16.220.152:46055) with ID 1
2019-07-15 09:58:50 INFO  BlockManagerMasterEndpoint:54 - Registering block manager 172.16.220.152:53876 with 366.3 MB RAM, BlockManagerId(1, 172.16.220.152, 53876, None)
2019-07-15 09:58:51 INFO  Datastore:77 - The class "org.apache.hadoop.hive.metastore.model.MFieldSchema" is tagged as "embedded-only" so does not have its own datastore table.
2019-07-15 09:58:51 INFO  Datastore:77 - The class "org.apache.hadoop.hive.metastore.model.MOrder" is tagged as "embedded-only" so does not have its own datastore table.
2019-07-15 09:58:51 INFO  Datastore:77 - The class "org.apache.hadoop.hive.metastore.model.MFieldSchema" is tagged as "embedded-only" so does not have its own datastore table.
2019-07-15 09:58:51 INFO  Datastore:77 - The class "org.apache.hadoop.hive.metastore.model.MOrder" is tagged as "embedded-only" so does not have its own datastore table.
2019-07-15 09:58:52 INFO  Query:77 - Reading in results for query "org.datanucleus.store.rdbms.query.SQLQuery@0" since the connection used is closing
2019-07-15 09:58:52 INFO  MetaStoreDirectSql:139 - Using direct SQL, underlying DB is DERBY
2019-07-15 09:58:52 INFO  ObjectStore:272 - Initialized ObjectStore
2019-07-15 09:58:52 INFO  HiveMetaStore:663 - Added admin role in metastore
2019-07-15 09:58:52 INFO  HiveMetaStore:672 - Added public role in metastore
2019-07-15 09:58:52 INFO  HiveMetaStore:712 - No user is added in admin role, since config is empty
2019-07-15 09:58:52 INFO  HiveMetaStore:746 - 0: get_all_databases
2019-07-15 09:58:52 INFO  audit:371 - ugi=root	ip=unknown-ip-addr	cmd=get_all_databases	
2019-07-15 09:58:52 INFO  HiveMetaStore:746 - 0: get_functions: db=default pat=*
2019-07-15 09:58:52 INFO  audit:371 - ugi=root	ip=unknown-ip-addr	cmd=get_functions: db=default pat=*	
2019-07-15 09:58:52 INFO  Datastore:77 - The class "org.apache.hadoop.hive.metastore.model.MResourceUri" is tagged as "embedded-only" so does not have its own datastore table.
2019-07-15 09:58:52 INFO  SessionState:641 - Created local directory: /tmp/8fd07625-00e6-4c7e-ab25-7f6ce58ff886_resources
2019-07-15 09:58:52 INFO  SessionState:641 - Created HDFS directory: /tmp/hive/root/8fd07625-00e6-4c7e-ab25-7f6ce58ff886
2019-07-15 09:58:52 INFO  SessionState:641 - Created local directory: /tmp/root/8fd07625-00e6-4c7e-ab25-7f6ce58ff886
2019-07-15 09:58:52 INFO  SessionState:641 - Created HDFS directory: /tmp/hive/root/8fd07625-00e6-4c7e-ab25-7f6ce58ff886/_tmp_space.db
2019-07-15 09:58:52 INFO  HiveClientImpl:54 - Warehouse location for Hive client (version 1.2.2) is file:/opt/spark-2.3.1/bin/spark-warehouse
2019-07-15 09:58:52 INFO  HiveMetaStore:746 - 0: get_database: default
2019-07-15 09:58:52 INFO  audit:371 - ugi=root	ip=unknown-ip-addr	cmd=get_database: default	
2019-07-15 09:58:52 INFO  HiveMetaStore:746 - 0: get_database: global_temp
2019-07-15 09:58:52 INFO  audit:371 - ugi=root	ip=unknown-ip-addr	cmd=get_database: global_temp	
2019-07-15 09:58:52 WARN  ObjectStore:568 - Failed to get database global_temp, returning NoSuchObjectException
2019-07-15 09:58:53 INFO  HiveMetaStore:746 - 0: get_database: traffic
2019-07-15 09:58:53 INFO  audit:371 - ugi=root	ip=unknown-ip-addr	cmd=get_database: traffic	
2019-07-15 09:58:53 WARN  ObjectStore:568 - Failed to get database traffic, returning NoSuchObjectException
Exception in thread "main" org.apache.spark.sql.catalyst.analysis.NoSuchDatabaseException: Database 'traffic' not found;
	at org.apache.spark.sql.catalyst.catalog.SessionCatalog.org$apache$spark$sql$catalyst$catalog$SessionCatalog$$requireDbExists(SessionCatalog.scala:174)
	at org.apache.spark.sql.catalyst.catalog.SessionCatalog.setCurrentDatabase(SessionCatalog.scala:256)
	at org.apache.spark.sql.execution.command.SetDatabaseCommand.run(databases.scala:59)
	at org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult$lzycompute(commands.scala:70)
	at org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult(commands.scala:68)
	at org.apache.spark.sql.execution.command.ExecutedCommandExec.executeCollect(commands.scala:79)
	at org.apache.spark.sql.Dataset$$anonfun$6.apply(Dataset.scala:190)
	at org.apache.spark.sql.Dataset$$anonfun$6.apply(Dataset.scala:190)
	at org.apache.spark.sql.Dataset$$anonfun$52.apply(Dataset.scala:3254)
	at org.apache.spark.sql.execution.SQLExecution$.withNewExecutionId(SQLExecution.scala:77)
	at org.apache.spark.sql.Dataset.withAction(Dataset.scala:3253)
	at org.apache.spark.sql.Dataset.<init>(Dataset.scala:190)
	at org.apache.spark.sql.Dataset$.ofRows(Dataset.scala:75)
	at org.apache.spark.sql.SparkSession.sql(SparkSession.scala:641)
	at com.producedate2hive.Data2Hive.main(Data2Hive.java:13)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52)
	at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:894)
	at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:198)
	at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:228)
	at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:137)
	at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
2019-07-15 09:58:53 INFO  SparkContext:54 - Invoking stop() from shutdown hook
2019-07-15 09:58:53 INFO  AbstractConnector:318 - Stopped Spark@53f83f19{HTTP/1.1,[http/1.1]}{0.0.0.0:4040}
2019-07-15 09:58:53 INFO  SparkUI:54 - Stopped Spark web UI at http://node4:4040
2019-07-15 09:58:54 INFO  StandaloneSchedulerBackend:54 - Shutting down all executors
2019-07-15 09:58:54 INFO  CoarseGrainedSchedulerBackend$DriverEndpoint:54 - Asking each executor to shut down
2019-07-15 09:58:54 INFO  MapOutputTrackerMasterEndpoint:54 - MapOutputTrackerMasterEndpoint stopped!
2019-07-15 09:58:54 INFO  MemoryStore:54 - MemoryStore cleared
2019-07-15 09:58:54 INFO  BlockManager:54 - BlockManager stopped
2019-07-15 09:58:54 INFO  BlockManagerMaster:54 - BlockManagerMaster stopped
2019-07-15 09:58:54 INFO  OutputCommitCoordinator$OutputCommitCoordinatorEndpoint:54 - OutputCommitCoordinator stopped!
2019-07-15 09:58:54 INFO  SparkContext:54 - Successfully stopped SparkContext
2019-07-15 09:58:54 INFO  ShutdownHookManager:54 - Shutdown hook called
2019-07-15 09:58:54 INFO  ShutdownHookManager:54 - Deleting directory /tmp/spark-1634ff5f-c1ba-4ce5-9f04-c2eb82e5aac0
2019-07-15 09:58:54 INFO  ShutdownHookManager:54 - Deleting directory /tmp/spark-632c2c0e-2443-438f-8166-4558633eb6de
hello_zzw
关注
  • 1
    点赞
  • 踩
  • 3
    收藏
    觉得还不错? 一键收藏
  • 2
    评论
专栏目录
org.apache.spark.sql.catalyst.analysis.NoSuchDatabaseException: Database ‘dwd‘ not found
打工日常
03-06 3012
spark操作hive时，报错如下： 实际上hive中是有这个库的，后来查看了一下，是因为spark没有开启对hive的支持 加上这个就好了
Mastering.Apache.Spark.2.x
07-20
The book opens with an overview of the Spark ecosystem. The book will introduce you to Project Catalyst and Tungsten. You will understand how Memory Management and Binary Processing, Cache-aware Computation, and Code Generation are used to speed things up dramatically. The book goes on to show how to incorporate H20 and Deeplearning4j for machine learning and Juypter Notebooks, Zeppelin, Docker and Kubernetes for cloud-based Spark. During the course of the book, you will also learn about the latest enhancements in Apache Spark 2.2, such as using the DataFrame and Dataset APIs exclusively, building advanced, fully automated Machine Learning pipelines with SparkML and perform graph analysis using the new GraphFrames API.
Catalyst电压监控器具1-至16-kbits串行接口的EEPROM..
01-19
Catalyst半导体公司新推出了3个系列的电压监控器件，它们包含业内宽选择范围的嵌入式存储器。CAT130xx系列具有1-，2-，4-，8-或16-kbit Microwire串行接口的嵌入式EEPROM。CAT140xx和CAT150xx系列则具有2-，4-，8-或16-kbit I2C和SPI串行接口的嵌入式EEPROM。       这3种新的电压监控器件系列产品为2.5V，3V，3.3V和5V系统提供在2.32V到4.63V之间的7个复位门限。复位输出可以是CMOS低电平或CMOS高电平。       Catalyst半导体公司行销副总裁Scott Brown说，“Catalys
Spark SQL 2.3.0：深入浅出
08-20
Spark SQL 2.3.0：深入浅出，看了下，还行，希望对大家有帮助
Spark SQL源码概览.pdf
01-11
Spark SQL 包含 3 个子项目：Core、Catalyst、Hive。其中 Catalyst 是核心的查询优化引 擎，独立于 Spark 平台；Spark SQL Core 封装 Catalyst，向应用程序提供 SparkSession、Dataset、 DataFrame 等 API（DataFrame 是 Dataset[Row]的别名）；Spark SQL Hive 则提供操作 Hive 的 接口。本文主要关注查询执行过程，不涉及 Hive。
spark1.6升级2.3读取hive表报错NoSuchDatabaseException: Database 'ods' not found
zhiaixiatian的专栏
10-30 2214
升级完spark后，通过spark-submit 操作hive报错，如下： org.apache.spark.sql.catalyst.analysis.NoSuchDatabaseException: Database 'ods' not found; at org.apache.spark.sql.catalyst.catalog.SessionCatalog.org$ap...
spark读取Hive出错： Database 'test1' not found
BigData_Mining的博客
07-03 5936
错误：找不到数据库。Exception in thread "main" org.apache.spark.sql.catalyst.analysis.NoSuchDatabaseException: Database 'test1' not found;解决方案：加上enableHiveSupport()即可，否则默认不读hive-site.xml。val spark = SparkSessio...
org.apache.spark.sql.AnalysisException: Try to map struct<>to Tuple1 异常
遥望......
10-01 3178
spark.read.parquet(mypath).as[MyEntity]会遇到下面异常 org.apache.spark.sql.AnalysisException: Try to map struct to Tuple1, but failed as the number of fields does not line up.; at org.apache.spark.sql.ca
spark写入hive中提示找不到数据库解决办法
这个人很懒什么都没有留下
04-26 3641
WARN metastore.ObjectStore: Failed to get database odsl, returning NoSuchObjectException Exception in thread “main” org.apache.spark.sql.catalyst.analysis.NoSuchDatabaseException: Database ‘odsl’ not found; 解决方法 将hive配置复制到spark cp hive-site.xml /opt/soft/s
Spark SQL优化器系统Catalyst的深入解析和应用.pdf
09-19
Spark SQL优化器系统Catalyst的深入解析和应用.pdf
spark-catalyst_2.11-2.4.0-cdh6.1.1.jar
05-14
spark-catalyst_2.11-2.4.0-cdh6.1.1.jar 是CDH版本的jar包。不好下载啊。
HiveContext 执行报错Exception in org.apache.spark.sql.AnalysisException: Table or view not found
weixin_37417954的博客
06-27 1108
-- 昨夜西风凋碧树，独上高楼，望尽天涯路 问题描述 通过 Spark 的 HiveContext 查询 Hive 中的数据的时候报错如下： Exception in thread "main" org.apache.spark.sql.AnalysisException: Table or view not found:...
Schema initialization FAILED! Metastore state would be inconsistent !!
热门推荐
zzw_17600691357的博客
04-29 1万+
[root@node3 hive-2.3.4]# schematool -dbType mysql -initSchema SLF4J: Class path contains multiple SLF4J bindings. SLF4J: Found binding in [jar:file:/opt/hive-2.3.4/lib/log4j-slf4j-impl-2.6.2.jar!/org/...
beeline连接hiveserver2
zzw_17600691357的博客
04-24 4954
在hive服务器端启动hiveserver2（即metastore服务所在的服务器） hiveserver2 通过beeline连接 有两种连接方式 1.beeline -u jdbc:hive2://localhost:10000/default -n root 2.beeline !connect jdbc:hive2://node3:10000/default 在beeline进行连...
hive --service metastore异常
zzw_17600691357的博客
04-23 4241
hive --service metastore启动元数据服务时报一下错误 MetaException(message:Version information not found in metastore. ) at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:8...
Hive Runtime Error while processing row {"id":202,"name":"zhangsan202","age":18}
zzw_17600691357的博客
04-29 2532
在查询数据时，查询的字段，由于和要插入的表的字段不对应，因此报了以下错误： Error: java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing row {"id":202,"name":"zhangsan202","age":...
hive执行sql时死机
zzw_17600691357的博客
04-29 994
在执行MR时，一直暂停，无法执行，如下： hive> select count(*) from person_basic; WARNING: Hive-on-MR is deprecated in Hive 2 and may not be available in the future versions. Consider using a different execution engin...
Hive jdbc运行方式
zzw_17600691357的博客
04-25 346
1.在服务器端启动hiveserver2，metastore服务所在的服务器 hive --service hiveserver2 2.代码和访问mysql数据库的一样 package com.bjsxt.hive; import java.sql.Connection; import java.sql.DriverManager; import java.sql.ResultSet; imp...
Exception in thread "main" org.apache.spark.sql.catalyst.analysis.NoSuchDatabaseException: Database 'test' not found;
最新发布
05-17
这个错误提示是因为你在Spark中尝试访问一个不存在的数据库。...spark.sql("CREATE DATABASE IF NOT EXISTS test") ``` 这将在Spark中创建一个名为“test”的新数据库，如果该数据库已经存在，则不会进行任何操作。

“相关推荐”对你有帮助么？

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
评论 2
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值