Sparksql 一个错误：org.apache.spark.sql.AnalysisException: character '　' not supported here;

最新推荐文章于 2022-11-15 17:31:43 发布

mtj66

最新推荐文章于 2022-11-15 17:31:43 发布

阅读量2.4w

点赞数

文章标签： sparksql

本文链接：https://blog.csdn.net/mtj66/article/details/52609809

版权

遇到如下错误，但是在hive中单独运行，或者是在spark-shell中单独运行的也是毫无问题的，为何偏偏在sparksql中出问题，而且不存在所说的那个 character ’　’ 。
还有就是我这个临时表本来是采用insert overwrite的也是正确的。
或许应该是，解析有点问题。

错误日志：
Exception in thread “main” org.apache.spark.sql.AnalysisException: character ’　’ not supported here; line 1 pos 34
at org.apache.spark.sql.hive.HiveQl $.createPlan(HiveQl.scala:318) at org.apache.spark.sql.hive.ExtendedHiveQlParser$ $anonfun$ hiveQl $1.apply(ExtendedHiveQlParser.scala:41) at org.apache.spark.sql.hive.ExtendedHiveQlParser$ $anonfun$ hiveQl $1.apply(ExtendedHiveQlParser.scala:40) at scala.util.parsing.combinator.Parsers$ Success.map(Parsers.scala:136)
at scala.util.parsing.combinator.Parsers $Success.map(Parsers.scala:135) at scala.util.parsing.combinator.Parsers$ Parser

a n o n f u n $ m a p $ 1. a p p l y (P a r s e r s . s c a l a : 242) a t s c a l a . u t i l . p a r s i n g . c o m b i n a t o r . P a r s e r s $ P a r s e r

$anonfun$map$1.apply(Parsers.scala:242) at scala.util.parsing.combinator.Parsers$Parser$ anonfun

map $map$ 1.apply(Parsers.scala:242)
at scala.util.parsing.combinator.Parsers

a n o n $ 3. a p p l y (P a r s e r s . s c a l a : 222) a t s c a l a . u t i l . p a r s i n g . c o m b i n a t o r . P a r s e r s $ P a r s e r

$anon$3.apply(Parsers.scala:222) at scala.util.parsing.combinator.Parsers$Parser$ anonfun

append $append$ 1

a n o n f u n $ a p p l y $ 2. a p p l y (P a r s e r s . s c a l a : 254) a t s c a l a . u t i l . p a r s i n g . c o m b i n a t o r . P a r s e r s $ P a r s e r

$anonfun$apply$2.apply(Parsers.scala:254) at scala.util.parsing.combinator.Parsers$Parser$ anonfun

append $append$ 1

a n o n f u n $ a p p l y $ 2. a p p l y (P a r s e r s . s c a l a : 254) a t s c a l a . u t i l . p a r s i n g . c o m b i n a t o r . P a r s e r s $ F a i l u r e . a p p e n d (P a r s e r s . s c a l a : 202) a t s c a l a . u t i l . p a r s i n g . c o m b i n a t o r . P a r s e r s $ P a r s e r

$anonfun$apply$2.apply(Parsers.scala:254) at scala.util.parsing.combinator.Parsers$Failure.append(Parsers.scala:202) at scala.util.parsing.combinator.Parsers$Parser$ anonfun

append $append$ 1.apply(Parsers.scala:254)
at scala.util.parsing.combinator.Parsers

Parser $Parser$

anonfun $anonfun$ append

1.apply(Parsers.scala:254)atscala.util.parsing.combinator.Parsers $1.apply(Parsers.scala:254) at scala.util.parsing.combinator.Parsers$

anon $anon$ 3.apply(Parsers.scala:222) at scala.util.parsing.combinator.Parsers

a n o n $ 2

$anon$2$ anonfun

apply $apply$ 14.apply(Parsers.scala:891)
at scala.util.parsing.combinator.Parsers

a n o n $ 2

$anon$2$ anonfun

apply $apply$ 14.apply(Parsers.scala:891)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:57)
at scala.util.parsing.combinator.Parsers

a n o n $ 2. a p p l y (P a r s e r s . s c a l a : 890) a t s c a l a . u t i l . p a r s i n g . c o m b i n a t o r . P a c k r a t P a r s e r s

$anon$2.apply(Parsers.scala:890) at scala.util.parsing.combinator.PackratParsers$ anon

1.apply(PackratParsers.scala:110)atorg.apache.spark.sql.catalyst.AbstractSparkSQLParser.parse(AbstractSparkSQLParser.scala:34)atorg.apache.spark.sql.hive.HiveQl $1.apply(PackratParsers.scala:110) at org.apache.spark.sql.catalyst.AbstractSparkSQLParser.parse(AbstractSparkSQLParser.scala:34) at org.apache.spark.sql.hive.HiveQl$ .parseSql(HiveQl.scala:295)
at org.apache.spark.sql.hive.HiveQLDialect

a n o n f u n $ p a r s e $ 1. a p p l y (H i v e C o n t e x t . s c a l a : 66) a t o r g . a p a c h e . s p a r k . s q l . h i v e . H i v e Q L D i a l e c t

$anonfun$parse$1.apply(HiveContext.scala:66) at org.apache.spark.sql.hive.HiveQLDialect$ anonfun

parse $parse$ 1.apply(HiveContext.scala:66)
at org.apache.spark.sql.hive.client.ClientWrapper

a n o n f u n $ w i t h H i v e S t a t e $ 1. a p p l y (C l i e n t W r a p p e r . s c a l a : 281) a t o r g . a p a c h e . s p a r k . s q l . h i v e . c l i e n t . C l i e n t W r a p p e r . l i f t e d T r e e 1 $ 1 (C l i e n t W r a p p e r . s c a l a : 228) a t o r g . a p a c h e . s p a r k . s q l . h i v e . c l i e n t . C l i e n t W r a p p e r . r e t r y L o c k e d (C l i e n t W r a p p e r . s c a l a : 227) a t o r g . a p a c h e . s p a r k . s q l . h i v e . c l i e n t . C l i e n t W r a p p e r . w i t h H i v e S t a t e (C l i e n t W r a p p e r . s c a l a : 270) a t o r g . a p a c h e . s p a r k . s q l . h i v e . H i v e Q L D i a l e c t . p a r s e (H i v e C o n t e x t . s c a l a : 65) a t o r g . a p a c h e . s p a r k . s q l . S Q L C o n t e x t

$anonfun$withHiveState$1.apply(ClientWrapper.scala:281) at org.apache.spark.sql.hive.client.ClientWrapper.liftedTree1$1(ClientWrapper.scala:228) at org.apache.spark.sql.hive.client.ClientWrapper.retryLocked(ClientWrapper.scala:227) at org.apache.spark.sql.hive.client.ClientWrapper.withHiveState(ClientWrapper.scala:270) at org.apache.spark.sql.hive.HiveQLDialect.parse(HiveContext.scala:65) at org.apache.spark.sql.SQLContext$ anonfun

2.apply(SQLContext.scala:211)atorg.apache.spark.sql.SQLContext $2.apply(SQLContext.scala:211) at org.apache.spark.sql.SQLContext$

anonfun $anonfun$ 2.apply(SQLContext.scala:211) at org.apache.spark.sql.execution.SparkSQLParser

a n o n f u n $ o r g $ a p a c h e $ s p a r k $ s q l $ e x e c u t i o n $ S p a r k S Q L P a r s e r

$anonfun$org$apache$spark$sql$execution$SparkSQLParser$ others

1.apply(SparkSQLParser.scala:114)atorg.apache.spark.sql.execution.SparkSQLParser $1.apply(SparkSQLParser.scala:114) at org.apache.spark.sql.execution.SparkSQLParser$

anonfun $anonfun$ org

apache $apache$ spark

sql $sql$ execution

SparkSQLParser $SparkSQLParser$

others $others$ 1.apply(SparkSQLParser.scala:113) at scala.util.parsing.combinator.Parsers

Success.map(Parsers.scala:136)atscala.util.parsing.combinator.Parsers $Success.map(Parsers.scala:136) at scala.util.parsing.combinator.Parsers$ Success.map(Parsers.scala:135) at scala.util.parsing.combinator.Parsers

Parser $Parser$

anonfun $anonfun$ map

1.apply(Parsers.scala:242)atscala.util.parsing.combinator.Parsers $1.apply(Parsers.scala:242) at scala.util.parsing.combinator.Parsers$ Parser

a n o n f u n $ m a p $ 1. a p p l y (P a r s e r s . s c a l a : 242) a t s c a l a . u t i l . p a r s i n g . c o m b i n a t o r . P a r s e r s

$anonfun$map$1.apply(Parsers.scala:242) at scala.util.parsing.combinator.Parsers$ anon

3.apply(Parsers.scala:222)atscala.util.parsing.combinator.Parsers $3.apply(Parsers.scala:222) at scala.util.parsing.combinator.Parsers$ Parser

a n o n f u n $ a p p e n d $ 1

$anonfun$append$1$ anonfun

apply $apply$ 2.apply(Parsers.scala:254) at scala.util.parsing.combinator.Parsers

Parser $Parser$

anonfun $anonfun$ append

1 $1$

anonfun $anonfun$ apply

2.apply(Parsers.scala:254)atscala.util.parsing.combinator.Parsers $2.apply(Parsers.scala:254) at scala.util.parsing.combinator.Parsers$ Failure.append(Parsers.scala:202) at scala.util.parsing.combinator.Parsers

Parser $Parser$

anonfun $anonfun$ append

1.apply(Parsers.scala:254)atscala.util.parsing.combinator.Parsers $1.apply(Parsers.scala:254) at scala.util.parsing.combinator.Parsers$ Parser

a n o n f u n $ a p p e n d $ 1. a p p l y (P a r s e r s . s c a l a : 254) a t s c a l a . u t i l . p a r s i n g . c o m b i n a t o r . P a r s e r s

$anonfun$append$1.apply(Parsers.scala:254) at scala.util.parsing.combinator.Parsers$ anon

3.apply(Parsers.scala:222)atscala.util.parsing.combinator.Parsers $3.apply(Parsers.scala:222) at scala.util.parsing.combinator.Parsers$

anon $anon$ 2

a n o n f u n $ a p p l y $ 14. a p p l y (P a r s e r s . s c a l a : 891) a t s c a l a . u t i l . p a r s i n g . c o m b i n a t o r . P a r s e r s

$anonfun$apply$14.apply(Parsers.scala:891) at scala.util.parsing.combinator.Parsers$ anon

2 $2$

anonfun $anonfun$ apply

14.apply(Parsers.scala:891)atscala.util.DynamicVariable.withValue(DynamicVariable.scala:57)atscala.util.parsing.combinator.Parsers $14.apply(Parsers.scala:891) at scala.util.DynamicVariable.withValue(DynamicVariable.scala:57) at scala.util.parsing.combinator.Parsers$

anon $anon$ 2.apply(Parsers.scala:890) at scala.util.parsing.combinator.PackratParsers

a n o n $ 1. a p p l y (P a c k r a t P a r s e r s . s c a l a : 110) a t o r g . a p a c h e . s p a r k . s q l . c a t a l y s t . A b s t r a c t S p a r k S Q L P a r s e r . p a r s e (A b s t r a c t S p a r k S Q L P a r s e r . s c a l a : 34) a t o r g . a p a c h e . s p a r k . s q l . S Q L C o n t e x t

$anon$1.apply(PackratParsers.scala:110) at org.apache.spark.sql.catalyst.AbstractSparkSQLParser.parse(AbstractSparkSQLParser.scala:34) at org.apache.spark.sql.SQLContext$ anonfun

1.apply(SQLContext.scala:208)atorg.apache.spark.sql.SQLContext $1.apply(SQLContext.scala:208) at org.apache.spark.sql.SQLContext$

anonfun $anonfun$ 1.apply(SQLContext.scala:208)
at org.apache.spark.sql.execution.datasources.DDLParser.parse(DDLParser.scala:43)
at org.apache.spark.sql.SQLContext.parseSql(SQLContext.scala:231)
at org.apache.spark.sql.hive.HiveContext.parseSql(HiveContext.scala:333)
at org.apache.spark.sql.SQLContext.sql(SQLContext.scala:817)
at telematic.spark.hbase.TestConvertPythonToHiveUseUDF

.main(TestConvertPythonToHiveUseUDF.scala:111)attelematic.spark.hbase.TestConvertPythonToHiveUseUDF.main(TestConvertPythonToHiveUseUDF.scala)atsun.reflect.NativeMethodAccessorImpl.invoke0(NativeMethod)atsun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)atsun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)atjava.lang.reflect.Method.invoke(Method.java:483)atorg.apache.spark.deploy.SparkSubmit $.main(TestConvertPythonToHiveUseUDF.scala:111) at telematic.spark.hbase.TestConvertPythonToHiveUseUDF.main(TestConvertPythonToHiveUseUDF.scala) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:483) at org.apache.spark.deploy.SparkSubmit$ .org

apache $apache$ spark

deploy $deploy$ SparkSubmit$$runMain(SparkSubmit.scala:731)
at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:181)
at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:206)
at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:121)
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
16/09/21 16:31:48 INFO spark.SparkContext: Invoking stop() from shutdown hook
16/09/21 16:31:48 INFO zookeeper.ZooKeeper: Session: 0x35701e8ba03a482 closed
16/09/21 16:31:48 INFO CuratorFrameworkSingleton: Closing ZooKeeper client.
16/09/21 16:31:48 INFO zookeeper.ClientCnxn: EventThread shut down
16/09/21 16:31:48 INFO ui.SparkUI: Stopped Spark web UI at http://10.172.10.167:4041
16/09/21 16:31:48 INFO cluster.YarnClientSchedulerBackend: Shutting down all executors
16/09/21 16:31:48 INFO cluster.YarnClientSchedulerBackend: Asking each executor to shut down
16/09/21 16:31:48 INFO cluster.YarnClientSchedulerBackend: Interrupting monitor thread
16/09/21 16:31:48 INFO cluster.YarnClientSchedulerBackend: Stopped
16/09/21 16:31:48 INFO spark.MapOutputTrackerMasterEndpoint: MapOutputTrackerMasterEndpoint stopped!
16/09/21 16:31:48 INFO storage.MemoryStore: MemoryStore cleared
16/09/21 16:31:48 INFO storage.BlockManager: BlockManager stopped
16/09/21 16:31:48 INFO storage.BlockManagerMaster: BlockManagerMaster stopped
16/09/21 16:31:48 INFO scheduler.OutputCommitCoordinator

OutputCommitCoordinatorEndpoint:OutputCommitCoordinatorstopped!16/09/2116:31:48INFOspark.SparkContext:SuccessfullystoppedSparkContext16/09/2116:31:48INFOutil.ShutdownHookManager:Shutdownhookcalled16/09/2116:31:48INFOutil.ShutdownHookManager:Deletingdirectory/tmp/spark−a6c81865−4411−4be7−bdc0−93c82f4e75d216/09/2116:31:48INFOutil.ShutdownHookManager:Deletingdirectory/tmp/spark−95d498f0−416f−4d6d−a559−aa083556877516/09/2116:31:48INFOremote.RemoteActorRefProvider $OutputCommitCoordinatorEndpoint: OutputCommitCoordinator stopped! 16/09/21 16:31:48 INFO spark.SparkContext: Successfully stopped SparkContext 16/09/21 16:31:48 INFO util.ShutdownHookManager: Shutdown hook called 16/09/21 16:31:48 INFO util.ShutdownHookManager: Deleting directory /tmp/spark-a6c81865-4411-4be7-bdc0-93c82f4e75d2 16/09/21 16:31:48 INFO util.ShutdownHookManager: Deleting directory /tmp/spark-95d498f0-416f-4d6d-a559-aa0835568775 16/09/21 16:31:48 INFO remote.RemoteActorRefProvider$ RemotingTerminator: Shutting down remote daemon.
16/09/21 16:31:48 INFO remote.RemoteActorRefProvider$RemotingTerminator: Remote daemon shut down; proceeding with flushing remote transports.

mtj66

关注

0
点赞
踩
0

收藏

觉得还不错? 一键收藏
打赏
1
评论
Sparksql 一个错误：org.apache.spark.sql.AnalysisException: character '　' not supported here;

遇到如下错误，但是在hive中单独运行，或者是在spark-shell中单独运行的也是毫无问题的，为何偏偏在sparksql中出问题，而且不存在所说的那个 character ’　’ 。还有就是我这个临时表本来是采用insert overwrite的也是正确的。或许应该是，解析有点问题。错误日志： Exception in thread “main” org.apache.spark.sq
复制链接

扫一扫