I have installed MySql in a Linux VM (in a windows desktop), I am trying to connect to MySql from a Spark application running in a parent windows OS. While connecting I am getting below error...
Here is the sample code
def main(args: Array[String]) {
val conf = new SparkConf()
.setAppName("LoadMySql")
.setMaster("local[*]")
.set("spark.sql.warehouse.dir", "file:///C:/temp")
val spark = SparkSession
.builder
.config(conf)
.getOrCreate()
val option = Map("url" -> "jdbc:mysql://VMIP:3306/TestDB", "dbtable" -> "Users", "user" -> "root", "password" -> "*****", "driver" -> "com.mysql.jdbc.Driver")
val dataRDD = spark.sqlContext.read.format("org.apache.spark.sql.jdbc").options(option).load()
}
Error...
Exception in thread "main" java.lang.NullPointerException
at org.apache.spark.sql.execution.datasources.jdbc.JDBCRDD$.resolveTable(JDBCRDD.scala:158)
at org.apache.spark.sql.execution.datasources.jdbc.JDBCRelation.(JDBCRelation.scala:117)
at org.apache.spark.sql.execution.datasources.jdbc.JdbcRelationProvider.createRelation(JdbcRelationProvider.scala:53)
at org.apache.spark.sql.execution.datasources.DataSource.resolveRelation(DataSource.scala:315)
at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:149)
at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:122)
at com.Lab.SparkAPI.LoadMySql$.main(LoadMySql.scala:29)
at com.Lab.SparkAPI.LoadMySql.main(LoadMySql.scala)
I already changed bind-address property in /etc/mysql/mysql.conf.d/mysqld.cnf.
I have tried few options for bind-address - linux VM IP, 0.0.0.0 and completely commenting it.
None of the options worked so far.
My guess is, my windows application is not able to access MySql which is running in a VM.
Can somebody help me in resolving this issue?
Thanks in advance.
解决方案
Finally I am able to connect MySQL remotely...
I did below 2 settings...
GRANT ALL ON DBName.* TO "root"@'%' IDENTIFIED BY 'password';
modify /etc/mysql/mysql.conf.d/mysqld.cnf to comment #bind-address = 127.0.0.1 under [mysqld] section
This question was answered here