通过spark-shell向指定的sftp服务器写文件,报错:
我在spark-shell中执行,读取某个服务器上的数据,这个地方没有sftp,用的ssh的,出现了下面的问题
val df = spark.read.format("com.springml.spark.sftp").option("host", "10.10.16.8").option("username", "xxx").option("password", "xxx").option("fileType", "txt").load("/home/lsx/a.txt")
import com.jcraft.jsch.JSch
def sftp_scsh(username:String,password:String,host:String): Unit ={
val jsch = new JSch
jsch.getSession(username, host, 21)
val sshSession = jsch.getSession(username, host, 21)
System.out.println("Session created.")
sshSession.setPassword(password)
//认证
sshSession.setConfig("PreferredAuthentications", "password,gssapi-with-mic,publickey,keyboard-interactive")
sshSession.connect
System.out.println("Session connected.")
import com.jcraft.jsch.ChannelSftp
System.out.println("Opening Channel.")
val channel = sshSession.openChannel("ftp")
channel.connect
val sftp = channel.asInstanceOf[ChannelSftp]
System.out.println("Connected to " + host + ".")
System.out.println("Connect " + (if (sftp.isConnected) "successed"
else "failed"))
}
带访问服务器的/etc/ssh/ssh_config下的配置;这个yes,不允许通过api的方式访问服务器,
而通过FileZilla工具却可以,其实是因为这个工具做了代码设置
session.setConfig("userauth.gssapi-with-mic", "no");
session.setConfig("StrictHostKeyChecking", "no");