最近接到一个需求:监控windows环境目录下面的文件,如果有更新,那么检查这个文件在Linux服务器上面是否存在,如果不存在则传输到Linux上面
首先考虑使用ftplib模块,但是在使用中发现windows的文件在一个局域网共享目录下面,而不是一个ftp服务器,所以行不通
然后在网上查到了paramiko模块,安装:
pip install paramiko
首先创建一个连接Linux的方法:
def connection_server(server): try: ssh = paramiko.SSHClient() //这个方法可以屏蔽knowhost的识别,否则如果你的连接端设备不在Linux的knohost里面,将会被Linux拒绝连接 ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy()) ssh.connect(server, port=22, username=username, password=password, timeout=5) print(ssh) stdin,stdout,stderr=client.exec_command("cd/home/public/Release/Nightly_Builds/Release;ls") result = stdout.read().decode('utf-8') print("result:", result) except Exception as e: print("------connect to server error-------") return False return ssh finally: 在finally里面关闭连接,以免在文件拷贝过程中出错,但是进程仍然存在,无论拷贝是否成功,要关闭连接,释放进程 ssh.close()
连接服务器后,可以使用这个方法来远程执行Linux命令:
def check_file_on_server(windows, linux): download_list = check_file_time(windows) client = connection_server(linux) print("type for client:", client) //使用分号可以作为管道来执行Linux命令 stdin, stdout, stderr = client.exec_command("cd /home/test;ls") result = stdout.read().decode('utf-8') print("result:", result)
从windows向Linux传输文件:
//put_file接收参数:连接实例,需要传输的文件,目标文件 def put_file(windows_file, linux_file, img): # client = connection_server(linux_server) # sftp = client.open_sftp() //定义最大文件size MAX_TRANSFER_SIZE = 5242880 client = paramiko.Transport(linux_server, 22) client.connect(username="cnbwu", password="123456") sftp = paramiko.SFTPClient.from_transport(client, window_size=MAX_TRANSFER_SIZE, max_packet_size=MAX_TRANSFER_SIZE) #这里使用os.path把路径格式化为标准路径,否则可能因为中文字符、代码拷贝过程中导致字符串修改无法识别路径而报错 sftp.put(os.path.join(windows_file, img), os.path.join(linux_file, img)) # with open(os.path.join(windows_file, img), 'rb') as fp: # shutil.copyfileobj(fp, sftp.open(os.path.join(linux_file, img)), mode="w") client.close()
需要注意的是,使用sftp方法传输速度默认只有几十KB,经过一番搜索,发现在paramiko库下面的sftp_client.py中有一个_transfer_with_callback方法定义了默认的值只有几十KB,可以直接修改最大传输速度:
15:32:54 sftp.put(windows_file, linux_file)
15:32:54 File "C:\Users\AppData\Local\Programs\Python\Python36\lib\site-packages\paramiko\sftp_client.py", line 759, in put
15:32:54 return self.putfo(fl, remotepath, file_size, callback, confirm)
15:32:54 File "C:\Users\AppData\Local\Programs\Python\Python36\lib\site-packages\paramiko\sftp_client.py", line 717, in putfo
15:32:54 reader=fl, writer=fr, file_size=file_size, callback=callback
15:32:54 File "C:\Users\AppData\Local\Programs\Python\Python36\lib\site-packages\paramiko\sftp_client.py", line 678, in _transfer_with_callback
15:32:54 data = reader.read(3145728)
15:32:54 OSError: [Errno 22] Invalid argument
15:32:54 Build step 'Execute Python script' marked build as failure
def _transfer_with_callback(self, reader, writer, file_size, callback):
size = 0
while True://我手动修改到了10MB大小,以字节来计算(实际速度还是慢。。)
data = reader.read(10145728)
writer.write(data)
size += len(data)#修改为判断文件读取完毕再中断循环:
#if len(data) == 0:if size == file_size:
break
if callback is not None:
callback(size, file_size)
return size