python上传下载集群hdfs（转载）

Ruthless people ?

已于 2022-03-23 10:34:37 修改

阅读量473

点赞数

文章标签： hdfs python

于 2022-03-22 17:49:58 首次发布

本文链接：https://blog.csdn.net/qq_46363373/article/details/123667088

版权

pyhdfs HDFS 文件上传文件下载本地文件系统

关键词由CSDN通过智能技术生成

class FileManager(object):

    # upload file to hdfs from local file system
    def file_upload(self, host, user_name, local_path, hdfs_path):
        print("file upload start")
        fs = pyhdfs.HdfsClient(hosts=host, user_name=user_name)
        print(fs.listdir('/'))
        fs.copy_from_local(local_path, hdfs_path)
        print("file upload finish")

    # # download file from hdfs file system
    # def file_down(self, host, user_name, local_path, hdfs_path):
    #     print("file download start")
    #     fs = pyhdfs.HdfsClient(hosts=host, user_name=user_name)
    #     fs.copy_to_local(hdfs_path, local_path)
    #     print("file download finish")


if __name__ == '__main__':
    file_manager = FileManager()
    file_manager.file_upload("192.168.12.101:50070", "admin", u"airport_inf.xls",
                             u"/airport_fare_data/his_data/airport_inf.xls")
    # file_manager.file_down("ip:50070", "admin", u"local_file.xlsx", u"hdfs_file.xlsx")