【Azure 存储服务】Hadoop集群中使用ADLS(Azure Data Lake Storage)过程中遇见执行PUT操作报错

🚀 优质资源分享 🚀

学习路线指引（点击解锁）	知识定位	人群定位
🧡 Python实战微信订餐小程序 🧡	进阶级	本课程是python flask+微信小程序的完美结合，从项目搭建到腾讯云部署上线，打造一个全栈订餐系统。
💛Python量化交易实战💛	入门级	手把手带你打造一个易扩展、更安全、效率更高的量化交易系统

问题描述

在Hadoop集中中，使用ADLS 作为数据源，在执行PUT操作（上传文件到ADLS中），遇见 400错误【put: Operation failed: “An HTTP header that’s mandatory for this request is not specified.”, 400】

启用Debug输出详细日志：

错误消息文本内容：

[hdfs@hadoop001 ~]$ hadoop fs -put a.txt abfs://adsl@xxxxxxxxxxxxx.blob.core.chinacloudapi.cn/test/a.txt
22/07/13 15:46:05 DEBUG util.Shell: setsid exited with exit code 0
22/07/13 15:46:05 DEBUG conf.Configuration: parsing URL jar:file:/usr/hdp/3.1.4.0-315/hadoop/hadoop-common-3.1.1.3.1.4.0-315.jar!/core-default.xml
22/07/13 15:46:05 DEBUG conf.Configuration: parsing input stream sun.net.www.protocol.jar.JarURLConnection$JarURLInputStream@4fe3c938
22/07/13 15:46:05 DEBUG conf.Configuration: parsing URL file:/etc/hadoop/3.1.4.0-315/0/core-site.xml
22/07/13 15:46:05 DEBUG conf.Configuration: parsing input stream java.io.BufferedInputStream@467aecef
22/07/13 15:46:05 DEBUG security.SecurityUtil: Setting hadoop.security.token.service.use\_ip to true
22/07/13 15:46:05 DEBUG security.Groups:  Creating new Groups object
22/07/13 15:46:05 DEBUG util.NativeCodeLoader: Trying to load the custom-built native-hadoop library...
22/07/13 15:46:05 DEBUG util.NativeCodeLoader: Loaded the native-hadoop library
22/07/13 15:46:05 DEBUG security.JniBasedUnixGroupsMapping: Using JniBasedUnixGroupsMapping for Group resolution
22/07/13 15:46:05 DEBUG security.JniBasedUnixGroupsMappingWithFallback: Group mapping impl=org.apache.hadoop.security.JniBasedUnixGroupsMapping
22/07/13 15:46:05 DEBUG security.Groups: Group mapping impl=org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback; cacheTimeout=300000; warningDeltaMs=5000
22/07/13 15:46:06 DEBUG core.Tracer: sampler.classes = ; loaded no samplers
22/07/13 15:46:06 DEBUG core.Tracer: span.receiver.classes = ; loaded no span receivers
22/07/13 15:46:06 DEBUG security.UserGroupInformation: hadoop login
22/07/13 15:46:06 DEBUG security.UserGroupInformation: hadoop login commit
22/07/13 15:46:06 DEBUG security.UserGroupInformation: using local user:UnixPrincipal: hdfs
22/07/13 15:46:06 DEBUG security.UserGroupInformation: Using user: "UnixPrincipal: hdfs" with name hdfs
22/07/13 15:46:06 DEBUG security.UserGroupInformation: User entry: "hdfs"
22/07/13 15:46:06 DEBUG security.UserGroupInformation: UGI loginUser:hdfs (auth:SIMPLE)
22/07/13 15:46:06 DEBUG core.Tracer: sampler.classes = ; loaded no samplers
22/07/13 15:46:06 DEBUG core.Tracer: span.receiver.classes = ; loaded no span receivers
22/07/13 15:46:06 DEBUG fs.FileSystem: Loading filesystems
22/07/13 15:46:06 DEBUG fs.FileSystem: file:// = class org.apache.hadoop.fs.LocalFileSystem from /usr/hdp/3.1.4.0-315/hadoop/hadoop-common-3.1.1.3.1.4.0-315.jar
22/07/13 15:46:06 DEBUG fs.FileSystem: viewfs:// = class org.apache.hadoop.fs.viewfs.ViewFileSystem from /usr/hdp/3.1.4.0-315/hadoop/hadoop-common-3.1.1.3.1.4.0-315.jar
22/07/13 15:46:06 DEBUG fs.FileSystem: har:// = class org.apache.hadoop.fs.HarFileSystem from /usr/hdp/3.1.4.0-315/hadoop/hadoop-common-3.1.1.3.1.4.0-315.jar
22/07/13 15:46:06 DEBUG fs.FileSystem: http:// = class org.apache.hadoop.fs.http.HttpFileSystem from /usr/hdp/3.1.4.0-315/hadoop/hadoop-common-3.1.1.3.1.4.0-315.jar
22/07/13 15:46:06 DEBUG fs.FileSystem: https:// = class org.apache.hadoop.fs.http.HttpsFileSystem from /usr/hdp/3.1.4.0-315/hadoop/hadoop-common-3.1.1.3.1.4.0-315.jar
22/07/13 15:46:06 DEBUG fs.FileSystem: hdfs:// = class org.apache.hadoop.hdfs.DistributedFileSystem from /usr/hdp/3.1.4.0-315/hadoop-hdfs/hadoop-hdfs-client-3.1.1.3.1.4.0-315.jar
22/07/13 15:46:06 DEBUG fs.FileSystem: webhdfs:// = class org.apache.hadoop.hdfs.web.WebHdfsFileSystem from /usr/hdp/3.1.4.0-315/hadoop-hdfs/hadoop-hdfs-client-3.1.1.3.1.4.0-315.jar
22/07/13 15:46:06 DEBUG fs.FileSystem: swebhdfs:// = class org.apache.hadoop.hdfs.web.SWebHdfsFileSystem from /usr/hdp/3.1.4.0-315/hadoop-hdfs/hadoop-hdfs-client-3.1.1.3.1.4.0-315.jar
22/07/13 15:46:06 DEBUG fs.FileSystem: gs:// = class com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystem from /usr/hdp/3.1.4.0-315/hadoop-mapreduce/gcs-connector-1.9.10.3.1.4.0-315-shaded.jar
22/07/13 15:46:06 DEBUG fs.FileSystem: s3n:// = class org.apache.hadoop.fs.s3native.NativeS3FileSystem from /usr/hdp/3.1.4.0-315/hadoop-mapreduce/hadoop-aws-3.1.1.3.1.4.0-315.jar
22/07/13 15:46:06 DEBUG fs.FileSystem: Looking for FS supporting abfs
22/07/13 15:46:06 DEBUG fs.FileSystem: looking for configuration option fs.abfs.impl
22/07/13 15:46:06 DEBUG fs.FileSystem: Filesystem abfs defined in configuration option
22/07/13 15:46:06 DEBUG fs.FileSystem: FS for abfs is class org.apache.hadoop.fs.azurebfs.AzureBlobFileSystem
22/07/13 15:46:06 DEBUG azurebfs.AzureBlobFileSystem: Initializing AzureBlobFileSystem for abfs://adsl@xxxxxxxxxxxxx.blob.core.chinacloudapi.cn/test/a.txt
22/07/13 15:46:06 DEBUG security.Groups: GroupCacheLoader - load.
22/07/13 15:46:06 WARN utils.SSLSocketFactoryEx: Failed to load OpenSSL. Falling back to the JSSE default.
22/07/13 15:46:06 DEBUG utils.SSLSocketFactoryEx: Removed Cipher - TLS\_ECDHE\_ECDSA\_WITH\_AES\_256\_GCM\_SHA384
22/07/13 15:46:06 DEBUG utils.SSLSocketFactoryEx: Removed Cipher - TLS\_ECDHE\_ECDSA\_WITH\_AES\_128\_GCM\_SHA256
22/07/13 15:46:06 DEBUG utils.SSLSocketFactoryEx: Removed Cipher - TLS\_ECDHE\_RSA\_WITH\_AES\_256\_GCM\_SHA384
22/07/13 15:46:06 DEBUG utils.SSLSocketFactoryEx: Removed Cipher - TLS\_RSA\_WITH\_AES\_256\_GCM\_SHA384
22/07/13 15