目录
背景
目前公司业务主要是在专有网络中展开,所以对于云相关应用较少。在某项目中,需要将lob保存至OSS,但是在保存过程中,出现了很多问题。包括但不限于OSS权限问题、应用程序链接模式问题等等。过程中需要指导现场运维人员明确判断问题点,并且将判断结果及时向开发人员、阿里云支持人员沟通。
简介
Object Storage Service 对象存储服务(OSS),是阿里云提供的海量、安全、低成本、高可靠的云存储服务,提供了12个9的数据持久性和4个9的数据可用性。
阿里提供了很全面且方便的学习资料,可参考。
https://help.aliyun.com/learn/learningpath/oss.html?spm=5176.7933691.J_7281917710.7.195e4c59hR0qHd
JAVA SDK
我们的应用程序调用OSS JAVA SDK实现对OSS的使用,但程序源码并未开放,且开发人员支持有限,因此需要直接上手,根据Demo验证OSS的使用。
过程分为如下步骤
1.通过Maven安装sdk
在pom.xml中增加依赖。
<dependency>
<groupId>com.aliyun.oss</groupId>
<artifactId>aliyun-sdk-oss</artifactId>
<version>3.10.2</version>
</dependency>
2.定义参数
该程序需要发给驻场运维使用,因此需要将参数暴露出来,使其可配置。
String endpoint = "http://oss-cn-beijing.aliyuncs.com";
String accessKeyId = "accessKeyId ";
String accessKeySecret = "accessKeySecret ";
String bucketName = "yeqiyu-test";
String firstKey = "my-first-key";
Boolean supportCname = false;
String protocol = "HTTP";
Boolean verifySSLEnable = true;
Boolean redirectEnable = true;
Boolean SLDEnabled = false;
Boolean deleteFiles=true;
3.新建OSSClient
OSS ossClient = new OSSClientBuilder().build(endpoint, accessKeyId, accessKeySecret, conf);
4.验证Bucket
if (ossClient.doesBucketExist(bucketName)) {
System.out.println("您已经创建Bucket:" + bucketName + "。");
} else {
System.out.println("您的Bucket不存在,创建Bucket:" + bucketName + "。");
// 创建Bucket。详细请参看“SDK手册 > Java-SDK > 管理Bucket”。
// 链接地址是:https://help.aliyun.com/document_detail/oss/sdk/java-sdk/manage_bucket.html?spm=5176.docoss/sdk/java-sdk/init
ossClient.createBucket(bucketName);
}
// 查看Bucket信息。详细请参看“SDK手册 > Java-SDK > 管理Bucket”。
// 链接地址是:https://help.aliyun.com/document_detail/oss/sdk/java-sdk/manage_bucket.html?spm=5176.docoss/sdk/java-sdk/init
BucketInfo info = ossClient.getBucketInfo(bucketName);
System.out.println("Bucket " + bucketName + "的信息如下:");
System.out.println("\t数据中心:" + info.getBucket().getLocation());
System.out.println("\t创建时间:" + info.getBucket().getCreationDate());
System.out.println("\t用户标志:" + info.getBucket().getOwner());
5.写入文件
// 把字符串存入OSS,Object的名称为firstKey。详细请参看“SDK手册 > Java-SDK > 上传文件”。
// 链接地址是:https://help.aliyun.com/document_detail/oss/sdk/java-sdk/upload_object.html?spm=5176.docoss/user_guide/upload_object
InputStream is = new ByteArrayInputStream("Hello OSS".getBytes());
ossClient.putObject(bucketName, firstKey, is);
System.out.println("Object:" + firstKey + "存入OSS成功。");
// 下载文件。详细请参看“SDK手册 > Java-SDK > 下载文件”。
// 链接地址是:https://help.aliyun.com/document_detail/oss/sdk/java-sdk/download_object.html?spm=5176.docoss/sdk/java-sdk/manage_object
OSSObject ossObject = ossClient.getObject(bucketName, firstKey);
InputStream inputStream = ossObject.getObjectContent();
StringBuilder objectContent = new StringBuilder();
BufferedReader reader = new BufferedReader(new InputStreamReader(inputStream));
while (true) {
String line = reader.readLine();
if (line == null)
break;
objectContent.append(line);
}
inputStream.close();
System.out.println("Object:" + firstKey + "的内容是:" + objectContent);
// 文件存储入OSS,Object的名称为fileKey。详细请参看“SDK手册 > Java-SDK > 上传文件”。
// 链接地址是:https://help.aliyun.com/document_detail/oss/sdk/java-sdk/upload_object.html?spm=5176.docoss/user_guide/upload_object
String fileKey = "README.md";
ossClient.putObject(bucketName, fileKey, new File("README.md"));
System.out.println("Object:" + fileKey + "存入OSS成功。");
// 查看Bucket中的Object。详细请参看“SDK手册 > Java-SDK > 管理文件”。
// 链接地址是:https://help.aliyun.com/document_detail/oss/sdk/java-sdk/manage_object.html?spm=5176.docoss/sdk/java-sdk/manage_bucket
ObjectListing objectListing = ossClient.listObjects(bucketName);
List<OSSObjectSummary> objectSummary = objectListing.getObjectSummaries();
System.out.println("您有以下Object:");
for (OSSObjectSummary object : objectSummary) {
System.out.println("\t" + object.getKey());
}
6.删除文件
if(deleteFiles) {
ossClient.deleteObject(bucketName, firstKey);
System.out.println("删除Object:" + firstKey + "成功。");
ossClient.deleteObject(bucketName, fileKey);
System.out.println("删除Object:" + fileKey + "成功。");
}
过程中的问题
错误SignatureDoesNotMatch
Unable to execute HTTP request: The request signature we calculated does not match the signature you provided. Check your key and signing method.
[ErrorCode]: SignatureDoesNotMatch
[RequestId]: xxxx
[HostId]: xxxxx
[ResponseError]:
<?xml version="1.0" encoding="UTF-8"?>
<Error>
<Code>SignatureDoesNotMatch</Code>
<Message>The request signature we calculated does not match the signature you provided. Check your key and signing method.</Message>
<RequestId>xxxx</RequestId>
<HostId>xxxxxxx</HostId>
<OSSAccessKeyId>xxxxx</OSSAccessKeyId>
<SignatureProvided>xxxx/sus/6+SITgQ=</SignatureProvided>
<StringToSign>GET
Fri, 28 May 2021 11:00:20 GMT
/?acl</StringToSign>
<StringToSignBytes>47 45 54 0A 0A 0A 46 72 69 2C 20 32 38 20 4D 61 79 20 32 30 32 31 20 31 31 3A 30 30 3A 32 30 20 47 4D 54 0A 2F 3F 61 63 6C </StringToSignBytes>
</Error>
根据错误提示SignatureDoesNotMatch,大致判断为签名不匹配。但是,通过OSS Tool可以连接,因此可以排除账号权限问题,问题点在于程序调用时,提交的参数与OSS Tool不同。
修改参数supportCname改为fals后解决。
尝试过的其他的但无效的方法:
配置时,将Bucket放在endpoint后边。
更换SDK版本。
错误AccessDenied
该错误是因为账号并没有GetBucketInfo权限,因此Demo在获取Bucket信息时失败。联系阿里云客服,增加该权限后解决。
附录1 权限常见错误描述
OSS返回的权限相关的错误及原因见下表。
错误 | 原因 | 解决方法 |
ErrorCode: AccessDenied ErrorMessage: The bucket you are attempting to access must be addressed using the specified endpoint. Please send all future requests to this endpoint. | Bucket和Endpoint不符 | AccessDenied.The bucket you are attempting to…错误 |
ErrorCode: AccessDenied | 说明访问OSS的用户没有当前操作的权限。 | AccessDenied.AccessDenied错误 |
ErrorCode: InvalidAccessKeyId | 原因可能是AccessKeyID禁用或不存在。 | InvalidAccessKeyId.The OSS Access Key Id…错误 |
ErrorCode: SignatureDoesNotMatch | 签名不匹配。 | SignatureDoesNotMatch.The request signature we calculated…错误 |
ErrorCode: AccessDenied ErrorMessage: You are forbidden to list buckets. | 无ListBuckets权限 | 如果需要修改权限,可参考基于读写权限ACL的权限控制中的权限列表赋予相应的权限。 |
ErrorCode: AccessDenied ErrorMessage: You do not have write acl permission on this object | 无SetObjectAcl权限 | |
ErrorCode: AccessDenied ErrorMessage: You do not have read acl permission on this object. | 无GetObjectAcl权限 | |
ErrorCode: AccessDenied ErrorMessage: The bucket you access does not belong to you. | 子用户没有Bucket管理的权限(如GetBucketAcl CreateBucket、DeleteBucket SetBucketReferer、 GetBucketReferer等) | 如果需要修改权限,可参考教程示例:使用RAM Policy控制OSS的访问权限修改权限。 |
ErrorCode: AccessDenied ErrorMessage: You have no right to access this object because of bucket acl. | 子用户/临时用户没有访问Object的权限(如putObject getObject、appendObject deleteObject、postObject)等 | |
ErrorCode: AccessDenied ErrorMessage: Access denied by authorizer’s policy. | 临时用户访问无权限,该临时用户角色扮演指定授权策略,该授权策略无权限 | |
ErrorCode: AccessDenied ErrorMessage: You have no right to access this object. | 子用户/临时用户无当前操作权限(如initiateMultipartUpload等) | |
ErrorCode: AccessDenied ErrorMessage: Invalid according to Policy: Policy expired. | PostObject中Policy无效 | PostObject |
ErrorCode: AccessDenied ErrorMessage: Invalid according to Policy: Policy Condition failed:["eq", "$Content-Type", "application/octet-stream"] … | Content-Type限定了,比如请求中Content-Type限定为image/png,而实际与限定不符。 | 设置Content-Type |
附录2 java sdk demo
/**
* 示例说明
*
* HelloOSS是OSS Java SDK的示例程序,您可以修改endpoint、accessKeyId、accessKeySecret、bucketName后直接运行。
* 运行方法请参考README。
*
* 本示例中的并不包括OSS Java SDK的所有功能,详细功能及使用方法,请参看“SDK手册 > Java-SDK”,
* 链接地址是:https://help.aliyun.com/document_detail/oss/sdk/java-sdk/preface.html?spm=5176.docoss/sdk/java-sdk/。
*
* 调用OSS Java SDK的方法时,抛出异常表示有错误发生;没有抛出异常表示成功执行。
* 当错误发生时,OSS Java SDK的方法会抛出异常,异常中包括错误码、错误信息,详细请参看“SDK手册 > Java-SDK > 异常处理”,
* 链接地址是:https://help.aliyun.com/document_detail/oss/sdk/java-sdk/exception.html?spm=5176.docoss/api-reference/error-response。
*
* OSS控制台可以直观的看到您调用OSS Java SDK的结果,OSS控制台地址是:https://oss.console.aliyun.com/index#/。
* OSS控制台使用方法请参看文档中心的“控制台用户指南”, 指南的来链接地址是:https://help.aliyun.com/document_detail/oss/getting-started/get-started.html?spm=5176.docoss/user_guide。
*
* OSS的文档中心地址是:https://help.aliyun.com/document_detail/oss/user_guide/overview.html。
* OSS Java SDK的文档地址是:https://help.aliyun.com/document_detail/oss/sdk/java-sdk/install.html?spm=5176.docoss/sdk/java-sdk。
*
*/
package com.aliyun.oss.demo;
import java.io.BufferedReader;
import java.io.ByteArrayInputStream;
import java.io.File;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.util.List;
import com.aliyun.oss.ClientBuilderConfiguration;
import com.aliyun.oss.ClientException;
import com.aliyun.oss.OSS;
import com.aliyun.oss.OSSClientBuilder;
import com.aliyun.oss.OSSException;
import com.aliyun.oss.common.comm.Protocol;
import com.aliyun.oss.model.BucketInfo;
import com.aliyun.oss.model.OSSObject;
import com.aliyun.oss.model.OSSObjectSummary;
import com.aliyun.oss.model.ObjectListing;
import org.apache.log4j.Logger;
import org.apache.log4j.PropertyConfigurator;
public class HelloOSS {
static Logger logger = Logger.getLogger(HelloOSS.class);
public static void main(String[] args) {
// 日志配置,OSS Java SDK使用log4j记录错误信息。示例程序会在工程目录下生成“oss-demo.log”日志文件,默认日志级别是INFO。
// 日志的配置文件是“conf/log4j.properties”,如果您不需要日志,可以没有日志配置文件和下面的日志配置。
PropertyConfigurator.configure("conf/log4j.properties");
logger.info("Started");
String endpoint = "http://oss-cn-beijing.aliyuncs.com";
String accessKeyId = "accessKeyId ";
String accessKeySecret = "accessKeySecret ";
String bucketName = "yeqiyu-test";
String firstKey = "my-first-key";
Boolean supportCname = false;
String protocol = "HTTP";
Boolean verifySSLEnable = true;
// Boolean redirectEnable = true;
Boolean SLDEnabled = false;
Boolean deleteFiles=true;
FileUtil fu = new FileUtil();
List<String> cf = fu.read("OSSDemo.properties");
System.out.println("----------读取配置文件----------");
for (String c : cf) {
if (c.indexOf("endpoint") > -1) {
endpoint = c.split("=")[1];
System.out.println("endpoint="+endpoint);
}
if (c.indexOf("accessKeyId") > -1) {
accessKeyId = c.split("=")[1];
System.out.println("accessKeyId="+accessKeyId);
}
if (c.indexOf("accessKeySecret") > -1) {
accessKeySecret = c.split("=")[1];
System.out.println("accessKeySecret="+accessKeySecret);
}
if (c.indexOf("bucketName") > -1) {
bucketName = c.split("=")[1];
System.out.println("bucketName="+bucketName);
}
if (c.indexOf("firstKey") > -1) {
firstKey = c.split("=")[1];
System.out.println("firstKey="+firstKey);
}
if (c.indexOf("supportCname") > -1) {
if("true".equals(c.split("=")[1])) {
supportCname=true;
}else {
supportCname=false;
}
System.out.println("supportCname="+supportCname);
}
if (c.indexOf("protocol") > -1) {
protocol = c.split("=")[1];
System.out.println("protocol="+protocol);
}
if (c.indexOf("verifySSLEnable") > -1) {
if("true".equals(c.split("=")[1])) {
verifySSLEnable=true;
}else {
verifySSLEnable=false;
}
System.out.println("verifySSLEnable="+verifySSLEnable);
}
if (c.indexOf("SLDEnabled") > -1) {
if("true".equals(c.split("=")[1])) {
SLDEnabled=true;
}else {
SLDEnabled=false;
}
System.out.println("SLDEnabled="+SLDEnabled);
}
if (c.indexOf("DeleteFiles") > -1) {
if("true".equals(c.split("=")[1])) {
deleteFiles=true;
}else {
deleteFiles=false;
}
System.out.println("deleteFiles="+deleteFiles);
}
}
System.out.println("----------读取配置文件完成----------");
ClientBuilderConfiguration conf = new ClientBuilderConfiguration();
// 设置是否支持将自定义域名作为Endpoint,默认支持。
conf.setSupportCname(supportCname);
// // 设置是否开启二级域名的访问方式,默认不开启。
conf.setSLDEnabled(SLDEnabled);
// 设置连接OSS所使用的协议(HTTP或HTTPS),默认为HTTP。
if ("HTTPS".equals(protocol)) {
conf.setProtocol(Protocol.HTTPS);
} else if ("HTTP".equals(protocol)) {
conf.setProtocol(Protocol.HTTP);
} else {
conf.setProtocol(Protocol.HTTP);
}
//
// // 设置是否开启HTTP重定向,默认开启。
// conf.setRedirectEnable(redirectEnable);
// 设置是否开启SSL证书校验,默认开启。
conf.setVerifySSLEnable(verifySSLEnable);
// 生成OSSClient,您可以指定一些参数,详见“SDK手册 > Java-SDK > 初始化”,
// 链接地址是:https://help.aliyun.com/document_detail/oss/sdk/java-sdk/init.html?spm=5176.docoss/sdk/java-sdk/get-start
OSS ossClient = new OSSClientBuilder().build(endpoint, accessKeyId, accessKeySecret, conf);
try {
// 判断Bucket是否存在。详细请参看“SDK手册 > Java-SDK > 管理Bucket”。
// 链接地址是:https://help.aliyun.com/document_detail/oss/sdk/java-sdk/manage_bucket.html?spm=5176.docoss/sdk/java-sdk/init
if (ossClient.doesBucketExist(bucketName)) {
System.out.println("您已经创建Bucket:" + bucketName + "。");
} else {
System.out.println("您的Bucket不存在,创建Bucket:" + bucketName + "。");
// 创建Bucket。详细请参看“SDK手册 > Java-SDK > 管理Bucket”。
// 链接地址是:https://help.aliyun.com/document_detail/oss/sdk/java-sdk/manage_bucket.html?spm=5176.docoss/sdk/java-sdk/init
ossClient.createBucket(bucketName);
}
// 查看Bucket信息。详细请参看“SDK手册 > Java-SDK > 管理Bucket”。
// 链接地址是:https://help.aliyun.com/document_detail/oss/sdk/java-sdk/manage_bucket.html?spm=5176.docoss/sdk/java-sdk/init
BucketInfo info = ossClient.getBucketInfo(bucketName);
System.out.println("Bucket " + bucketName + "的信息如下:");
System.out.println("\t数据中心:" + info.getBucket().getLocation());
System.out.println("\t创建时间:" + info.getBucket().getCreationDate());
System.out.println("\t用户标志:" + info.getBucket().getOwner());
// 把字符串存入OSS,Object的名称为firstKey。详细请参看“SDK手册 > Java-SDK > 上传文件”。
// 链接地址是:https://help.aliyun.com/document_detail/oss/sdk/java-sdk/upload_object.html?spm=5176.docoss/user_guide/upload_object
InputStream is = new ByteArrayInputStream("Hello OSS".getBytes());
ossClient.putObject(bucketName, firstKey, is);
System.out.println("Object:" + firstKey + "存入OSS成功。");
// 下载文件。详细请参看“SDK手册 > Java-SDK > 下载文件”。
// 链接地址是:https://help.aliyun.com/document_detail/oss/sdk/java-sdk/download_object.html?spm=5176.docoss/sdk/java-sdk/manage_object
OSSObject ossObject = ossClient.getObject(bucketName, firstKey);
InputStream inputStream = ossObject.getObjectContent();
StringBuilder objectContent = new StringBuilder();
BufferedReader reader = new BufferedReader(new InputStreamReader(inputStream));
while (true) {
String line = reader.readLine();
if (line == null)
break;
objectContent.append(line);
}
inputStream.close();
System.out.println("Object:" + firstKey + "的内容是:" + objectContent);
// 文件存储入OSS,Object的名称为fileKey。详细请参看“SDK手册 > Java-SDK > 上传文件”。
// 链接地址是:https://help.aliyun.com/document_detail/oss/sdk/java-sdk/upload_object.html?spm=5176.docoss/user_guide/upload_object
String fileKey = "README.md";
ossClient.putObject(bucketName, fileKey, new File("README.md"));
System.out.println("Object:" + fileKey + "存入OSS成功。");
// 查看Bucket中的Object。详细请参看“SDK手册 > Java-SDK > 管理文件”。
// 链接地址是:https://help.aliyun.com/document_detail/oss/sdk/java-sdk/manage_object.html?spm=5176.docoss/sdk/java-sdk/manage_bucket
ObjectListing objectListing = ossClient.listObjects(bucketName);
List<OSSObjectSummary> objectSummary = objectListing.getObjectSummaries();
System.out.println("您有以下Object:");
for (OSSObjectSummary object : objectSummary) {
System.out.println("\t" + object.getKey());
}
if(deleteFiles) {
ossClient.deleteObject(bucketName, firstKey);
System.out.println("删除Object:" + firstKey + "成功。");
ossClient.deleteObject(bucketName, fileKey);
System.out.println("删除Object:" + fileKey + "成功。");
}
} catch (OSSException oe) {
oe.printStackTrace();
} catch (ClientException ce) {
ce.printStackTrace();
} catch (Exception e) {
e.printStackTrace();
} finally {
ossClient.shutdown();
}
logger.info("Completed");
}
}
参考文章
https://blog.csdn.net/qq_383698639/article/details/100152585