aws java sdk 访问ceph radosgw (hammer)

1. 安装maven[1]

- 下载

$ wget http://mirrors.cnnic.cn/apache/maven/maven-3/3.3.9/binaries/apache-maven-3.3.9-bin.tar.gz

- 解压

    $ mkdir ~/install/
    $ tar -zxf apache-maven-3.3.9-bin.tar.gz -C ~/install/ 

- 添加bin的路径到PATH
打开 ~/.bashrc 文件，添加如下内容

export PATH=/home/joseph/install/apache-maven-3.3.9/bin/:$PATH

- 修改maven的本地仓库路径

默认的路径是:

${user.home}/.m2/repository

可以在如下配置文件中修改

~/install/apache-maven-3.3.9/conf/settings.xml

- 添加中国区内的maven源

如果需要的所有java包都从国外网站下载会很慢。先添加ali的源。
还是修改 settings.xml 文件，添加mirrors相关的配置：

  ...
  <mirrors>                                                                                            
    ... // 省略                                                                                           
    <mirror>                                                                                           
      <id>alimaven</id>                                                                                                                                                 
      <mirrorOf>central</mirrorOf>                                                                     
      <name>aliyun maven</name>                                                                        
      <url>http://maven.aliyun.com/nexus/content/repositories/central/</url>                           
    </mirror>                                                                                          
  </mirrors> 
  ...

2. 下载编译aws-java-sample[2]

- clone aws-java-sample 仓库

$ git clone https://github.com/awslabs/aws-java-sample.git 

- 修改示例程序

    // src/main/java/com/amazonaws/samples/S3Sample.java
    /*
     * Copyright 2010-2013 Amazon.com, Inc. or its affiliates. All Rights Reserved.
     *
     * Licensed under the Apache License, Version 2.0 (the "License").
     * You may not use this file except in compliance with the License.
     * A copy of the License is located at
     *
     *  http://aws.amazon.com/apache2.0
     *
     * or in the "license" file accompanying this file. This file is distributed
     * on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either
     * express or implied. See the License for the specific language governing
     * permissions and limitations under the License.
     */
    package com.amazonaws.samples;
    
    import java.io.BufferedReader;
    import java.io.File;
    import java.io.FileOutputStream;
    import java.io.IOException;
    import java.io.InputStream;
    import java.io.InputStreamReader;
    import java.io.OutputStreamWriter;
    import java.io.Writer;
    import java.util.UUID;
    
    import com.amazonaws.AmazonClientException;
    import com.amazonaws.AmazonServiceException;
    import com.amazonaws.regions.Region;
    import com.amazonaws.regions.Regions;
    import com.amazonaws.services.s3.AmazonS3;
    import com.amazonaws.services.s3.AmazonS3Client;
    import com.amazonaws.services.s3.model.Bucket;
    import com.amazonaws.services.s3.model.GetObjectRequest;
    import com.amazonaws.services.s3.model.ListObjectsRequest;
    import com.amazonaws.services.s3.model.ObjectListing;
    import com.amazonaws.services.s3.model.PutObjectRequest;
    import com.amazonaws.services.s3.model.S3Object;
    import com.amazonaws.services.s3.model.S3ObjectSummary;
    
    / add by Yang Honggang
    import com.amazonaws.auth.BasicAWSCredentials; 
    import com.amazonaws.auth.AWSCredentials;
    import com.amazonaws.ClientConfiguration;
    import com.amazonaws.Protocol;
    import com.amazonaws.services.s3.model.ObjectMetadata;
    import java.util.List;
    
    /**
     * This sample demonstrates how to make basic requests to Amazon S3 using
     * the AWS SDK for Java.
     * <p>
     * <b>Prerequisites:</b> You must have a valid Amazon Web Services developer
     * account, and be signed up to use Amazon S3. For more information on
     * Amazon S3, see http://aws.amazon.com/s3.
     * <p>
     * <b>Important:</b> Be sure to fill in your AWS access credentials in
     * ~/.aws/credentials (C:\Users\USER_NAME\.aws\credentials for Windows
     * users) before you try to run this sample.
     */
    public class S3Sample {
    
        public static void main(String[] args) throws IOException {
    
         String accessKey = "7VEMLNMNYRVGDDFL2XCO";
         String secretKey = "MN4oaaTNNrxgoVVyYR1ptfssD1nZ0XYhDscpNnPf";
         // 如果默认的端口不是80（假如是90）
         // String endpoint = "10.41.25.35:90";
         String endpoint = "10.41.25.35";
        
         AWSCredentials credentials = new BasicAWSCredentials(accessKey, secretKey);
         ClientConfiguration clientConfig = new ClientConfiguration();
         clientConfig.setSignerOverride("S3SignerType");
         clientConfig.setProtocol(Protocol.HTTP);
         AmazonS3 conn = new AmazonS3Client(credentials, clientConfig);
         conn.setEndpoint(endpoint);
        
         String name = "from-java";
        
         // create buckets
         Bucket bucket = conn.createBucket(name);
         String bucketName = bucket.getName();
         // list buckets
         List<Bucket> buckets = conn.listBuckets();
         for(Bucket b : buckets){
                 System.out.println("bucketName:"+b.getName());
         }                                                                                                          
    
         // put Object                                                       
         String objName = "firstobj";
         conn.putObject(bucketName, objName, "my first object");                                                             
         System.out.println(">>> put object: firstobj");                              
                                                                                                                    
         // get Object                                                                                               
         conn.getObject(bucketName, objName);                                                                   
         ObjectMetadata se = conn.getObject(new GetObjectRequest(bucketName, objName), new File("/tmp/xx1"));
         System.out.println(">>> get object");                 
                                                                                                                    
         // list objects in the bucket                                                                                             
          ObjectListing objs = conn.listObjects(bucketName);                                                        
                                            
         System.out.println(">>> list all objects in bucket:");
         List<S3ObjectSummary> sums = objs.getObjectSummaries();
                         for(S3ObjectSummary s3 : sums){
                                 System.out.println("\t" + s3.getKey());
                         }
        
        
         System.out.println("all done!");
        }
    }

.. Note:: hammer版的rgw不支持awsv4签名，所以必须显式使用v2的签名。

          clientConfig.setSignerOverride("S3SignerType")

更多关于 aws sdk for java的 api的介绍，参见[4]。

- 第一次编译使用默认的 v1.9.6 的 aws java sdk

    // pom.xml
    <dependencies>                                                                                     
        <dependency>                                                                                   
            <groupId>com.amazonaws</groupId>                                                           
            <artifactId>aws-java-sdk</artifactId>                                                      
            <version>1.9.6</version>                                                                   
        </dependency>                                                                                  
    </dependencies>  

编译、运行命令:

mvn clean compile exec:java

效果：

    ...
    [INFO] --- exec-maven-plugin:1.2.1:java (default-cli) @ aws-java-sample ---
    bucketName:from-java
    >>> put object: firstobj
    >>> get object
    >>> list all objects in bucket:
            firstobj
    all done!
    [INFO] ------------------------------------------------------------------------
    [INFO] BUILD SUCCESS
    [INFO] ------------------------------------------------------------------------
    [INFO] Total time: 6.675 s
    [INFO] Finished at: 2016-12-07T13:44:54+08:00
    [INFO] Final Memory: 23M/220M
    [INFO] ------------------------------------------------------------------------

- 使用新版 1.11.64

将 pom.xml 中的 1.9.6 修改为 1.11.64 后，重新执行编译、运行命令。

参考

[1] http://maven.apache.org/install.html

[2] https://github.com/aws/aws-sdk-java

[3] https://github.com/aws/aws-sdk-java/issues/372

[4] http://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/overview-summary.html

[5] http://www.cnblogs.com/wuyouwulv/p/aws_java_sdk__jAWSClient.html