Upload Images with API

Upload Images with API

参考文档

1. 官方文档
2. Harbor官网

时序图

上传流程

1. 文件解压和校验

   nginx.tar文件解压之后，读取元数据文件，对解压文件进行匹配校验，校验tar包格式是否正确

   文件夹内容如下：

   nginx
     - 0c57c3990987c1de66e09ecb3994eaf8996c0e27cbe7df5a268b121a85330857/
       - VERSION
       - json
       - layer.tar
     - 10540c3bf8d2de8ddc8e1a65745b58b388aa9359ac1185d7bd09bd4364593e8e/
       - VERSION
       - json
       - layer.tar
     - 1b8af5822f713034d3234d790d451ecd17bb14d4b14161d52f6b677807a946e8/
       - VERSION
       - json
       - layer.tar
     - b8fb848ce2879be8af7f92ca0f0c51a6820a1ebce8bc5dff28a1ee7def9178d5/
       - VERSION
       - json
       - layer.tar
     - e31b458cefedf63be64f469399172dc5a5fec8cade135faae081b15ebf62b827/
       - VERSION
       - json
       - layer.tar
     - e81f4966e9808f7043066e9d67a7bdb6e42cbc2a327ed73d84fddfada63d92dd/
       - VERSION
       - json
       - layer.tar
     - d868a2ccd9b148b984a40e49ab0b16e1434d5bca8f0bf8f2714ce7352c3d4555.json
     - manifest.json
     - repositories
   

   manifest.json：镜像的元数据文件，包含镜像名称和标签，配置文件名称和layer清单

   d868a2ccd9b148b984a40e49ab0b16e1434d5bca8f0bf8f2714ce7352c3d4555.json: 镜像配置文件

   []：镜像layer文件，包含版本文件，配置文件和tar包

2. 获取上传操作认证信息

   获取上传用的认证参数，包括token，cookie和csrf

   # 1. 获取cookie
   resp = curl -k -XGET {harbor}/v2/_catalog
   cookie = resp.header.Set-Cookie
   # 2. 获取token和csrf
   resp = curl -k -XGET {harbor}/service/token?service=harbor-registry&scope=repository:{projectName}:pull,push,delete
   token = resp.body.token
   csrf = resp.header.X-Harbor-Csrf-Token
   

3. 创建上传仓库

   为上传镜像创建仓库，若仓库已存在，返回409，反之返回201

   body = {
   "project_name": "", 仓库名称
   "count_limit": 0, 镜像数量，0表示无限制
   "storage_limit": -1, 存储限制，-1表示无限制
   "public": true  是否共享
   
   }
   curl -k -XPOST {harbor}/api/v2.0/projects -d body
   

4. 循环上传Layer

   上传操作可能出现502，503，504的错误，需要引入重试机制，避免上传失败，原文如下：

   # https://docs.docker.com/registry/spec/api/
   If an 502, 503 or 504 error is received, the client should assume that the download can proceed due to a temporary condition, honoring the appropriate retry mechanism. Other 5xx errors should be treated as terminal.
   

   从manifest.json获取镜像和layer列表，按照下述流程,依次分片上传每个镜像的每一个Layer：

   1. 校验Layer是否存在

      计算Layer的hash256值，从harbor获取当前layer是否存在，若接口返回200，代表harbor已存在当前Layer，无需再次上传

      curl -k -XHEAD {harbor}/v2/{prohectName/imageName}/blobs/sha256:{hash256}
      

   2. 开始上传任务，获取第一次分片上传的url

      resp = curl -k -XPOST {harbor}/v2/{prohectName/imageName}/blobs/uploads/
      uploadUrl = resp.header.location
      

   3. 分片上传Layer，从响应中获取下一次上传的url，直至上传结束

      resp = curl -k -XPATCH {uploadUrl}
      uploadUrl = resp.header.location
      

   4. 当分片上传结束时，发送请求(body长度为0)，通知harbor已上传完毕

      原文描述:

      Optionally, if all chunks have already been uploaded, a PUT request with a digest parameter and zero-length body may be sent to complete and validate the upload. Multiple “digest” parameters may be provided with different digests. The server may verify none or all of them but must notify the client if the content is rejected.
      

      # hash256通过计算上传Layer的字节获得
      curl -k -XPUT {uploadUrl}&digest=sha256:{hash256}
      

5. 上传配置文件

   从manifest.json获取config文件,开始上传任务

   1. 校验config文件是否已上传

      curl -k -XHEAD {harbor}/v2/{prohectName/imageName}/blobs/sha256:{hash256}
      

   2. 开始上传任务，获取上传URL

      resp = curl -k -XPOST {harbor}/v2/{prohectName/imageName}/blobs/uploads/
      uploadUrl = resp.header.location
      

   3. 全量上传config文件

      curl -k -XPUT {uploadUrl}&digest=sha256:{hash256}
      

6. 上传元数据文件

   最后上传元数据文件manifest.json

   1. 组装元数据对象

      ManifestV2 manifestV2 = new ManifestV2()
                      .setMediaType("application/vnd.docker.distribution.manifest.v2+json")
                      .setSchemaVersion(2);
              File configFile = new File(configPath);
              String hash256 = TarUtil.hash256(configFile);
              Config config = new Config()
                      .setMediaType("application/vnd.docker.container.image.v1+json")
                      .setDigest("sha256:" + hash256)
                      .setSize((int) configFile.length());
              manifestV2.setConfig(config);
              manifestV2.setLayers(layerList);
      

   2. 上传元数据文件

      curl -k -XPUT -H "Content-Type: application/vnd.docker.distribution.manifest.v2+json"  {harbor}/v2/{project}/{imageName}/manifests/{imageTag}