docker命令之pull

pull

1 usage

NAME
       docker-pull - Pull an image or a repository from the registry
SYNOPSIS
       docker pull NAME[:TAG]
DESCRIPTION
       This  command  pulls down an image or a repository from the registry.  If there is more than one image for a repository
       (e.g.  fedora) then all images for that repository name are pulled down including any tags.

docker client:1.0

docker registry:0.9

2 代码分析

以docker pull example.com/ns/repo为例

2.1docker客户端

docker客户端执行的操作有两个动作：pull和login

首先做pull，如果registry返回的状态码是401，那么docker客户端就要执行login，如果login成功，就可以再次做pull。

func (cli *DockerCli) CmdPull(args ...string) error {
	cmd := cli.Subcmd("pull", "NAME[:TAG]", "Pull an image or a repository from the registry")
	allTags := cmd.Bool([]string{"a", "-all-tags"}, false, "Download all tagged images in the repository")
	if err := cmd.Parse(args); err != nil {
		return nil
	}

	if cmd.NArg() != 1 {
		cmd.Usage()
		return nil
	}
	var (
		v         = url.Values{}
		remote    = cmd.Arg(0)
		newRemote = remote
	)

解析出来的taglessRemote：example.com/ns/repo，tag：空。如何没有tag，使用默认的latest

	taglessRemote, tag := parsers.ParseRepositoryTag(remote)

<span style="font-family: Arial, Helvetica, sans-serif;">		</span><span style="font-family: Arial, Helvetica, sans-serif;">if tag == "" && !*allTags { </span>

<pre>		newRemote = taglessRemote + ":latest"
	}
	if tag != "" && *allTags {

return fmt.Errorf("tag can't be used with --all-tags/-a")}v.Set("fromImage", newRemote) //fromImage = newRemote:example.com/namespace/repo:latest

获取HOME下的.dockercfg，并解析之

<span style="white-space:pre">	</span>// Resolve the Repository name from fqn to hostname + name
<span style="white-space:pre">	</span>hostname, _, err := registry.ResolveRepositoryName(taglessRemote)//hostname:example.com
<span style="white-space:pre">	</span>if err != nil {return err}cli.LoadConfigFile()
<span style="white-space:pre">	</span>// Resolve the Auth config relevant for this server
<span style="white-space:pre">	</span>authConfig := cli.configFile.ResolveAuthConfig(hostname)  

pull动作的执行体。主要指发起POST方法，路由为/images/create的请求

<span style="white-space:pre">
</span>
<span style="white-space:pre">	</span>pull := func(authConfig registry.AuthConfig) error {
<span style="white-space:pre">		</span>buf, err := json.Marshal(authConfig)
<span style="white-space:pre">			</span>if err != nil {
<span style="white-space:pre">				</span>return err
<span style="white-space:pre">			</span>}

		registryAuthHeader := []string{
			base64.URLEncoding.EncodeToString(buf),
		}
		return cli.stream("POST", "/images/create?"+v.Encode(), nil, cli.out, map[string][]string{
			"X-Registry-Auth": registryAuthHeader,
		})
	} 

向docker httpserver发起pull请求，如果httpserver返回状态码401，docker客户端执行login，loging成功，再次发起pull请求

	if err := pull(authConfig); err != nil {

		if strings.Contains(err.Error(), "Status 401") {
			fmt.Fprintln(cli.out, "\nPlease login prior to pull:")

			if err := cli.CmdLogin(hostname); err != nil {
				return err
			}

			authConfig := cli.configFile.ResolveAuthConfig(hostname)
			return pull(authConfig)
		}
		return err
	}

	return nil
}

---------------------------------------------------------------------------------

分析：cli.stream("POST", "/images/create?"+v.Encode(), nil, cli.out, map[string][]string{"X-Registry-Auth": registryAuthHeader})
代码位置：api/client/utils.go,这个streamHelper主要的功能是构建http包内容，并发起请求，处理请求的响应

func (cli *DockerCli) stream(method, path string, in io.Reader, out io.Writer, headers map[string][]string) error {
	return cli.streamHelper(method, path, true, in, out, nil, headers)
}
在api/server/server.go中createRouter方法中可以找到POST /images/create对应的handler：postImagesCreate
// Creates an image from Pull or from Import
func postImagesCreate(engWriter *engine.Engine, version version.Version, w http.Response, r *http.Request, vars map[string]string) error {
</pre><p><span style="font-family:Arial,Helvetica,sans-serif;">解析http request请求内容</span></p>
<span style="font-family: Arial, Helvetica, sans-serif;">if err := parseForm(r); err != nil {</span>
		return err
	}

	var (
		image = r.Form.Get("fromImage")  //  example.com
		repo  = r.Form.Get("repo")  //  namespace/repo
		tag   = r.Form.Get("tag")   //  latest
		job   *engine.Job
	)
<span style="font-family: Arial, Helvetica, sans-serif;">//认证相关</span>
	authEncoded := r.Header.Get("X-Registry-Auth")
	authConfig := &registry.AuthConfig{}
	if authEncoded != "" {
		authJson := base64.NewDecoder(base64.URLEncoding, strings.NewReader(authEncoded))
		if err := json.NewDecoder(authJson).Decode(authConfig); err != nil {
			// for a pull it is not an error if no auth was given
			// to increase compatibility with the existing api it is defaulting to be empty
			authConfig = &registry.AuthConfig{}
		}
	}
<span style="white-space:pre">	</span>//如果image不为空，执行pull
	if image != "" { //pull
		if tag == "" {
			image, tag = parsers.ParseRepositoryTag(image)
		}
		metaHeaders := map[string][]string{}
		for k, v := range r.Header {
			if strings.HasPrefix(k, "X-Meta-") {
				metaHeaders[k] = v
			}
		}
		job = eng.Job("pull", image, tag)
		job.SetenvBool("parallel", version.GreaterThan("1.3"))
		job.SetenvJson("metaHeaders", metaHeaders)
		job.SetenvJson("authConfig", authConfig)
	} else { //import
		if tag == "" {
			repo, tag = parsers.ParseRepositoryTag(repo)
		}
		job = eng.Job("import", r.Form.Get("fromSrc"), repo, tag)
		job.Stdin.Add(r.Body)
	}

	if version.GreaterThan("1.0") {
		job.SetenvBool("json", true)
		streamJSON(job, w, true)
	} else {
		job.Stdout.Add(utils.NewWriteFlusher(w))
	}
<span style="white-space:pre">	</span>//执行pull操作
	if err := job.Run(); err != nil {
		if !job.Stdout.Used() {
			return err
		}
		sf := utils.NewStreamFormatter(version.GreaterThan("1.0"))
		w.Write(sf.FormatError(err))
	}

	return nil
}

job具体由registry/session.go中的GetRepositoryData执行
func (r *Session) GetRepositoryData(remote string) (*RepositoryData, error) {
	indexEp := r.indexEndpoint
	repositoryTarget := fmt.Sprintf("%srepositories/%s/images", indexEp, remote)

	log.Debugf("[registry] Calling GET %s", repositoryTarget)

	req, err := r.reqFactory.NewRequest("GET", repositoryTarget, nil)
	if err != nil {
		return nil, err
	}
	if r.authConfig != nil && len(r.authConfig.Username) > 0 {
		req.SetBasicAuth(r.authConfig.Username, r.authConfig.Password)
	}
	req.Header.Set("X-Docker-Token", "true")

	res, _, err := r.doRequest(req)
	if err != nil {
		return nil, err
	}
	defer res.Body.Close()
	if res.StatusCode == 401 {
		return nil, errLoginRequired
	}
	// TODO: Right now we're ignoring checksums in the response body.
	// In the future, we need to use them to check image validity.
	if res.StatusCode != 200 {
		return nil, utils.NewHTTPRequestError(fmt.Sprintf("HTTP code: %d", res.StatusCode), res)
	}

	var tokens []string
	if res.Header.Get("X-Docker-Token") != "" {
		tokens = res.Header["X-Docker-Token"]
	}

	var endpoints []string
	if res.Header.Get("X-Docker-Endpoints") != "" {
		endpoints, err = buildEndpointsList(res.Header["X-Docker-Endpoints"], indexEp)
		if err != nil {
			return nil, err
		}
	} else {
		// Assume the endpoint is on the same host
		u, err := url.Parse(indexEp)
		if err != nil {
			return nil, err
		}
		endpoints = append(endpoints, fmt.Sprintf("%s://%s/v1/", u.Scheme, req.URL.Host))
	}

	checksumsJSON, err := ioutil.ReadAll(res.Body)
	if err != nil {
		return nil, err
	}
	remoteChecksums := []*ImgData{}
	if err := json.Unmarshal(checksumsJSON, &remoteChecksums); err != nil {
		return nil, err
	}

	// Forge a better object from the retrieved data
	imgsData := make(map[string]*ImgData)
	for _, elem := range remoteChecksums {
		imgsData[elem.ID] = elem
	}

	return &RepositoryData{
		ImgList:   imgsData,
		Endpoints: endpoints,
		Tokens:    tokens,
	}, nil
}
2.2 registry端
 
 

  
  docker-registry端使用的是flask框架，docker客户端使用GET方法，路由/v1/repositories/<path:repository>/images向docker registry发起pull请求
 
 
 
 

  
  处理的handler在docker_registry/index.py下的get_repository_images方法
 
 
 
 

  
  入口参数是namespace和repository，本例中namespace=ns，repository=repo，返回给docker客户端的pull请求分三步完成
 
 
 
 

 
 

  
  1、获取镜像的存储路径
 
 
path = store.index_images_path(namespace, repository)
2、在存储路径下找到镜像的真实数据
 
 

  
  
data = store.get_content(path)
3、返回请求数据
 
 
headers = generate_headers(namespace, repository, "read")
return toolket.response(data, 200, headers, True)

3.pull流程图谱
1.在docker client要pull image之前，也login一样，也是要做ping操作
2.registry返回给client 200 OK
3.client端携带需要pull的镜像信息和授权信息：basic authorization，向registry请求镜像


 
 

  
  4.registry返回给client请求镜像的所有分层id
 
 
 
 

  
  
  
  

 
 
 
 

  
  5.client向registry请求此镜像的tag信息
 
 
 
 

  
  

  
  
 
 
 
 

  
  
6.registry返回给client此镜像的tag信息和此tag对应的lay id
 
 
 
 

  
   
 
 
 
 

  
  
 
 
 
 

  
  7.client获取祖先lay信息
 
 
 
 

  
  
 
 
 
 

  
   
 
 
 
 

  
  8.registry返回此祖先对应的所有lay信息
 
 
 
 

  
  

  
  
 
 
 
 

  
  如果lay id在本地已经存在的话，client就不会download此lay了，如果不存在，那么先请求此lay的json信息，然后请求lay tar包