kubernetes Pull Image 鉴权过程

昨天遇到一个docker拉取镜像报错的问题，后来查询到是镜像的鉴权有问题，下面就和我跟着kubernetes 1.7.6 的代码看看整个过程。首先看拉取镜像的地方，
pkg/kubelet/kuberuntime/kuberuntime_image.go

func (m *kubeGenericRuntimeManager) PullImage(image kubecontainer.ImageSpec, pullSecrets []v1.Secret) (string, error) {
    img := image.Image
    repoToPull, _, _, err := parsers.ParseImageName(img)
    if err != nil {
        return "", err
    }

    keyring, err := credentialprovider.MakeDockerKeyring(pullSecrets, m.keyring)
    if err != nil {
        return "", err
    }

    imgSpec := &runtimeapi.ImageSpec{Image: img}
    creds, withCredentials := keyring.Lookup(repoToPull)
    if !withCredentials {
        glog.V(3).Infof("Pulling image %q without credentials", img)

        imageRef, err := m.imageService.PullImage(imgSpec, nil)
        if err != nil {
            glog.Errorf("Pull image %q failed: %v", img, err)
            return "", err
        }

        return imageRef, nil
    }

    var pullErrs []error
    for _, currentCreds := range creds {
        authConfig := credentialprovider.LazyProvide(currentCreds)
        auth := &runtimeapi.AuthConfig{
            Username:      authConfig.Username,
            Password:      authConfig.Password,
            Auth:          authConfig.Auth,
            ServerAddress: authConfig.ServerAddress,
            IdentityToken: authConfig.IdentityToken,
            RegistryToken: authConfig.RegistryToken,
        }

        imageRef, err := m.imageService.PullImage(imgSpec, auth)
        // If there was no error, return success
        if err == nil {
            return imageRef, nil
        }

        pullErrs = append(pullErrs, err)
    }

    return "", utilerrors.NewAggregate(pullErrs)
}

其中PullImage方法负责拉取镜像，里面用到了

creds, withCredentials := keyring.Lookup(repoToPull)

获取鉴权
这是个接口 pkg/credentialprovider/keyring.go

type DockerKeyring interface {
    Lookup(image string) ([]LazyAuthConfiguration, bool)
}

主要有四个实现的结构体
lazyDockerKeyring
BasicDockerKeyring
FakeKeyring
unionDockerKeyring
那么到底是谁在被使用，还是都被用到呢，那么就看上面的keyring的具体实现是在哪里！在pkg/kubelet/kuberuntime/kuberuntime_manager.go的初始化里面

kubeRuntimeManager := &kubeGenericRuntimeManager{
...
        keyring:             credentialprovider.NewDockerKeyring(),
    }

这个是创建的地方，具体看内部实现，

func NewDockerKeyring() DockerKeyring {
    keyring := &lazyDockerKeyring{
        Providers: make([]DockerConfigProvider, 0),
    }

    // TODO(mattmoor): iterating over the map is non-deterministic.  We should
    // introduce the notion of priorities for conflict resolution.
    for name, provider := range providers {
        if provider.Enabled() {
            glog.V(4).Infof("Registering credential provider: %v", name)
            keyring.Providers = append(keyring.Providers, provider)
        }
    }

    return keyring
}

原理如此，就是lazyDockerKeyring。而这个里面就初始化了一个DockerConfigProvider的providers，如果是在其他云的环境中会有aws、rancher等其他的providers，不过在我们的环境只有DockerConfigProvider。
那么我就分别看一下lazyDockerKeyring和它里面的DockerConfigProvider
先看pkg/credentialprovider/keyring.go 里面lazyDockerKeyring，上面调用lookup的函数

func (dk *lazyDockerKeyring) Lookup(image string) ([]LazyAuthConfiguration, bool) {
    keyring := &BasicDockerKeyring{}

    for _, p := range dk.Providers {
        keyring.Add(p.Provide())
    }

    return keyring.Lookup(image)
}

很简单，遍历Providers，这个里面就只有DockerConfigProvider
下面看看pkg/credentialprovider/provider.go DockerConfigProvider这个provider里面的Provide方法

func (d *defaultDockerConfigProvider) Provide() DockerConfig {
    // Read the standard Docker credentials from .dockercfg
    if cfg, err := ReadDockerConfigFile(); err == nil {
        return cfg
    } else if !os.IsNotExist(err) {
        glog.V(4).Infof("Unable to parse Docker config file: %v", err)
    }
    return DockerConfig{}
}

核心是ReadDockerConfigFile，看看具体实现pkg/credentialprovider/config.go

func ReadDockerConfigFile() (cfg DockerConfig, err error) {
    if cfg, err := ReadDockerConfigJSONFile(nil); err == nil {
        return cfg, nil
    }
    // Can't find latest config file so check for the old one
    return ReadDockercfgFile(nil)
}

这个里面之所以要读取两个文件是因为兼容老版的docker，新版的是config.json，老版的是.dockercfg。
两个函数的内容是相似的，我们看一个ReadDockerConfigJSONFile就行了。

func ReadDockerConfigJSONFile(searchPaths []string) (cfg DockerConfig, err error) {
    if len(searchPaths) == 0 {
        searchPaths = DefaultDockerConfigJSONPaths()
    }
    for _, configPath := range searchPaths {
        absDockerConfigFileLocation, err := filepath.Abs(filepath.Join(configPath, configJsonFileName))
        if err != nil {
            glog.Errorf("while trying to canonicalize %s: %v", configPath, err)
            continue
        }
        glog.V(4).Infof("looking for %s at %s", configJsonFileName, absDockerConfigFileLocation)
        cfg, err = ReadSpecificDockerConfigJsonFile(absDockerConfigFileLocation)
        if err != nil {
            if !os.IsNotExist(err) {
                glog.V(4).Infof("while trying to read %s: %v", absDockerConfigFileLocation, err)
            }
            continue
        }
        glog.V(4).Infof("found valid %s at %s", configJsonFileName, absDockerConfigFileLocation)
        return cfg, nil
    }
    return nil, fmt.Errorf("couldn't find valid %s after checking in %v", configJsonFileName, searchPaths)

}

由于没有传入路径，这个里面使用默认的路径DefaultDockerConfigJSONPaths，看看这个路径内容，

func DefaultDockerConfigJSONPaths() []string {
    return []string{GetPreferredDockercfgPath(), workingDirPath, homeJsonDirPath, rootJsonDirPath}
}

这个里面定义了四个路径分别是GetPreferredDockercfgPath(), workingDirPath, homeJsonDirPath, rootJsonDirPath。分别看一下吧
1.GetPreferredDockercfgPath
这个是在初始指定的cmd/kubelet/app/server.go

credentialprovider.SetPreferredDockercfgPath(kubeCfg.RootDirectory)

它就是kubelet的root目录。

2.workingDirPath
这个是work目录，就是工作目录，当然运行的目录

3.homeJsonDirPath

homeDirPath       = os.Getenv("HOME")
homeJsonDirPath   = filepath.Join(homeDirPath, ".docker")

这个是主目录，一般docker登录过后都会在这个目录下面生成鉴权文件。

4.rootJsonDirPath
这个是根目录

rootDirPath       = "/"
rootJsonDirPath   = filepath.Join(rootDirPath, ".docker")

好了。看完了它遍历的所以目录。剩下就是读取文件了，