kafka 2.4 新版 Java Authorizer API解析

背景

    kafka 2.4 的release note 除了引入MirrorMaker 2.0 之外，还有一项改动是引入了新的java版的认证API接口。原因是这样的：
    kafka使用scla版的kafka.security.auth.Authorizer的trait来支持可插拔的授权接口。 但是KIP-50中已经同意用“client”模块中的org.apache.kafka.common包中的Java接口和Java ACL类来替换了它，但是之前一直都没有合并。于是社区就就添加了新版的java 授权接口， 这些Java接口能提供比Scala Trait更好的兼容性，能让开发者更容易开发自己的认证模块。

接口方法解析

    新版接口的工作流程是这样的：

1. 判断是否配置了"authorizer.class.name"，如果配置了就实例化一个Authorizer接口
2. 调用configure()和start()方法，初始化Authorizer接口对应的实现类里的元信息
3. broker启动socketServer接收请求，调用authorize()方法来处理每一个请求

    如果在配置中实现了Reconfigurable这个类的话还可以动态变更Authorizer 的实现类而不用重启broker，还有三点需要注意的是：

1. 所有的授权操作和Acl更新必须是线程安全的
2. Acl 更新操作是异步的，
3. 也可以在start()方法中定义其他的需要用的线程或者线程池，但是切记在close()方法里关掉他们

    除了上面提到的那些方法，还有下面三个方法
createAcls()和deleteAcls()：用来添加和删除Acl
acls()：用来获取符合查询条件的Acl

@InterfaceStability.Evolving
public interface Authorizer extends Configurable, Closeable {
    Map<Endpoint, ? extends CompletionStage<Void>> start(AuthorizerServerInfo serverInfo);
    List<AuthorizationResult> authorize(AuthorizableRequestContext requestContext, List<Action> actions);
    List<? extends CompletionStage<AclCreateResult>> createAcls(AuthorizableRequestContext requestContext, List<AclBinding> aclBindings);
    List<? extends CompletionStage<AclDeleteResult>> deleteAcls(AuthorizableRequestContext requestContext, List<AclBindingFilter> aclBindingFilters);
    Iterable<AclBinding> acls(AclBindingFilter filter);
}

实现类AclAuthorizer解析

kafka2.4中同样提供了一个"SimpleAclAuthorizer"的实现，只不过这次它去掉了Simple，就叫"AclAuthorizer"，功能和SimpleAclAuthorizer基本一样，只是实现方式变了，可以看下具体的方法：

start方法

    好像什么也没干

  override def start(serverInfo: AuthorizerServerInfo): util.Map[Endpoint, _ <: CompletionStage[Void]] = {
    serverInfo.endpoints.asScala.map { endpoint =>
      endpoint -> CompletableFuture.completedFuture[Void](null) }.toMap.asJava
  }

configure方法

    在configure方法里读取了kafka的配置，生成了zk的客户端和aclChangeListeners（用来感知zk节点变化，若有变化，则更新acl的cache），并通过loadCache()加载了zk中已经有 的权限。

  override def configure(javaConfigs: util.Map[String, _]): Unit = {
    val configs = javaConfigs.asScala
    val props = new java.util.Properties()
    configs.foreach { case (key, value) => props.put(key, value.toString) }

    superUsers = configs.get(AclAuthorizer.SuperUsersProp).collect {
      case str: String if str.nonEmpty => str.split(";").map(s => SecurityUtils.parseKafkaPrincipal(s.trim)).toSet
    }.getOrElse(Set.empty[KafkaPrincipal])

    shouldAllowEveryoneIfNoAclIsFound = configs.get(AclAuthorizer.AllowEveryoneIfNoAclIsFoundProp).exists(_.toString.toBoolean)

    // Use `KafkaConfig` in order to get the default ZK config values if not present in `javaConfigs`. Note that this
    // means that `KafkaConfig.zkConnect` must always be set by the user (even if `AclAuthorizer.ZkUrlProp` is also
    // set).
    val kafkaConfig = KafkaConfig.fromProps(props, doLog = false)
    val zkUrl = configs.get(AclAuthorizer.ZkUrlProp).map(_.toString).getOrElse(kafkaConfig.zkConnect)
    val zkConnectionTimeoutMs = configs.get(AclAuthorizer.ZkConnectionTimeOutProp).map(_.toString.toInt).getOrElse(kafkaConfig.zkConnectionTimeoutMs)
    val zkSessionTimeOutMs = configs.get(AclAuthorizer.ZkSessionTimeOutProp).map(_.toString.toInt).getOrElse(kafkaConfig.zkSessionTimeoutMs)
    val zkMaxInFlightRequests = configs.get(AclAuthorizer.ZkMaxInFlightRequests).map(_.toString.toInt).getOrElse(kafkaConfig.zkMaxInFlightRequests)

    val zkClientConfig = AclAuthorizer.zkClientConfigFromKafkaConfigAndMap(kafkaConfig, configs)
    val time = Time.SYSTEM
    zkClient = KafkaZkClient(zkUrl, kafkaConfig.zkEnableSecureAcls, zkSessionTimeOutMs, zkConnectionTimeoutMs,
      zkMaxInFlightRequests, time, "kafka.security", "AclAuthorizer", name=Some("ACL authorizer"),
      zkClientConfig = zkClientConfig)
    zkClient.createAclPaths()

    extendedAclSupport = kafkaConfig.interBrokerProtocolVersion >= KAFKA_2_0_IV1

    // Start change listeners first and then populate the cache so that there is no timing window
    // between loading cache and processing change notifications.
    startZkChangeListeners()
    loadCache()
  }

authorize方法

    authorize方法应该是直接调用了authorizeAction()，在这个方法里Authorizer从AuthorizableRequestContext中获取了请求的principal和host；从Action中获取了resource和operation。
鉴权流程是这样的：

1. 判断是否超级用户，是则直接返回AuthorizationResult.ALLOWED
2. 调用matchingAcls获取对应resource的所有权限，这里和之前的版本不一样的就是多了literal匹配和前缀匹配
3. 判断是否配置了allow.everyone.if.no.acl.found，如果该资源没有配置任何权限，且该配置打开，则返回AuthorizationResult.ALLOWED
4. 判断是否匹配上了DENY类的权限，若有，则直接返回AuthorizationResult.DENIED
5. 判断是否匹配上了ALLOW类的权限，若有，则直接返回AuthorizationResult.ALLOWED

  private def authorizeAction(requestContext: AuthorizableRequestContext, action: Action): AuthorizationResult = {
    val resource = action.resourcePattern
    if (resource.patternType != PatternType.LITERAL) {
      throw new IllegalArgumentException("Only literal resources are supported. Got: " + resource.patternType)
    }

    // ensure we compare identical classes
    val sessionPrincipal = requestContext.principal
    val principal = if (classOf[KafkaPrincipal] != sessionPrincipal.getClass)
      new KafkaPrincipal(sessionPrincipal.getPrincipalType, sessionPrincipal.getName)
    else
      sessionPrincipal

    val host = requestContext.clientAddress.getHostAddress
    val operation = action.operation

    def isEmptyAclAndAuthorized(acls: Set[AclEntry]): Boolean = {
      if (acls.isEmpty) {
        // No ACLs found for this resource, permission is determined by value of config allow.everyone.if.no.acl.found
        authorizerLogger.debug(s"No acl found for resource $resource, authorized = $shouldAllowEveryoneIfNoAclIsFound")
        shouldAllowEveryoneIfNoAclIsFound
      } else false
    }

    def denyAclExists(acls: Set[AclEntry]): Boolean = {
      // Check if there are any Deny ACLs which would forbid this operation.
      matchingAclExists(operation, resource, principal, host, DENY, acls)
    }

    def allowAclExists(acls: Set[AclEntry]): Boolean = {
      // Check if there are any Allow ACLs which would allow this operation.
      // Allowing read, write, delete, or alter implies allowing describe.
      // See #{org.apache.kafka.common.acl.AclOperation} for more details about ACL inheritance.
      val allowOps = operation match {
        case DESCRIBE => Set[AclOperation](DESCRIBE, READ, WRITE, DELETE, ALTER)
        case DESCRIBE_CONFIGS => Set[AclOperation](DESCRIBE_CONFIGS, ALTER_CONFIGS)
        case _ => Set[AclOperation](operation)
      }
      allowOps.exists(operation => matchingAclExists(operation, resource, principal, host, ALLOW, acls))
    }

    def aclsAllowAccess = {
      //we allow an operation if no acls are found and user has configured to allow all users
      //when no acls are found or if no deny acls are found and at least one allow acls matches.
      val acls = matchingAcls(resource.resourceType, resource.name)
      isEmptyAclAndAuthorized(acls) || (!denyAclExists(acls) && allowAclExists(acls))
    }

    // Evaluate if operation is allowed
    val authorized = isSuperUser(principal) || aclsAllowAccess

    logAuditMessage(requestContext, action, authorized)
    if (authorized) AuthorizationResult.ALLOWED else AuthorizationResult.DENIED
  }

createAcls

    先根据传入的List<AclBinding>获取一个aclsToCreate，获取写锁之后，逐条更新Acl，并将newAcls和CurrentAcls的两个list进行合并。返回给上层。