nifi集群_kerberos授权集群部署NIFI

接着上一遍文章单机NIFI使用kerberos授权登录和权限分配

192.168.42.6 nifi0 KDC192.168.42.3 nifi1 客户端192.168.42.4 nifi2 客户端192.168.42.5 nifi3 客户端

上面文章已经实现了nifi1服务器节点的kerberos授权和登录，下面开始安装集群NIFI

第一步：把nifi1的nifi-1.12.0-bin.tar.gz发送到nifi2和nifi3节点

scp /opt/nifi-1.12.0-bin.tar.gz root@nifi2:/opt/scp /opt/nifi-1.12.0-bin.tar.gz root@nifi3:/opt/

nifi0分发kerberos秘钥

scp /opt/nifi-service.keytab root@nifi1:/opt/scp /opt/nifi-service.keytab root@nifi2:/opt/scp /opt/nifi-service.keytab root@nifi3:/opt/

第二步：把nifi2和nifi3节点的nifi包解压

tar -zxvf /opt/nifi-1.12.0-bin.tar.gz

第三步：在nifi1节点上生成集群各节点的ssl证书

/opt/nifi-toolkit-1.12.0/bin/tls-toolkit.sh standalone -c ca.nifi.com -n 'nifi[1-3]' -o './target'

第四步：nifi1分发ssl证书

cp -rf  /opt/nifi-toolkit-1.12.0/bin/target/nifi1/* /opt/nifi-1.12.0/conf/scp /opt/nifi-toolkit-1.12.0/bin/target/nifi2/* root@nifi2:/opt/nifi-1.12.0/conf/scp /opt/nifi-toolkit-1.12.0/bin/target/nifi3/* root@nifi3:/opt/nifi-1.12.0/conf/

第三步：nifi1配置集群参数

修改nifi1、nifi2、nifi3自带zk相关配置/opt/nifi-1.12.0/conf/nifi.properties

sed -i "s#nifi.state.management.embedded.zookeeper.start=false#nifi.state.management.embedded.zookeeper.start=true#g" /opt/nifi-1.12.0/conf/nifi.propertiessed -i "s#nifi.cluster.is.node=false#nifi.cluster.is.node=true#g" /opt/nifi-1.12.0/conf/nifi.propertiessed -i "s#nifi.cluster.protocol.is.secure=false#nifi.cluster.protocol.is.secure=true#g" /opt/nifi-1.12.0/conf/nifi.propertiessed -i "s#nifi.zookeeper.connect.string=#nifi.zookeeper.connect.string=nifi1:2181,nifi2:2181,nifi3:2181#g" /opt/nifi-1.12.0/conf/nifi.properties

修改nifi1上/opt/nifi-1.12.0/conf/state-management.xml

<?xml version="1.0" encoding="UTF-8" standalone="yes"?><stateManagement>    <local-provider>        <id>local-providerid>        <class>org.apache.nifi.controller.state.providers.local.WriteAheadLocalStateProviderclass>        <property name="Directory">./state/localproperty>        <property name="Always Sync">falseproperty>        <property name="Partitions">16property>        <property name="Checkpoint Interval">2 minsproperty>    local-provider>    <cluster-provider>        <id>zk-providerid>        <class>org.apache.nifi.controller.state.providers.zookeeper.ZooKeeperStateProviderclass>        <property name="Connect String">nifi1:2181,nifi2:2181,nifi3:2181property>        <property name="Root Node">/nifiproperty>        <property name="Session Timeout">10 secondsproperty>        <property name="Access Control">Openproperty>    cluster-provider>stateManagement>

修改nifi1、nifi2、nifi3上/opt/nifi-1.12.0/conf/zookeeper.properties

sed -i "s#server.1=#server.1=nifi1:2888:3888;2181#g" /opt/nifi-1.12.0/conf/zookeeper.propertiessed -i '$a server.2=nifi2:2888:3888;2181' /opt/nifi-1.12.0/conf/zookeeper.propertiessed -i '$a server.3=nifi3:2888:3888;2181' /opt/nifi-1.12.0/conf/zookeeper.properties

initLimit=10autopurge.purgeInterval=24syncLimit=5tickTime=2000dataDir=./state/zookeeperautopurge.snapRetainCount=30server.1=nifi1:2888:3888;2181server.2=nifi2:2888:3888;2181server.3=nifi3:2888:3888;2181

所有接点执行于目录下/opt/nifi-1.12.0/conf/创建./state/zookeeper/文件夹

mkdir -p /opt/nifi-1.12.0/state/zookeeper

nifi1节点

echo 1 > /opt/nifi-1.12.0/state/zookeeper/myid

nifi2节点

echo 2 > /opt/nifi-1.12.0/state/zookeeper/myid

nifi3节点

echo 3 > /opt/nifi-1.12.0/state/zookeeper/myid

第四步：nifi1配置认证相关参数

修改nifi1、nifi2、nifi3/opt/nifi-1.12.0/conf/nifi.properties

krb_path=/etc/krb5.confsed -i "s#nifi.kerberos.krb5.file=#nifi.kerberos.krb5.file=$krb_path#g" /opt/nifi-1.12.0/conf/nifi.propertiessed -i "s#nifi.kerberos.service.principal=#nifi.kerberos.service.principal=nifi-service@EXAMPLE.COM#g" /opt/nifi-1.12.0/conf/nifi.propertieskeytab_path=/opt/nifi-service.keytabsed -i "s#nifi.kerberos.service.keytab.location=#nifi.kerberos.service.keytab.location=$keytab_path#g" /opt/nifi-1.12.0/conf/nifi.propertiessed -i "s#nifi.security.user.login.identity.provider=#nifi.security.user.login.identity.provider=kerberos-provider#g" /opt/nifi-1.12.0/conf/nifi.properties

修改nifi1上/opt/nifi-1.12.0/conf/login-identity-providers.xml

<?xml version="1.0" encoding="UTF-8" standalone="yes"?><loginIdentityProviders>    <provider>        <identifier>kerberos-provideridentifier>        <class>org.apache.nifi.kerberos.KerberosProviderclass>        <property name="Default Realm">EXAMPLE.COMproperty>        <property name="Authentication Expiration">12 hoursproperty>    provider>loginIdentityProviders>

修改nifi1上/opt/nifi-1.12.0/conf/authorizers.xml

<?xml version="1.0" encoding="UTF-8" standalone="yes"?><authorizers>    <userGroupProvider>        <identifier>file-user-group-provideridentifier>        <class>org.apache.nifi.authorization.FileUserGroupProviderclass>        <property name="Users File">./conf/users.xmlproperty>        <property name="Legacy Authorized Users File">property>           <property name="Initial User Identity 1">nifi-user@EXAMPLE.COMproperty>        <property name="Initial User Identity 2">CN=nifi1, OU=NIFIproperty>        <property name="Initial User Identity 3">CN=nifi2, OU=NIFIproperty>        <property name="Initial User Identity 4">CN=nifi3, OU=NIFIproperty>     userGroupProvider>    <accessPolicyProvider>        <identifier>file-access-policy-provideridentifier>        <class>org.apache.nifi.authorization.FileAccessPolicyProviderclass>        <property name="User Group Provider">file-user-group-providerproperty>        <property name="Authorizations File">./conf/authorizations.xmlproperty>        <property name="Initial Admin Identity">nifi-user@EXAMPLE.COMproperty>        <property name="Legacy Authorized Users File">property>        <property name="Node Identity 1">CN=nifi1, OU=NIFIproperty>        <property name="Node Identity 2">CN=nifi2, OU=NIFIproperty>        <property name="Node Identity 3">CN=nifi3, OU=NIFIproperty>         <property name="Node Group">property>    accessPolicyProvider>    <authorizer>        <identifier>managed-authorizeridentifier>        <class>org.apache.nifi.authorization.StandardManagedAuthorizerclass>        <property name="Access Policy Provider">file-access-policy-providerproperty>    authorizer>authorizers>

第四步：把nifi1的state-management.xml、zookeeper.properties、login-identity-providers.xml、authorizers.xml文件scp到nifi2和nifi3

scp /opt/nifi-1.12.0/conf/state-management.xml root@nifi2:/opt/nifi-1.12.0/conf/scp /opt/nifi-1.12.0/conf/zookeeper.properties root@nifi2:/opt/nifi-1.12.0/conf/scp /opt/nifi-1.12.0/conf/login-identity-providers.xml root@nifi2:/opt/nifi-1.12.0/conf/scp /opt/nifi-1.12.0/conf/authorizers.xml root@nifi2:/opt/nifi-1.12.0/conf/scp /opt/nifi-1.12.0/conf/state-management.xml root@nifi3:/opt/nifi-1.12.0/conf/scp /opt/nifi-1.12.0/conf/zookeeper.properties root@nifi3:/opt/nifi-1.12.0/conf/scp /opt/nifi-1.12.0/conf/login-identity-providers.xml root@nifi3:/opt/nifi-1.12.0/conf/scp /opt/nifi-1.12.0/conf/authorizers.xml root@nifi3:/opt/nifi-1.12.0/conf/

第五步：逐个启动那个nifi节点

/opt/nifi-1.12.0/bin/nifi.sh install/opt/nifi-1.12.0/bin/nifi.sh start

第六步：验证

附录一：配置不对可能遇到的错误