HADOOP 安全体系-Authentication 1

Authentication

验明身份的合法性，hadoop里需要验证加入集群的节点是合法的。要求链接的用户是合法的。

MIT Kerberos

MIT Kerberos 是一种中心化的认证工具，hadoop的认证体系默认采用 kerberos,简称 krb5   

krb5 分为服务端与客户端，通过 Relam 组成网络

krb5 理解为一个数据库，里面的对象是 principal 

principal 对应物可以是 service，可以是 host，也可以是 user，需要为其设置密码（这个密码也被称之为 Key）

krb5 安装与配置

1. 准备工作：

   ==为集群内所有机器的JDK安装JCE：==
   unzip -o -j -q jce_policy-8.zip -d /usr/jdk64/jdk1.8.0_40/jre/lib/security/
   

2. 服务端：

   yum install krb5-server krb5-libs krb5-workstation
   
   ==vi /etc/krb5.conf==
   
     [libdefaults]
     renew_lifetime = 7d
     forwardable = true
     default_realm = HADOOP.COM
     ticket_lifetime = 24h
     dns_lookup_realm = false
     dns_lookup_kdc = false
     default_ccache_name = /tmp/krb5cc_%{uid}
     #default_tgs_enctypes = aes des3-cbc-sha1 rc4 des-cbc-md5
     #default_tkt_enctypes = aes des3-cbc-sha1 rc4 des-cbc-md5
   
     [domain_realm]
     hadoop.com = HADOOP.COM
   
   [logging]
     default = FILE:/var/log/krb5kdc.log
     admin_server = FILE:/var/log/kadmind.log
     kdc = FILE:/var/log/krb5kdc.log
   
   [realms]
     HADOOP.COM = {
       admin_server = dev06.youedata
       kdc = dev06.youedata
     }
   
   ==vi /var/kerberos/krb5kdc/kdc.conf==
   
   default_realm = HADOOP.COM
   
   [kdcdefaults]
    kdc_ports = 88
    kdc_tcp_ports = 88
   
   [realms]
    HADOOP.COM = {
     master_key_type = aes256-cts
     acl_file = /var/kerberos/krb5kdc/kadm5.acl
     dict_file = /usr/share/dict/words
     admin_keytab = /var/kerberos/krb5kdc/kadm5.keytab
     supported_enctypes = aes256-cts:normal aes128-cts:normal des3-hmac-sha1:normal arcfour-hmac:normal camellia256-cts:normal camellia128-cts:normal des-hmac-sha1:normal des-cbc-md5:normal des-cbc-crc:normal
    }
   
    ==编辑权限控制表,使得管理员账户可以控制域的所有权限==
   
    vi /var/kerberos/krb5kdc/kadm5.acl
   
       */admin@HADOOP.COM    *
   

   1. 客户端：

      yum install krb5-workstation

   2. 拷贝服务端配置文件到客户端:

      scp /etc/krb5.conf ambari01:/etc/krb5.conf

   3. 创建数据库:

      kdb5_util create -r HADOOP.COM -s

   4. 启动服务:在服务端执行

      $chkconfig –level 35 krb5kdc on$ chkconfig –level 35 kadmin on
      $service krb5kdc start$ service kadmin start

   5. 创建 kerberos 管理员: 在服务端执行

      kadmin.local -q “addprinc root/admin”

Ambari 启用 krb5

2. Log in to Ambari Web and Browse to Admin > Kerberos.
3. Click “Enable Kerberos” to launch the wizard.
4. Select the type of KDC you are using and confirm you have met the prerequisites.
    ==使用Automated Setup==
5. Provide information about the KDC and admin account.
    a. In the KDC section, enter the following information:
        • In the KDC Host field, the IP address or FQDN for the KDC host. Optionally a port
number may be included.
        • In the Realm name field, the default realm to use when creating service principals.
        • (Optional) In the Domains field, provide a list of patterns to use to map hosts in the
cluster to the appropriate realm.  
        ==这里填/etc/krb5.conf 里面domain_realm 的内容就行 填完后点击测试按钮，如果不通过 看ambari后台的错误信息==

    b. In the Kadmin section, enter the following information：
        • In the Kadmin Host field, the IP address or FQDN for the KDC administrative host.
Optionally a port number may be included.
        • The Admin principal and password that will be used to create principals and
keytabs.
        • (Optional) If you have configured Ambari for encrypted passwords, the Save
Admin Credentials option will be enabled. With this option, you can have Ambari
store the KDC Admin credentials to use when making cluster changes. Refer to
Managing Admin Credentials for more information on this option.

==完成上述操作后 Ambari 会配置重启相关组件==

Java 远端访问 HDFS

==Ambari 为 hdfs 等组件的服务生成了密码文件 .keytab 存储在/etc/security/keytabs 下==

1、下载 /etc/krb5.conf  ,  /etc/security/keytabs/hdfs.headless.keytab  到本地目录
2、通过Ambari 控制台下载所有组件的配置文件
3、将core-site.xml  hdfs-site.xml 放入程序的构建路径上
4、代码如下：
System.setProperty("java.security.krb5.conf", "C:/Program Files/MIT/Kerberos/krb5.conf" );  //服务器端文件下载到本地的位置
Configuration conf = new Configuration();
UserGroupInformation.setConfiguration(conf);
try {
            //hdfs-ue_ambari@YOUEDATA.COM 是Ambari创建的 
            //通过Ambari控制台可以查看每个组件的默认用户，到服务器上切换到相应用户，如 su - hdfs
            //输入klist -e 可以看到当前使用的Principal

            UserGroupInformation.loginUserFromKeytab("hdfs-ue_ambari@YOUEDATA.COM", "C:/Program Files/MIT/Kerberos/keytabs/hdfs.headless.keytab");
        } catch (IOException e) {
            // TODO Auto-generated catch block
            e.printStackTrace();
        }

FileSystem fs;

try {
    fs = FileSystem.get(conf);
    fs.create(new Path("/kertest"));
    FileStatus[] fsStatus = fs.listStatus(new Path("/"));
    for(int i = 0; i < fsStatus.length; i++){
       System.out.println(fsStatus[i].getPath().toString());
    }
} catch (IOException e) {
    // TODO Auto-generated catch block
    e.printStackTrace();
}

输出：

hdfs://xx:8020/app-logs
hdfs://xx:8020/apps
hdfs://xx:8020/ats
hdfs://xx:8020/hdp
hdfs://xx:8020/kertest