92.给HttpFS服务配置SSL

最新推荐文章于 2024-04-28 21:45:00 发布

大勇任卷舒

最新推荐文章于 2024-04-28 21:45:00 发布

阅读量277

点赞数

分类专栏： Hadoop 文章标签： ssl centos java

@大勇若怯任卷舒

本文链接：https://blog.csdn.net/m0_47454596/article/details/122467148

版权

Hadoop 专栏收录该内容

145 篇文章 3 订阅

订阅专栏

92.1 演示环境介绍

CentOS版本：7.2
集群未启用Kerberos

92.2 操作演示

1.KeyStore文件生成

在部署了HttpFS服务的节点上，执行如下命令生成KeyStore文件
- 需要输入密码和CN
  - CN必须为HttpFS服务所在节点的hostname
- 执行成功后证书文件默认生成在/var/lib/hadoop-httpfs目录下的 .keystore文件

[root@cdh01 ~]# sudo -u httpfs /usr/java/jdk1.7.0_67-cloudera/bin/keytool -genkey -alias tomcat -keyalg RSA
Enter keystore password:  （输入密码）
Re-enter new password: （再次输入密码）
What is your first and last name?
  [Unknown]:  cdh01.macro.com
What is the name of your organizational unit?
  [Unknown]:  fayson
What is the name of your organization?
  [Unknown]:  fayson
What is the name of your City or Locality?
  [Unknown]:  fayson
What is the name of your State or Province?
  [Unknown]:  fayson
What is the two-letter country code for this unit?
  [Unknown]:  fason
Is CN=fayson, OU=cloudera, O=cloudera, L=shanghai, ST=shanghai, C=shanghai correct?
  [no]:  yes
Enter key password for <tomcat>
        (RETURN if same as keystore password):  （输入密码）
Re-enter new password: （再次输入密码）
[root@cdh01 ~]#

完成证书文件的生成后，接下来通过CM界面为HttpFS服务配置SSL

2.SSL的配置

使用管理员用户登录CM的WEB界面
- 进入HDFS服务配置页面，范围选择【HttpFS】，类别选择【安全性】
配置启用SSL和Keystore文件
- 保存配置，回到CM主页根据提示重启相应服务

3.验证

浏览器验证是否能正常访问HDFS文件列表，查看HDFS根目录下所有文件
- 请求成功，使用的是https
通过curl查看是否正常访问

[root@cdh01 ~]# curl -k "https://192.168.10.4:14000/webhdfs/v1/?op=liststatus&user.name=hdfs"

4.Java客户端访问

将HttpFS服务器上的.keystore证书下载至本地命名为httpfs.keystore
使用Maven创建一个java工程，pom.xml文件中增加如下依赖

<dependency>
    <groupId>org.apache.httpcomponents</groupId>
    <artifactId>httpclient</artifactId>
    <version>4.5.4</version>
</dependency>
<dependency>
    <groupId>org.apache.httpcomponents</groupId>
    <artifactId>httpmime</artifactId>
    <version>4.5.4</version>
</dependency>

编写访问HttpFS的示例代码

package com.cloudera.hdfs.nonekerberos;
import javax.net.ssl.SSLContext;
import org.apache.http.HttpEntity;
import org.apache.http.client.methods.CloseableHttpResponse;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.conn.ssl.SSLConnectionSocketFactory;
import org.apache.http.conn.ssl.TrustSelfSignedStrategy;
import org.apache.http.impl.client.CloseableHttpClient;
import org.apache.http.impl.client.HttpClients;
import org.apache.http.ssl.SSLContexts;
import org.apache.http.util.EntityUtils;
import java.io.BufferedReader;
import java.io.File;
import java.io.InputStreamReader;
/**
 * package: com.cloudera.hdfs.nonekerberos
 * describe: Java使用HttpClient访问启用SSL的HttpFS服务
 * creat_user: Fayson
 * email: htechinfo@163.com
 * creat_date: 2018/1/13
 * creat_time: 下午9:56
 * 公众号：碧茂大数据
 */
public class HttpFSSSLDemo {
    private static String HTTPFS_HOST = "cdh01.macro.com";
    public static void main(String[] args) {
        try{
            SSLContext sslcontext = SSLContexts.custom()
                    .loadTrustMaterial(new File("/Volumes/Transcend/keytab/ssl/httpfs-server.keystore"), "123456".toCharArray(), new TrustSelfSignedStrategy()).build();
            // Allow TLSv1 protocol only
            SSLConnectionSocketFactory sslsf = new SSLConnectionSocketFactory(
                    sslcontext,
                    new String[] { "TLSv1" },
                    null,
                    SSLConnectionSocketFactory.getDefaultHostnameVerifier());
            CloseableHttpClient httpclient = HttpClients.custom().setSSLSocketFactory(sslsf).build();
            System.out.println("===========Case No.1  List resources==========");
            String getRequest =  "https://" + HTTPFS_HOST + ":14000/webhdfs/v1/?op=liststatus&user.name=hdfs";
            HttpGet httpget = new HttpGet(getRequest);
            System.out.println("executing request " + httpget.getRequestLine());
            CloseableHttpResponse response = httpclient.execute(httpget);
            System.out.println("----------------------------------------");
            System.out.println(response.getStatusLine());
            HttpEntity entity = response.getEntity();
            BufferedReader br =
                    new BufferedReader(new InputStreamReader((entity.getContent())));
            String output;
            while ((output = br.readLine()) != null) {
                System.out.println(output);
            }
            EntityUtils.consume(entity);
        } catch (Exception e) {
            e.printStackTrace();
        }
    }
}

5.常见问题

问题如下：

javax.net.ssl.SSLPeerUnverifiedException: Certificate for <114.119.11.142> doesn't match any of the subject alternative names: []
   at org.apache.http.conn.ssl.SSLConnectionSocketFactory.verifyHostname(SSLConnectionSocketFactory.java:467)
   at org.apache.http.conn.ssl.SSLConnectionSocketFactory.createLayeredSocket(SSLConnectionSocketFactory.java:397)
   at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:355)
   at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142)
   at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:373)
   at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:381)
   at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:237)
   at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185)
   at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89)
   at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111)
   at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
   at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)
   at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:108)
   at com.cloudera.hdfs.nonekerberos.HttpFSSSLDemo.main(HttpFSSSLDemo.java:50)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   at java.lang.reflect.Method.invoke(Method.java:483)
   at com.intellij.rt.execution.application.AppMain.main(AppMain.java:147)