数据和安全②HTTPS单向和双向认证

前言

示例 使用SpringBoot模拟服务端和客户端，使用okhttp作为httpClient工具。

如果对https相关理论不太熟悉和理解的可以看上一篇数据和安全①加解密理论概述

Okhttp https设置 HTTPS - OkHttp

证书在线格式转换 证书格式转换

私钥格式转换 KEY私钥格式转换工具-中国数字证书CHINASSL

证书工具openssl和keytool

keytools没办法签发证书，openssl能够进行签发和证书链的管理;

现有的证书大都采用X-509规范，所有不同格式证书导出后格式内容一致。

openssl默认采用pem格式，采用base64编码。

注意openssl有些命令在wins环境并不生效，需要在linux环境执行。

/index.html openssl官网

Apache Tomcat 9 (9.0.53) - SSL/TLS Configuration How-To tomcat keytool使用

配置前提

新建目录 newcerts 、private； crl、certs

新建文件index.txt、serial

serial文件写入01；

如图，修改openssl的配置文件openssl.cfg ；设置自己的CA目录，win配置文件路径 C:\OpenSSL-Win64\bin

#### 文件夹说明
dir：默认的ssl工作目录，可以修改默认目录，这里是安装完默认的

certs：存放已经签发的证书
newcerts：存放CA新生成的证书
private：存放私钥
crl：存放已经吊销的证书
index.txt：已签发证书的文本数据库文件
serial：序列号存储文件，序列号为16进制数存储供证书签发使用序列号做参考

.rand：私有随机文件
生成随机数命令：openssl rand -out xxx/.rand 1024
1024表示随机数长度
在生成证书的临时目录里创建默认配置目录文件命令，一键梭哈：
mkdir -p ./demoCA/certs; mkdir -p ./demoCA/crl; mkdir ./democA/newcerts; mkdir -p ./demoCA/private; touch ./demoCA/index.txt; touch ./demoCA/serial; echo 01 > ./demoCA/serial;

单向验证：

 服务端：服务端私钥和证书（ca证书或者服务端证书均可）即可（根据私钥可以生成公钥）

客户端：客户端是需要证书、ca证书或者服务端证书均可；

猜测：私钥和公钥随用随时生成，毕竟不需要验证客户端证书。

问题：注册信息要相同啊、除了Common Name

The countryName field needed to be the same in the

1、生成ca私钥和ca证书

#生成ca私钥  放入private文件夹
openssl genrsa -out ca.key 2048
#根据ca私钥生成ca证书、私钥路径要写对不然找不到
##-subj "/C=CN/ST=beijing/L=beijing/O=beijing/OU=bj/CN=*.test.com/EM=test"
## 域名重要  *.test.com
openssl req -new -x509 -days 3650 -key C:/ssl/ca/private/ca.key -out ca.crt

字段	说明	示例
Country Name	ISO国家代码（两位字符）	CN
State or Province Name	所在省份	beijing
Locality Name	所在城市	beijing
Organization Name	公司名称	bj
Organizational Unit Name	部门名称	IT Dept.
Common Name	申请证书的域名	aa.test.com
Email Address	不需要输入
A challenge password	不需要输入

2、生成服务端私钥和服务端证书

##创建服务器私钥
openssl genrsa -out server.key 2048
## 根据服务器私钥生成证书请求  信息要相同/a challenge password  不需要输入、后面也不需要输入
### 信息要相同 CN可以不同 aa.test.com
## -subj "/C=CN/ST=beijing/L=beijing/O=beijing/OU=bj/CN=aa.test.com/EM=test"
openssl req -new -days 365 -key server.key -out server.csr
## 使用ca证书签署服务端证书
openssl ca -in server.csr -out server.crt -cert ca.crt -keyfile    C:/ssl/ca/private/ca.key 

 生成ca签发证书以后、serial的01变为02；index.txt新加了签发证书的信息；newcert文件夹多了一个01.pem文件、和server.crt内容相同；

3、通过服务端私钥和服务端证书转jks

1、在线传证书和服务器私钥转jks 证书格式转换

2、使用openssl和keytool转jks

## 转换成pkcs12格式  wins执行卡死不出现结果，切换linux执行
## 输入一个导出密码，然后确认  123456
openssl pkcs12 -export -in server.crt -inkey server.key -out server.p12

## 转换成jks、输入dt和sourese keystroe密码  123456，同下第四步的配置
##keystore包括私钥和证书，可能是整个证书链，一个keystore中还可以包含多个证书、私钥。
###一个keystore会有一个总的密码。keystore中的每个key(私钥)还可以设一个单独的密码。

keytool -importkeystore -srckeystore server.p12 -destkeystore server.jks -srcstoretype pkcs12 -deststoretype jks

 4、服务端配置

SpringBoot服务端开启，别的不需要，网上的太假

server:
  port: 11001
  ssl:
    key-store: classpath:127.0.0.1.jks
    key-store-password: 123456
    key-store-type: jks
    key-password: 123456

5、客户端配置

1、postman测试

setting---General SSL certificate verification 关闭证书校验

2、浏览器直接访问

3、客户端配置ca证书或者服务端证书

Java代码实现

Java核心代码类

TrustManagerFactory，它主要是用来导入自签名证书，用来验证来自服务器的连接。
KeyManagerFactory，当开启双向验证时，用来导入客户端的密钥对。
SSLContext，SSL 上下文，使用上面的两个类进行初始化，就是个上下文环境。
SSLSocketFactory ：通过sslContext得到
SSLSocketFactory可以包含TrustManagerFactory和KeyManagerFactory

主要代码如下，详细代码放入github :https://github.com/zhouxiaohei/spring-ssl-demo

## 根据证书，生成TrustManagerFactory
private void createAndInitTrustManagerFactory() {

    if(StringUtils.isEmpty(caCertContent)){
        throw new IllegalArgumentException("服务端证书不可为空");
    }
    try {
        KeyStore caKeyStore = KeyStore.getInstance(KeyStore.getDefaultType());
        caKeyStore.load(null, null);
        Certificate certificate = readCertFile(caCertContent);
        caKeyStore.setCertificateEntry(CA_CRT_ALIAS, certificate);
        //初始化trustManagerFactory
        trustManagerFactory = TrustManagerFactory.getInstance(TrustManagerFactory.getDefaultAlgorithm());
        trustManagerFactory.init(caKeyStore);
    } catch (Exception e) {
        log.error("初始化TrustManagerFactory失败,证书内容{}", caCertContent, e);
        throw new RuntimeException(e);
    }
}
## 根据TrustManagerFactory初始化SSLContext

public static class SSLParams {
    public SSLSocketFactory sSLSocketFactory;
    public X509TrustManager trustManager;
}

public SSLParams getSSLParams(){
    try {
        SSLParams sslParams = new SSLParams();
        //得到ssl上下文
        SSLContext sslContext = SSLContext.getInstance("TLS");
        sslContext.init(null, trustManagerFactory.getTrustManagers(), null);
        sslParams.trustManager = getX509TrustManager();
        sslParams.sSLSocketFactory = sslContext.getSocketFactory();
        return sslParams;
    } catch (NoSuchAlgorithmException e) {
        e.printStackTrace();
    } catch (KeyManagementException e) {
        e.printStackTrace();
    }
    return null;
}
private X509TrustManager getX509TrustManager(){
    X509TrustManager x509TrustManager = (X509TrustManager) trustManagerFactory.getTrustManagers()[0];
    return x509TrustManager;
}
### 根据SSLParams 设置okhttpClient的Https初始化参数
OkHttpClient client = okHttpClient.newBuilder().
        sslSocketFactory(sslParams.sSLSocketFactory, sslParams.trustManager)
        .hostnameVerifier( (a,b) -> true).build();  // 校验hostname，返回true
Request request = new Request.Builder().url("https://aa.test.com:11001/demo/bootswagger/person/123").get().build();
Response response = client.newCall(request).execute();
System.out.println(response.body().string());

 单向认证之-信任所有证书

public class TrustAllCerts implements X509TrustManager {
    @Override
    public void checkClientTrusted(X509Certificate[] x509Certificates, String s) throws CertificateException {
    }

    @Override
    public void checkServerTrusted(X509Certificate[] x509Certificates, String s) throws CertificateException {
    }

    @Override
    public X509Certificate[] getAcceptedIssuers() {
        return new X509Certificate[0];
    }

    public static SSLSocketFactory createSSLSocketFactory() {
        SSLSocketFactory ssfFactory = null;
        try {
            SSLContext sc = SSLContext.getInstance("TLS");
            sc.init(null, new TrustManager[]{new TrustAllCerts()}, new SecureRandom());

            ssfFactory = sc.getSocketFactory();
        } catch (Exception e) {
        }
        return ssfFactory;
    }
}

## 调用
  public static void testOneWayAllHttps(String url){
        try {
            // 方式1  过期
//            OkHttpClient client = okHttpClient.newBuilder().
//                    sslSocketFactory(ClientCredentials.createSSLSocketFactory())
//                    .hostnameVerifier( (a,b) -> true).build();  // 校验hostname，返回true
            // 方式2
            SSLContext sslContext = SSLContext.getInstance("TLS");
            TrustAllCerts trustAllCerts = new TrustAllCerts();
            sslContext.init(null, new TrustManager[]{trustAllCerts}, new SecureRandom());
            // sslContext初始化 TrustManager 为什么必须填  unable to find valid certification path to requested target
            //sslContext.init(null, null, new SecureRandom());
            OkHttpClient client = okHttpClient.newBuilder().
                    sslSocketFactory(sslContext.getSocketFactory(),  trustAllCerts)
                    .hostnameVerifier( (a,b) -> true).build();  // 校验hostname，返回true

            Request request = new Request.Builder().url(url).get().build();
            Response response = client.newCall(request).execute();
            System.out.println(response.body().string());
        } catch (IOException e) {
            e.printStackTrace();
        } catch (NoSuchAlgorithmException e) {
            e.printStackTrace();
        } catch (KeyManagementException e) {
            e.printStackTrace();
        }
    }

双向认证:

前提：服务端证书和客户端证书都由同一个ca证书签发

服务端：ca证书、客户端证书、服务端证书、服务端私钥

客户端： ca或者服务端证书、客户端证书、客户端私钥

需要文件：ca.crt、server.key、server.crt、client.key、client.crt

注意事项：

tomcat应该只支持JKS，PKCS11或 PKCS12格式的密钥库，不支持pem

java.io.IOException: Failed to load keystore type [pem] with path 。。。due to [pem not found]

前面的1、2、3步骤不变，沿用ca证书和服务端jks

4、生成客户端私钥和证书-- 和服务端生成方式一样

##创建客户端私钥
openssl genrsa -out client.key 2048
#### 信息要相同 CN可以不同 bb.test.com
## -subj "/C=CN/ST=beijing/L=beijing/O=beijing/OU=bj/CN=bb.test.com/EM=test"
openssl req -new -days 365 -key client.key -out client.csr
## 使用ca证书签署客户端证书
openssl ca -in client.csr -out client.crt -cert ca.crt -keyfile C:/ssl/ca/private/ca.key 

 5、将客户端证书添加到服务端jks中

## 将客户端证书导入服务端jks
keytool -import -v -file client.cer -keystore server.jks
### 查看证书文件  cer和crt证书文件一模一样，顶多去掉前缀
### 查看jks的证书文件，如下图
keytool -list -rfc -keystore server.jks -storepass 123456
## 服务端信任客户端证书、客户端证书信任服务端不行，要将根证书导入秘钥库
## 要将ca证书也导入证书库
keytool -import -file ca.crt -alias ca -keystore server.jks -storepass 123456

如果没有添加根证书到jks秘钥库，报错如下：

SSLHandshakeException: Received fatal alert: bad_certificate

6、配置SpingBoot server

ssl:
  key-store: classpath:server.jks
  key-store-password: 123456
  key-store-type: jks
  key-password: 123456
  ## 开启双向验证
  trust-store: classpath:server.jks
  trust-store-password: 123456
  trust-store-type: jks
  client-auth: need

7、访问地址https://aa.test.com:11001/demo/bootswagger/person/12

双向验证，浏览器就不好玩了。

8、使用postman添加客户端证书和客户端私钥正常访问接口；

双向验证java代码实现

客户端JKS模式

客户端配置，根据客户端证书和私钥生成客户端jks证书；

通过上面的方式用在线工具或者使用openssl和keytool转jks均可；

代码解读、通过ca证书或者服务端证书生成trustManagerFactory

通过client.jks得到keyManagerFactory然后初始化SSLContext ，最后得到SSLParams 设置okhttpClient的Https初始化参数

@Slf4j
public class JksClientCredentials {

    private TrustManagerFactory trustManagerFactory;
    private KeyManagerFactory keyManagerFactory;

    private static final String CA_CRT_ALIAS = "caCert-cert";

    private String caCertContent;
    private FileInputStream clientJksStream;
    private String keyStorePass;

    public JksClientCredentials(String caCertContent, FileInputStream clientJksStream, String keyStorePass) {
        this.caCertContent = caCertContent;
        this.clientJksStream = clientJksStream;
        this.keyStorePass = keyStorePass;
    }

    public static class SSLParams {
        public SSLSocketFactory sSLSocketFactory;
        public X509TrustManager trustManager;
    }

    public JksClientCredentials.SSLParams getSSLParams(){
        try {
            // 1、通过证书得到TrustManagerFactory
            createAndInitTrustManagerFactory();
            //2、如果客户端证书和私钥存在，得到KeyManagerFactory
            createAndInitKeyManagerFactory();
            //3、Okhttp取消了，单参数方法，返回多参数用于Okhttp初始化
            JksClientCredentials.SSLParams sslParams = new JksClientCredentials.SSLParams();
            //得到ssl上下文
            SSLContext sslContext = SSLContext.getInstance("TLS");
            sslContext.init(keyManagerFactory == null ? null : keyManagerFactory.getKeyManagers(), trustManagerFactory.getTrustManagers(), new SecureRandom());
            sslParams.trustManager = getX509TrustManager();
            sslParams.sSLSocketFactory = sslContext.getSocketFactory();
            return sslParams;
        } catch (NoSuchAlgorithmException e) {
            e.printStackTrace();
        } catch (KeyManagementException e) {
            e.printStackTrace();
        }
        return null;
    }

    /**
     * @Author JackZhou
     * @Description  证书文件的标签说明不需要处理
     * @Date 2020/6/4 15:08
     **/
    private void createAndInitTrustManagerFactory() {

        if(StringUtils.isEmpty(caCertContent)){
            throw new IllegalArgumentException("服务端证书不可为空");
        }
        try {
            KeyStore caKeyStore = KeyStore.getInstance(KeyStore.getDefaultType());
            caKeyStore.load(null, null);
            Certificate certificate = getCertificate(caCertContent);
            caKeyStore.setCertificateEntry(CA_CRT_ALIAS, certificate);
            //初始化trustManagerFactory
            trustManagerFactory = TrustManagerFactory.getInstance(TrustManagerFactory.getDefaultAlgorithm());
            trustManagerFactory.init(caKeyStore);
        } catch (Exception e) {
            log.error("初始化TrustManagerFactory失败,证书内容{}", caCertContent, e);
            throw new RuntimeException(e);
        }
    }

    private void createAndInitKeyManagerFactory(){
        try{
            KeyStore clientKeyStore = KeyStore.getInstance(KeyStore.getDefaultType());
            clientKeyStore.load(clientJksStream, keyStorePass.toCharArray());
            keyManagerFactory = KeyManagerFactory.getInstance(KeyManagerFactory.getDefaultAlgorithm());
            keyManagerFactory.init(clientKeyStore, keyStorePass.toCharArray());
        }catch (Exception e){
            log.error("初始化KeyManagerFactory失败", e);
            throw new RuntimeException(e);
        }
    }

    private Certificate getCertificate(String content) throws CertificateException {
        CertificateFactory certificateFactory = CertificateFactory.getInstance("X.509");
        Certificate certificate = certificateFactory.generateCertificate(new ByteArrayInputStream(content.getBytes()));
        return  certificate;
    }

    private X509TrustManager getX509TrustManager(){
        X509TrustManager x509TrustManager = (X509TrustManager) trustManagerFactory.getTrustManagers()[0];
        return x509TrustManager;
    }
}

### 调用方法
// JKS双向验证方式成功
public static void testTwoWayHttpsJks(){
    try {
        String caCertContent = FileUtils.readFile("src/main/resources/httpsClient/ca.crt");
        String clientJksfilePath = "src/main/resources/httpsClient/client.jks";
        JksClientCredentials jksClientCredentials = new JksClientCredentials(caCertContent, new FileInputStream(clientJksfilePath), "123456");
        JksClientCredentials.SSLParams sslParams = jksClientCredentials.getSSLParams();
        OkHttpClient client = okHttpClient.newBuilder().
                sslSocketFactory(sslParams.sSLSocketFactory, sslParams.trustManager)
                .hostnameVerifier( (a,b) -> true).build();  // 校验hostname，返回true
        Request request = new Request.Builder().url("https://aa.test.com:11001/demo/bootswagger/person/123").get().build();
        Response response = client.newCall(request).execute();
        System.out.println(response.body().string());
    } catch (IOException e) {
        e.printStackTrace();
    }
}

客户端PEM模式

原生pem的RSA秘钥加载报错

java.security.InvalidKeyException: IOException : algid parse error, not a sequence

把私钥改成pkcs8 格式才行

openssl pkcs8 -topk8 -inform PEM -in private.key -outform pem -nocrypt -out pkcs8.pem  

 Pem代码实现如下

@Slf4j
public class PemClientCredentials {

    private static final String CA_CRT_ALIAS = "caCert-cert";
    private static final String CRT_ALIAS = "cert";

    private TrustManagerFactory trustManagerFactory;
    private KeyManagerFactory keyManagerFactory;

    private String caCertContent;
    private String privateKeyContent;
    private String clientCertContent;

    public PemClientCredentials(String caCertContent, String privateKeyContent, String clientCertContent) {
        this.caCertContent = caCertContent;
        this.privateKeyContent = privateKeyContent;
        this.clientCertContent = clientCertContent;
    }

    public static class SSLParams {
        public SSLSocketFactory sSLSocketFactory;
        public X509TrustManager trustManager;
    }

    public SSLParams getSSLParams(){
        try {
            // 1、通过证书得到TrustManagerFactory
            createAndInitTrustManagerFactory();
            //2、如果客户端证书和私钥存在，得到KeyManagerFactory
            createAndInitKeyManagerFactory();
            //3、Okhttp取消了，单参数方法，返回多参数用于Okhttp初始化
            SSLParams sslParams = new SSLParams();
            //得到ssl上下文
            SSLContext sslContext = SSLContext.getInstance("TLS");
            sslContext.init(keyManagerFactory == null ? null : keyManagerFactory.getKeyManagers(), trustManagerFactory.getTrustManagers(), null);
            sslParams.trustManager = getX509TrustManager();
            sslParams.sSLSocketFactory = sslContext.getSocketFactory();
            return sslParams;
        } catch (NoSuchAlgorithmException e) {
            e.printStackTrace();
        } catch (KeyManagementException e) {
            e.printStackTrace();
        }
        return null;
    }
    /**
      * @Author JackZhou
      * @Description  证书文件的标签说明不需要处理
      * @Date 2020/6/4 15:08
     **/
    private void createAndInitTrustManagerFactory() {

        if(StringUtils.isEmpty(caCertContent)){
            throw new IllegalArgumentException("服务端证书不可为空");
        }
        try {
            KeyStore caKeyStore = KeyStore.getInstance(KeyStore.getDefaultType());
            caKeyStore.load(null, null);
            Certificate certificate = readCertFile(caCertContent);
            caKeyStore.setCertificateEntry(CA_CRT_ALIAS, certificate);
            //初始化trustManagerFactory
            trustManagerFactory = TrustManagerFactory.getInstance(TrustManagerFactory.getDefaultAlgorithm());
            trustManagerFactory.init(caKeyStore);
        } catch (Exception e) {
            log.error("初始化TrustManagerFactory失败,证书内容{}", caCertContent, e);
            throw new RuntimeException(e);
        }
    }

    private X509TrustManager getX509TrustManager(){
        X509TrustManager x509TrustManager = (X509TrustManager) trustManagerFactory.getTrustManagers()[0];
        return x509TrustManager;
    }

    private void createAndInitKeyManagerFactory(){
        if(StringUtils.isEmpty(privateKeyContent) || StringUtils.isEmpty(clientCertContent)){
            return;
        }
        try{
            KeyStore clientKeyStore = KeyStore.getInstance(KeyStore.getDefaultType());
            clientKeyStore.load(null, null);
            char[] passwordCharArray = "".toCharArray();

            Certificate certificate = readCertFile(clientCertContent);
            clientKeyStore.setCertificateEntry(CRT_ALIAS, certificate);
            clientKeyStore.setKeyEntry("private-key", readPrivateKeyFile(privateKeyContent), passwordCharArray, new Certificate[]{certificate});

            keyManagerFactory = KeyManagerFactory.getInstance(KeyManagerFactory.getDefaultAlgorithm());
            keyManagerFactory.init(clientKeyStore, passwordCharArray);
        }catch (Exception e){
            log.error("初始化KeyManagerFactory失败", e);
            throw new RuntimeException(e);
        }
    }

    private static PrivateKey readPrivateKeyFile(String fileContent) throws Exception {
        RSAPrivateKey privateKey = null;
        if (fileContent != null && !fileContent.isEmpty()) {
            fileContent = fileContent.replace("-----BEGIN PRIVATE KEY-----\n", "")
                    .replace("-----BEGIN PRIVATE KEY-----\r\n", "")
                    .replace("-----END PRIVATE KEY-----", "")
                    .replaceAll("\\s", "");
            byte[] decoded = Base64.decodeBase64(fileContent);
            KeyFactory keyFactory = KeyFactory.getInstance("RSA");
            privateKey = (RSAPrivateKey) keyFactory.generatePrivate(new PKCS8EncodedKeySpec(decoded));
        }
        return privateKey;
    }


    private X509Certificate readCertFile(String fileContent) throws Exception {
        X509Certificate certificate = null;
        if (fileContent != null && !fileContent.trim().isEmpty()) {
            fileContent = fileContent.replace("-----BEGIN CERTIFICATE-----\n", "")
                    .replace("-----BEGIN CERTIFICATE-----\r\n", "")
                    .replace("-----END CERTIFICATE-----", "");
            byte[] decoded = Base64.decodeBase64(fileContent);
            CertificateFactory certFactory = CertificateFactory.getInstance("X.509");
            certificate = (X509Certificate) certFactory.generateCertificate(new ByteArrayInputStream(decoded));
        }
        return certificate;
    }

}

public static void testTwoWayHttps(){
    try {
        // ca和server 证书都行
        String caCertContent = FileUtils.readFile("src/main/resources/httpsClient/ca.crt");
        //String caCertContent = FileUtils.readFile("src/main/resources/httpsClient/server.crt");
        String clientCertContent = FileUtils.readFile("src/main/resources/httpsClient/client.crt");
        String clientKeyContent = FileUtils.readFile("src/main/resources/httpsClient/target_pkcs8_privatekey.key");
        //String clientKeyContent = FileUtils.readFile("src/main/resources/httpsClient/client.key");
        PemClientCredentials credentials = new PemClientCredentials(caCertContent, clientKeyContent, clientCertContent);
        PemClientCredentials.SSLParams sslParams = credentials.getSSLParams();
        OkHttpClient client = okHttpClient.newBuilder().
                sslSocketFactory(sslParams.sSLSocketFactory, sslParams.trustManager)
                .hostnameVerifier( (a,b) -> true).build();  // 校验hostname，返回true
        Request request = new Request.Builder().url("https://aa.test.com:11001/demo/bootswagger/person/123").get().build();
        Response response = client.newCall(request).execute();
        System.out.println(response.body().string());
    } catch (IOException e) {
        e.printStackTrace();
    }
}

项目已经提交到github ：https://github.com/zhouxiaohei/spring-ssl-demo