利用 AES 对 log4j 日志文件加密

总览

本文简要介绍了 AES 算法加密的方式，以及如何利用 AES 对 log4j 输出的日志进行加密。

背景

在互联网时代下，JAVA 大多用来做后端开发，由于后端的程序大多都部署在自己的服务器上，客户接触不到程序的日志文件，因此，多数情况下，日志是没有加密的必要，log4j 本身也没有提供加密的方法。但有些客户端软件仍然是用 java 编写，客户端安装在客户的 PC上，我们想要了解软件的运行状态以及出错原因，就必须记下日志，这些日志可能包含有一些敏感的信息，我们不希望用户能直接看到，因此对日志加密是很有必要的。

AES 加密

既然要进行加密，那么首先得选择一个可靠的加密算法，网上搜索了下，大概有这三种：DES、AES、RSA。，其中 RSA 的解密是基于大数的因式分解，虽然安全性极高，但解密效率比其它两种低得多，不太适合。 DES是美国联邦政府采用过的一种加密方式，由于它的密钥只有56位，因此算法的理论安全强度是 $2^{56}$，但随着计算机的飞速发展，每秒能处理的密钥数越来越多，DES将不能够提供足够的安全性，因此美国国家标准技术研究所开始征集了 AES 用以取代 DES，研究所对 AES的要求是速度快(比三重 DES 速度快)、安全性高(至少与三重 DES一样安全)，最终 Rijndael 算法脱颖而出。因此对日志的加密解密选用 AES 比较适合。

JAVA 中的 AES

Java 早已提供标准的 AES 算法供大家使用，这里我提供一个简单的 AES 加密工具类(需要引入 apache 的 codec)

import java.nio.charset.Charset;
import java.security.GeneralSecurityException;

import javax.crypto.Cipher;
import javax.crypto.spec.IvParameterSpec;
import javax.crypto.spec.SecretKeySpec;

import org.apache.commons.codec.binary.Base64;

/**
 * @Title: AESUtil.java
 * @Description: AES 加密解密工具类
 * @author: weekdragon
 * @date: 2018年7月18日 下午4:58:22
 * @version V1.0
 */
public class AESUtil {

    private static final String KEY_ALGORITHM = "ThisIsASecretKey";
    private static final String DEFAULT_CIPHER_ALGORITHM = "AES/CBC/PKCS5Padding";// 默认的加密算法

    private static Cipher cipher = null;
    
    static {
        try {
            cipher = Cipher.getInstance(DEFAULT_CIPHER_ALGORITHM);
        } catch (Exception e) {}
    }
    
    /**
     * AES 加密操作
     *
     * @param content
     *            待加密内容
     * @param key
     *            加密密码
     * @return 返回Base64转码后的加密数据
     */
    public static String encrypt(String content, String key) throws GeneralSecurityException {
        if(cipher == null)return content;
        byte[] raw = key.getBytes(Charset.forName("UTF-8"));
        if (raw.length != 16) {
            throw new IllegalArgumentException("Invalid key size.");
        }
        SecretKeySpec skeySpec = new SecretKeySpec(raw, "AES");
        cipher.init(Cipher.ENCRYPT_MODE, skeySpec, new IvParameterSpec(new byte[16]));
        byte[] doFinal = cipher.doFinal(content.getBytes(Charset.forName("UTF-8")));
        return Base64.encodeBase64String(doFinal);
    }

    /**
     * AES 解密操作
     *
     * @param content
     * @param key
     * @return
     */
    public static String decrypt(String content, String key) throws GeneralSecurityException {
        byte[] encrypted = Base64.decodeBase64(content);
        byte[] raw = key.getBytes(Charset.forName("UTF-8"));
        if (raw.length != 16) {
            throw new IllegalArgumentException("Invalid key size.");
        }
        SecretKeySpec skeySpec = new SecretKeySpec(raw, "AES");
        Cipher cipher = Cipher.getInstance(DEFAULT_CIPHER_ALGORITHM);
        cipher.init(Cipher.DECRYPT_MODE, skeySpec, new IvParameterSpec(new byte[16]));
        byte[] original = cipher.doFinal(encrypted);
        return new String(original, Charset.forName("UTF-8"));
    }

}

对于 AES 加密，可以选择不同的密钥长度(16的整数倍)，还可以选择不同的补齐方法(加密数据不足16位时的补位策略)，这里只做了个最简单的实现。

对日志加密

log4j 本身没有提供日志加密的方法，但是用户可以自定义日志的 Appender，这个 Appender 就是负责日志输出的东西，目前我研究的加密方式有两种：
第一种是对整个输出流加密，自定义一个加密的输出流

import javax.crypto.*;
import javax.crypto.spec.IvParameterSpec;
import javax.crypto.spec.SecretKeySpec;
import java.io.*;
import java.security.*;


public class FlushableCipherOutputStream extends OutputStream
{
    private static int HEADER_LENGTH = 16;


    private SecretKeySpec key;
    private RandomAccessFile seekableFile;
    private boolean flushGoesStraightToDisk;
    private Cipher cipher;
    private boolean needToRestoreCipherState;

    /** the buffer holding one byte of incoming data */
    private byte[] ibuffer = new byte[1];

    /** the buffer holding data ready to be written out */
    private byte[] obuffer;



    /** Each time you call 'flush()', the data will be written to the operating system level, immediately available
     * for other processes to read. However this is not the same as writing to disk, which might save you some
     * data if there's a sudden loss of power to the computer. To protect against that, set 'flushGoesStraightToDisk=true'.
     * Most people set that to 'false'. */
    public FlushableCipherOutputStream(String fnm, SecretKeySpec _key, boolean append, boolean _flushGoesStraightToDisk)
            throws IOException
    {
        this(new File(fnm), _key, append,_flushGoesStraightToDisk);
    }

    public FlushableCipherOutputStream(File file, SecretKeySpec _key, boolean append, boolean _flushGoesStraightToDisk)
            throws IOException
    {
        super();

        if (! append)
            file.delete();
        seekableFile = new RandomAccessFile(file,"rw");
        flushGoesStraightToDisk = _flushGoesStraightToDisk;
        key = _key;

        try {
            cipher = Cipher.getInstance("AES/CBC/PKCS5Padding");

            byte[] iv = new byte[16];
            byte[] headerBytes = new byte[HEADER_LENGTH];
            long fileLen = seekableFile.length();
            if (fileLen % 16L != 0L) {
                throw new IllegalArgumentException("Invalid file length (not a multiple of block size)");
            } else if (fileLen == 0L) {
                // new file

                // You can write a 16 byte file header here, including some file format number to represent the
                // encryption format, in case you need to change the key or algorithm. E.g. "100" = v1.0.0
                headerBytes[0] = 100;
                seekableFile.write(headerBytes);

                // Now appending the first IV
                SecureRandom sr = new SecureRandom();
                sr.nextBytes(iv);
                seekableFile.write(iv);
                cipher.init(Cipher.ENCRYPT_MODE, key, new IvParameterSpec(iv));
            } else if (fileLen <= 16 + HEADER_LENGTH) {
                throw new IllegalArgumentException("Invalid file length (need 2 blocks for iv and data)");
            } else {
                // file length is at least 2 blocks
                needToRestoreCipherState = true;
            }
        } catch (InvalidKeyException e) {
            throw new IOException(e.getMessage());
        } catch (NoSuchAlgorithmException e) {
            throw new IOException(e.getMessage());
        } catch (NoSuchPaddingException e) {
            throw new IOException(e.getMessage());
        } catch (InvalidAlgorithmParameterException e) {
            throw new IOException(e.getMessage());
        }
    }


    /**
     * Writes one _byte_ to this output stream.
     */
    public void write(int b) throws IOException {
        if (needToRestoreCipherState)
            restoreStateOfCipher();
        ibuffer[0] = (byte) b;
        obuffer = cipher.update(ibuffer, 0, 1);
        if (obuffer != null) {
            seekableFile.write(obuffer);
            obuffer = null;
        }
    }

    /** Writes a byte array to this output stream. */
    public void write(byte data[]) throws IOException {
        write(data, 0, data.length);
    }

    /**
     * Writes <code>len</code> bytes from the specified byte array
     * starting at offset <code>off</code> to this output stream.
     *
     * @param      data     the data.
     * @param      off   the start offset in the data.
     * @param      len   the number of bytes to write.
     */
    public void write(byte data[], int off, int len) throws IOException
    {
        if (needToRestoreCipherState)
            restoreStateOfCipher();
        obuffer = cipher.update(data, off, len);
        if (obuffer != null) {
            seekableFile.write(obuffer);
            obuffer = null;
        }
    }


    /** The tricky stuff happens here. We finalise the cipher, write it out, but then rewind the
     * stream so that we can add more bytes without padding. */
    public void flush() throws IOException
    {
        try {
            if (needToRestoreCipherState)
                return; // It must have already been flushed.
            byte[] obuffer = cipher.doFinal();
            if (obuffer != null) {
                seekableFile.write(obuffer);
                if (flushGoesStraightToDisk)
                    seekableFile.getFD().sync();
                needToRestoreCipherState = true;
            }
        } catch (IllegalBlockSizeException e) {
            throw new IOException("Illegal block");
        } catch (BadPaddingException e) {
            throw new IOException("Bad padding");
        }
    }

    private void restoreStateOfCipher() throws IOException
    {
        try {
            // I wish there was a more direct way to snapshot a Cipher object, but it seems there's not.
            needToRestoreCipherState = false;
            byte[] iv = cipher.getIV(); // To help avoid garbage, re-use the old one if present.
            if (iv == null)
                iv = new byte[16];
            seekableFile.seek(seekableFile.length() - 32);
            seekableFile.read(iv);
            byte[] lastBlockEnc = new byte[16];
            seekableFile.read(lastBlockEnc);
            cipher.init(Cipher.DECRYPT_MODE, key, new IvParameterSpec(iv));
            byte[] lastBlock = cipher.doFinal(lastBlockEnc);
            seekableFile.seek(seekableFile.length() - 16);
            cipher.init(Cipher.ENCRYPT_MODE, key, new IvParameterSpec(iv));
            byte[] out = cipher.update(lastBlock);
            assert out == null || out.length == 0;
        } catch (Exception e) {
            throw new IOException("Unable to restore cipher state");
        }
    }

    public void close() throws IOException
    {
        flush();
        seekableFile.close();
    }
}

然后自定义一个 Appender ，重写 setFile 方法

public class AESRollingFileAppender extends RollingFileAppender {
	public final static byte[] keyBytes = new byte[] { 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15 };
	public final static SecretKeySpec key = new SecretKeySpec(keyBytes, "AES");
	public final static String cipherKey[] = new String[] { "AES/CBC/PKCS5Padding", "AES/CFB8/NoPadding" };
	public final static String Encoding = "UTF-8";
	
	private Writer fw;
	
	@Override
	public synchronized void setFile(String fileName, boolean append,
			boolean bufferedIO, int bufferSize) throws IOException {
		LogLog.debug("setFile called: " + fileName + ", " + append);
		
		// It does not make sense to have immediate flush and bufferedIO.
		if (bufferedIO) {
			setImmediateFlush(false);
		}

		reset();
		FlushableCipherOutputStream cstream = null;
		try {
			cstream = new FlushableCipherOutputStream(fileName, key, true, false);
		} catch (Exception ex) {
			LogLog.error("setFile error", ex);
			ex.printStackTrace();
		}

		setEncoding(Encoding);
		fw = createWriter(cstream);
		if (bufferedIO) {
			fw = new BufferedWriter(fw, bufferSize);
		}
		this.setQWForFiles(fw);
		this.fileName = fileName;
		this.fileAppend = append;
		this.bufferedIO = bufferedIO;
		this.bufferSize = bufferSize;
		writeHeader();
		LogLog.debug("setFile ended");
		
		if (append) {
			File f = new File(fileName);
			((CountingQuietWriter) qw).setCount(f.length());
		}
	}
}

这种方法是在 https://stackoverflow.com/questions/10283637/how-to-append-to-aes-encrypted-file 上找到的，实际测试中，发现客户端在某些极端情况下，日志输出的可能不完整，导致日志文件格式损坏，从损坏点开始起，日志不能解密，这是目前一个缺陷。但对于稳定运行的程序，并且日志并发量比较高，用这种方式加密可以节约时间。

第二种是对单条日志加密，由于会对访问到私有变量，因此直接拷贝 RollingFileAppender 类，重写 subAppend 方法即可

 /**
     * This method differentiates RollingFileAppender from its super class.
     * 
     * @since 0.9.0
     */
    protected void subAppend(LoggingEvent event) {
        this.qw.write(encrypt(this.layout.format(event)));
        if (layout.ignoresThrowable()) {
            String[] s = event.getThrowableStrRep();
            if (s != null) {
                int len = s.length;
                for (int i = 0; i < len; i++) {
                    this.qw.write(encrypt(s[i]));
                    this.qw.write(encrypt(Layout.LINE_SEP));
                }
            }
        }

        if (shouldFlush(event)) {
            this.qw.flush();
        }
        if (fileName != null && qw != null) {
            long size = ((CountingQuietWriter) qw).getCount();
            if (size >= maxFileSize && size >= nextRollover) {
                rollOver();
            }
        }
    }

    private String encrypt(String content) {
        try {
            return AESUtil.encrypt(content, AES_KEY) + "\n";
        } catch (Exception e) {
            e.printStackTrace();
            return content;
        }
    }

在 log4j 配置文件里替换掉默认的 Appender 即可

log4j.appender.AppenderName=package.AESRollingFileAppender

这种方式，日志是按行加密，每一条日志加密一行，相比前一种，加密时间有所增加，大概每 10w 条日志增加几秒钟的样子，对于客户端程序来说，日志记录得不会非常密集，并发量不会很高，完全可以满足要求，而且可以应对客户端在强退、断电等异常情况，即使日志记录不完整，也只会损坏单条，而不会影响全部。