word2vec1 (Mikolov et al., 2013a) toolkit can pre-train the
character embeddings on the Chinese corpus. The obtained embeddings are used to initialize the character lookup table instead of random initialization. Inspired by (Pei et al., 2014), we also can utilize bigram character embeddings which is simply
initialized as the average of embeddings of two
consecutive characters.