An End-to-End Block Autoencoder For Physical Layer Based On Neural Networks

Abstract

Deep learning has been applied in physical-layer communications systems in recent years and has demonstrated fascinating results that were comparable or even better than human expert systems. In this paper, a novel convolutional neural networks (CNNs)-based autoencoder communication system is proposed, which can work intelligently with arbitrary block length, can support different throughput and can operate under AWGN and Rayleigh fading channels as well as deviations from AWGN environments.  The proposed generalized communication system is comprised of carefully designed convolutional neural layers and, hence, inherits CNN’s breakthrough characteristics, such as generalization, feature learning, classification, and fast training convergence. On the other hand, the end-to-end architecture jointly performs the tasks of encoding/decoding and modulation/demodulation. Finally, we provide the numerous simulation results of the learned system in order to illustrate its generalization capability under various system conditions.

近年来，深度学习被应用于物理层通信系统中，并显示出与人类专家系统相当甚至更好的迷人效果。本文提出了一种新型的基于卷积神经网络(CNNs)的自动编码器通信系统，该系统可以在任意块长的情况下智能工作，可以支持不同的吞吐量，并且可以在AWGN和Rayleigh渐变信道以及偏离AWGN的环境下工作。 所提出的泛化通信系统由精心设计的卷积神经层组成，因此，继承了CNN的泛化、特征学习、分类和快速训练收敛等突破性特征。另一方面，端到端架构共同完成编码/解码和调制/解调的任务。最后，我们提供了学习系统的大量仿真结果，以说明其在各种系统条件下的泛化能力。

I. INTRODUCTION

In the past, conventional methods optimize the modules of communication system separately, such as encoder, modulator, to achieve the better transmission quality [1] [2]. Deep Learning has experienced fast development in the past decade and it also possesses great potential in wireless communication. There have been lots of model-driven applications based on Deep Learning [3], such as massive MIMO [4] and OFDM [5]. 

以往传统的方法是将通信系统的各个模块分别进行优化，如编码器、调制器，以达到较好的传输质量[1][2]。深度学习在过去十年中经历了快速发展，在无线通信领域也拥有巨大的潜力。基于深度学习[3]的模型驱动应用已经很多，如大规模MIMO[4]和OFDM[5]。

One important application of Deep Learning is to view communication system as an end-to-end autoencoder, in which the modules can be optimized jointly. The result in [6] has shown that autoencoders can readily match the performance of nearoptimal existing baseline modulation and coding schemes by learning the system during training. The transmitter maps a one-hot vector to particular constellation symbols for transmission. The signals distorted by channel are used to reconstruct the original vector. The authors of [7] have proved that an end-to-end structure need a differential channel model to optimize the transceiver. However, one-hot transmission scheme is limited because all information bits are only used to transmit one symbol, which decreases transmission efficiency seriously.

深度学习的一个重要应用是将通信系统看作是一个端到端的自动编码器，在这个系统中，各模块可以联合优化。在[6]中的结果已经表明，自动编码器通过在训练过程中学习系统，可以很容易地匹配现有基线调制和编码方案中接近最佳的性能。发射机将一个one-hot vector 映射到特定的星座符号上进行传输。用通道失真的信号来重建原始向量。作者[7]已经证明了端到端结构需要一个差分信道模型来优化收发器。但是，one-hot 传输方案由于所有信息位只用来传输一个符号，传输效率严重下降，因此受到限制。

Opposite to one-hot transmission scheme, block scheme is a transmission scheme which allows parallel inputs. It enables communication systems to transmit a stream of information bits instead of bits for one symbol [8]. In [9], block scheme [10] is introduced in autoencoder to deal with the transmission of batches of sequences. This structure supports arbitrary length of binary sequences as input, but its performance is not good enough for practical use.

与独热编码传输方案相对应，块方案是一种允许并行输入的传输方案。它使通信系统能够传输信息位流，而不是一个符号的位[8]。在[9]中，在自动编码器中引入了块方案[10]来处理批量序列的传输。这种结构支持任意长度的二进制序列作为输入，但其性能在实际使用中还不够好。

In this paper, we build up an end-to-end autoencoder with block transmission scheme. In order to improve its perfor mance, we also introduce memory mechanism into the neural networks. Our contributions are following:

• We propose a novel autoencoder structure based on neural networks. It introduces block scheme to deal with sequences in the form of blocks and allows arbitrary input length, which improves transmission efficiency. With the memory mechanism of recurrent neural networks (RNN), the autoencoder explores potential relationships between blocks for modulating. Through optimizing the transmitter and receiver jointly, the constellation diagram can be learned automatically for particular modulation mode. 
• We train and test the model under different channel models. The performance of the proposed model is better than other autoencoder-based communication systems under typical channels [9]. At the same time, the simulation result shows that lower code rate leads to a lower bit error rate (BER).

在本文中，我们建立了一个端到端的自动编码器，并采用了 块传输方案。为了提高其性能，我们还在神经系统中引入了记忆机制。网络。我们的贡献如下：

• 我们提出了一种基于神经网络的新型自动编码器结构。它引入块方案，以块的形式处理序列，并允许任意输入长度，提高了传输效率。借助于循环神经网络（RNN）的记忆机制，自动编码器探索区块之间的潜在关系进行调制。通过发射机和接收机的共同优化，可以自动学习特定调制模式的星座图。
• 我们在不同的通道模型下对模型进行训练和测试。在典型信道下，所提出的模型的性能优于其他基于自动编码器的通信系统[9]。同时，仿真结果表明，较低的码率会导致较低的误码率（BER）。

II. DEEP NEURAL NETWORK STRUCTURES

III. SYSTEM MODEL

We build up an end-to-end communication system using neural networks feeding with block data, which enables us to complete joint optimization of transceiver.

我们使用输入块数据的神经网络建立了端到端通信系统，这使我们能够完成收发器的联合优化。

A. Network Structure

The structure of block autoencoder is shown in Fig.2. It consists several parts as following.

• The input is a stream of bits. To solve the problem of block transmission, we set the number of blocks to M, and each block has S bits to be modulated, so the total length of input bits is S × M.
• In the first layer, we adopt a convolutional neural network to compress input bits into M blocks. The output is sent to several LSTM layers to produce the modulated M complex symbols. We combine the time distributed layer with LSTM layer in order to introduce some linear relationship between symbols. To satisfy power constraint, we normalize the output symbols at the end of the transmitter. The detailed parameters of our autoencoder are shown in table I.
• Since we add the operation of encoding into the network through adjusting output dimension of time-distributed layers, the number of complex symbols should be M' instead of M, which is dependent on the code rate we set.

块状自动编码器的结构如图2所示。它包括以下几个部分：

• 输入是一个比特流。为了解决块传输的问题，我们设置块数为M，每个块有S个位要调制，所以输入位的总长度为S×M。
• 在第一层，我们采用卷积神经网络将输入位压缩成M个块。输出送至多个LSTM层，得到已调制的M个复数符号。我们将时间分布层与LSTM层结合起来，以引入符号之间的一些线性关系。为了满足功率约束，我们对发射机末端的输出符号进行归一化处理。我们的自动编码器的详细参数如表I所示。
• 由于我们通过调整时间分布层的输出维度，在网络中加入了编码的操作，所以复数符号的数量应该是M'而不是M，这取决于我们设定的码率。

Following the encoding and modulating operation, the coded sequence z is transmitted over the communication channel by I and Q components of digital signal. In our model, the communication channel is non-trainable, which can be represented as h(z).

经过编码和调制操作后，编码序列z由数字信号的I和Q分量在通信信道上传输。在我们的模型中，通信通道是不可训练的，可以用h(z)表示。

The distorted signal is demodulated and decoded by the receiver. These layers reconstruct the input sequence. Each trainable layer of proposed autoencoder is followed by a batch normalization layer so that the training process will converge more quickly.

失真信号由接收机解调和解码。这些层重建输入序列。所提出的自动编码器的每个可训练层后面都有一个BN层，这样训练过程将更快地收敛。

B. Channel model

• First we consider AWGN channel models. AWGN channel is used to train and test our autoencoder. We add zero mean complex Gaussian noise to the transmitted symbol z. The variance of noise is calculated by given and block size S.
• In wireless communication, frequency selective fading is a radio propagation anomaly caused by partial cancellation of a radio signal by itself. The signal arrives at the receiver by several different paths. There exists intersymbol interference (ISI) that influences the signal to be received. For generalization, we also do experiments under frequency selective fading channels. Traditional methods add protective interval to avoid or decrease ISI. However, our autoencoder is an end-to-end system, so we simply increase the number of symbols instead of introducing extra artificial symbols into the end of transmitter. We train and test the models under two multipath channels. The channel models we use are shown in Fig.3. Channel A has two fading paths and the zero-delayed path is strong. Different from channel A, channel B has three fading paths, including a weak zero-delayed one.

• 首先我们考虑AWGN信道模型。AWGN信道用于训练和测试我们的自动编码器。我们在传输的符号z中加入零均值复高斯噪声，噪声的方差由给定的和块大小S计算。
• 在无线通信中，频率选择性衰落是由无线电信号自身的部分消除引起的一种无线电传播异常。信号通过几种不同的路径到达接收机。存在符号间干扰(ISI)，影响了信号的接收。为了便于推广，我们还做了频率选择性衰落信道下的实验。传统的方法是通过增加保护间隔来避免或减少ISI。然而，我们的自动编码器是一个端到端系统，所以我们只是增加了符号的数量，而不是在发射端引入额外的人工符号。我们在两个多径信道下对模型进行了训练和测试。我们使用的信道模型如图3所示。信道A有两条衰落路径，零延迟路径很强。与信道A不同的是，信道B有三条衰落路径，包括一条弱的零延迟路径。

IV. EXPERIMENTS

In order to obtain the performance of proposed autoencoder, we train and test the model in different scenarios. Bit error rate (BER) is a measure of the number of bit errors that occur in a given number of bit transmissions under all scenarios. For generalization, we simply select AWGN channel model. In fact, under the scenario of wireless communication, the channel would be more complex because signals arrive at the receiver through different paths which leads to ISI between symbols.

为了获得所提出的自动编码器的性能，我们在不同的场景下对模型进行了训练和测试。误码率（BER）是在所有情况下，在给定数量的比特传输中发生的比特错误数量的度量。为了推广，我们只选择AWGN信道模型。事实上，在无线通信的情况下，由于信号通过不同的路径到达接收机，导致符号之间的ISI，信道会变得更加复杂。

A. Settings

For simulation, we set the block size to 6 and block number to 400. So the autoencoder acts like a joint coding and modulating 64-QAM system. We compare the learned autoencoder with conventional coding and modulating method. The data sets are generated by random distributed {0, 1}. The number of samples is 40000 for training and 10000 for testing. We set batch size to 64 and use Adam optimizer with learning rate 0.001. We need to train the autoencoder under an SNR fixed channel. Through several experiments, we find the best training is 12dB.

仿真时，我们设置块大小为6，块数为400。所以自动编码器的作用就像一个联合编码和调制的64-QAM系统。我们将学习的自动编码器与传统的编码和调制方法进行比较。数据集由随机分布的{0，1}生成。训练样本数为40000，测试样本数为10000。我们设置批次大小为64，使用Adam优化器，学习率为0.001。我们需要在SNR固定的信道下训练自动编码器。通过多次实验，我们发现最佳训练为12dB。

B. AWGN Channel

The performance of the autoencoder under AWGN channel is shown in Fig.4. We also implement the autoencoder in [9] for comparison. We add redundant information to resist the influence of channel through increasing the number of symbols. The way that we adjust the code rate is to set different dimension to the time-distributed layer and the convolutional layer in the decoder. When code rate is set to 1, which means the sequence is uncoded, our autoencoder performs very closely to conventional MMSE decoding method. Clearly as shown in Fig.5, our block autoencoder gives better performance than autoencoder in [9]. When we decrease the code rate to 2/3, which means we add redundant information to the encoded sequence, the autoencoder’s performance is improved rationally. When code rate is set to 1/2, we compare it with Viterbi hard decoding method in 64QAM. We can find that our autoencoder performs far beyond Viterbi hard decoding method in low SNR situation. It requires lower power to reach the same BER as Viterbi’s method.

自动编码器在AWGN信道下的性能如图4所示。我们还实现了[9]中的自动编码器进行比较。我们通过增加符号数来增加冗余信息，以抵抗信道的影响。我们调整码率的方法是对解码器中的时间分布层和卷积层设置不同的维度。当码率设置为1，即序列未编码时，我们的自动编码器的性能与传统的MMSE解码方法非常接近。显然，如图5所示，我们的块状自动编码器的性能比[9]中的自动编码器更好。当我们将码率降低到2/3，即在编码序列中加入冗余信息时，自动编码器的性能得到了合理的提高。当码率设置为1/2时，我们将其与64QAM中的Viterbi硬解码方法进行比较。我们可以发现，在低信噪比情况下，我们的自动编码器的性能远远超过了Viterbi硬解码方法。它需要更低的功率才能达到与Viterbi方法相同的误码率。

We draw the constellation diagram of the trained autoencoder in Fig.5. We can see that the symbols plotted in complex plane are distributed in 64 clusters. In actual deployment, it is easy to transfer symbols through inphase and quadrature component according to the constellation diagram.

我们在图5中画出经过训练的自动编码器的星座图。 我们可以看到，绘制在复杂平面上的符号分布在64个簇中。 在实际部署中，很容易根据星座图通过同相和正交分量传输符号。

C. Fading Channel

The performance under two chosen channels is shown in Fig.6. We set the code rate to 1/2 and training to 20dB. Our autoencoder performs well in the noise ranging from -5dB to 10dB but faces an error floor when is more than 15dB. Compared with channel A, channel B’s BER is higher because it contains a weaker zero-delay path is weaker.

在两个选择的信道下的性能如图6所示。我们将编码速率设置为1/2，将训练为20dB。我们的自动编码器在-5dB到10dB的噪声范围内表现良好，但当大于15dB时会面临错误的下限，很难继续降低。与信道A相比，信道B的误码率更高，因为它包含的零延迟路径较弱。

To improve the autoencoder’s performance, we continue to decrease the code rate. As shown in Fig.7, its BER decreases when we amplify the number of symbols under the same channel B when we set training BER to 12dB. However, this will reduce the transmission efficiency so that the system is hard to be deployed on hardware. So trade-off strategy is important.

为了提高自动编码器的性能，我们继续降低码率。如图7所示，当我们将训练误码率设为12dB时，放大同一信道B下的符号数，其误码率会降低。但这样会降低传输效率，使系统难以在硬件上部署。所以权衡策略很重要。

V. CONCLUSION

In this paper, we propose a new communication structure combined with LSTMs and CNNs. The autoencoder performs better than other autoencoder-based communication systems under AWGN and multi-path fading channels. A regular constellation diagram can be learned with the limit of average power, which is easier to be deployed on hardware platform. Considering the wireless transmission scenario, the autoencoder needs extra symbols to resist the channel fading. The simulation result shows that the BER of proposed autoencoder can be decreased to an acceptable range through reducing code rate. We show that we can decrease the code rate to ensure a satisfying BER. Due to the property of CNNs and LSTMs, the autoencoder has no limit on the length of input sequence. Furthermore, we prove that the training and testing process do not need a particular channel model.

在本文中，我们提出了一种结合LSTMs和CNNs的新型通信结构。该自动编码器在AWGN和多路径衰落信道下的性能优于其他基于自动编码器的通信系统。在平均功率的限制下，可以学习到规则的星座图，更容易在硬件平台上部署。考虑到无线传输场景，自动编码器需要额外的符号来抵抗信道衰落。仿真结果表明，通过降低码率可以将自动编码器的误码率降低到可接受的范围。仿真结果表明，我们可以通过降低码率来保证满足误码率的要求。由于CNNs和LSTMs的特性，自动编码器对输入序列的长度没有限制。此外，我们证明训练和测试过程不需要特定的信道模型。

We may further discover other applications based on the block autoencoder in the following aspects.

• Our autoencoder is a kind of SISO system. The spectrum efficiency of SISO system is much lower than MIMO [2]. MIMO systems can enhance throughput without more bandwidth or transmit power expenditure. MIMO has become an essential element of wireless communication standards including IEEE 802.11n (Wi-Fi), IEEE 802.11ac (Wi-Fi), HSPA+ (3G), WiMAX (4G), and Long Term Evolution (4G LTE). Therefore, it is necessary for us to extend our system to a MIMO autoencoder.
• We mention that we can increase the number of symbols to reach to an ideal BER range. For proposed autoencoder, however, the code rate should be low to achieve the acceptable performance, which means we need to add more redundant information. So it is important to design a better structure based on block autoencoder, which shows more robustness to fading channels.

我们可以在以下几个方面进一步发现基于块状自动编码器的其他应用。

• 我们的自动编码器是一种SISO系统，SISO系统的频谱效率远低于MIMO[2]。SISO系统的频谱效率远低于MIMO[2]。MIMO系统可以在不增加带宽和发射功率支出的情况下提高吞吐量。MIMO已经成为IEEE 802.11n(Wi-Fi)、IEEE 802.11ac(Wi-Fi)、HSPA+(3G)、WiMAX(4G)和长期演进(4G LTE)等无线通信标准的重要元素。因此，我们有必要将系统扩展到MIMO自动编码器。
• 我们提到，我们可以增加符号的数量来达到理想的误码率范围。然而，对于所提出的自动编码器，为了达到可接受的性能，码率应该很低，这意味着我们需要添加更多的冗余信息。因此，设计一种更好的基于块状自动编码器的结构是很重要的，它对衰落信道的鲁棒性更强。