Channel estimation and signal detection with model-driven deep learning for massive multiuser MIMO-OFDM systems

Abstract

The emerging fifth generation (5G) and beyond wireless systems raise requirements on improved coverage, lower latency, higher data rates, and energy efficiency. These improvements can be provided by massive Multiple-Input Multiple-Output (MIMO), which is one of the essential technologies of 5G. However, due to a large number of antennas and radio frequency chains, the receiver design for massive MIMO systems becomes more challenging. Moreover, non-linear effects in communication systems like hardware impairment are hard to be modeled. Since conventional solutions struggle to address these challenges, deep learning (DL) is considered as a promising approach for the sixth generation cellular systems. The application of DL in the physical layer is still in a nascent stage. Most prior DL-based receivers are purely data-driven and aim mainly for performance improvement. By contrast, model-driven DL-based massive MIMO receivers can potentially achieve lower complexity, better interpretability and robustness by introducing expert knowledge, but this is not yet well-investigated in the literature. This PhD thesis focuses on designing lowcomplexity channel estimation and signal detection schemes with the application of model-driven DL techniques for massive MIMO orthogonal frequency-division multiplexing (OFDM) systems.

Firstly, a novel model-aided DL-based OFDM receiver, which integrates a convolutional neural network (CNN)-based channel estimator and a signal detection neural network (SD-NN), is proposed. By exploiting channel correlations in the time and frequency domain, the proposed CNN-based scheme can efficiently denoise and refine the image-like low-resolution channel with time-variant and frequency-selective effects. Then, the explicit channel information from the CNN is processed by a modelbased equalizer, and the output is used as a proper initialization for SD-NN. Moreover, several data preprocessing and model tuning strategies are developed to improve detection performance. As a result, unlike the data-driven solutions viewing the whole receiver as a black box, the proposed model-aided DL architecture achieves lower complexity and faster convergence with only a small number of pilots.

Secondly, considering the complexity problem of generic NN architectures in massive MIMO-OFDM systems, an efficient deep unfolding (DU)-based detection network is developed. Based on the domain knowledge of MIMO detection, the alternating direction method of multipliers (ADMM)-based network skeleton is first derived. To get a specialized architecture with improved model flexibility for DU-based MIMO-OFDM detection, the over-relaxation parameter and an additional step size are added to the network skeleton.With the help of DU techniques, the sets of trainable parameters can be optimized explicitly for different subcarriers to enhance the detection performance under realistic channels with severe spatial and frequency correlations. Furthermore, a differentiable projection function is designed to enable learning-based parameter optimization. Compared to existing baselines in the literature, the proposed approach can provide a better performance-complexity trade-off, especially for the cases of high user load and real-world correlated channels.

Thirdly, for high-loaded massive MIMO systems with large numbers of users, a frequency-orthogonal pilot scheme is designed to save time resources used for pilot transmission. By a special subtractive residual layer, the proposed denoising CNN learns the denoising mapping to eliminate channel noise in the delay domain instead of learning the labeled channel matrices directly. To further improve estimation performance, a residual CNN is proposed to exploit spatial-frequency correlations of channels and refine the output of the denoising CNN. With the help of a customized fast Fourier transform layer, these two CNNs can be jointly trained across different domains, resulting in an end-to-end channel estimation network. This dual CNN-based estimator is shown to achieve state-of-the-art performance and fast convergence with lower complexity than baseline approaches.

Finally, the research is extended to massive MIMO-OFDM systems with low-precision analog-to-digital converters (ADCs). To accurately detect the received signals with severe non-linear distortions under such complex systems, a novel model-driven detection network is proposed. Since well-established architectures like ADMM can not handle errors of coarse quantization, a flexible non-linear estimator is first derived to replace the general x-update of ADMM. Specifically, the scalar step size is upgraded to a learnable vector used as the multiplicative gradient correction, and an additive gradient correction is also added. Correspondingly, a specialized network skeleton with multiple trainable parameters and an adaptive proximal operator is designed. By fusing the model-based architecture and data-driven techniques, the proposed scheme shows superior detection performance and robustness in coarsely quantized massive multiuser MIMO-OFDM systems.