# Reinforcement Learning for Beam Pattern Design in Millimeter Wave and  Massive MIMO Systems

02/18/2021

∙

by   [Yu Zhang, et al.](/content/profile/yu-zhang/index.html)

Employing large antenna arrays is a key characteristic of millimeter wave
(mmWave) and terahertz communication systems. However, due to the adoption of
fully analog or hybrid analog/digital architectures, as well as non-ideal
hardware or arbitrary/unknown array geometries, the accurate channel state
information becomes hard to acquire. This impedes the design of
beamforming/combining [vectors](/content/machine-learning-glossary-and-terms/vector/index.html) that are crucial to fully exploit the potential
of large-scale antenna arrays in providing sufficient receive signal power. In
this paper, we develop a novel framework that leverages deep [reinforcement
learning](/content/machine-learning-glossary-and-terms/reinforcement-learning/index.html) (DRL) and a Wolpertinger-variant architecture and learns how to
iteratively optimize the beam pattern (shape) for serving one or a small set of
users relying only on the receive power measurements and without requiring any
explicit channel knowledge. The proposed model accounts for key hardware
constraints such as the phase-only, constant-modulus, and quantized-angle
constraints. Further, the proposed framework can efficiently optimize the beam
patterns for systems with non-ideal hardware and for arrays with unknown or
arbitrary array geometries. Simulation results show that the developed solution
is capable of finding near-optimal beam patterns based only on the receive
power measurements.

[READ FULL TEXT](http://arxiv.org/pdf/2102.09084v1)