The sixth-generation (6G) wireless networks will rely on ultra-dense multi-cell deployment to meet the high rate and connectivity demands. However, frequency reuse leads to severe inter-cell interference, particularly for cell-edge users, which limits the communication performance. To overcome this challenge, we investigate a beyond diagonal reconfigurable intelligent surface (BD-RIS) aided multi-cell multi-user downlink MIMO communication system, where a BD-RIS is deployed to enhance desired signals and suppress both intra-cell and inter-cell interference.We formulate the joint optimization problem of the transmit beamforming matrices at the BSs and the BD-RIS reflection matrix to maximize the weighted sum rate of all users, subject to the challenging unitary constraint of the BD-RIS reflection matrix and transmit power constraints at the BSs. To tackle this non-convex and difficult problem, we apply the weighted minimum mean squared error (WMMSE) method to transform the problem into an equivalent tractable form, and propose an efficient alternating optimization (AO) based algorithm to iteratively update the transmit beamforming and BD-RIS reflection using Lagrange duality theory and manifold optimization. Numerical results demonstrate the superiority of the proposed design over various benchmark schemes, and provide useful practical insights on the BD-RIS deployment strategy for multi-cell systems.

翻译：第六代（6G）无线网络将依赖超密集多小区部署以满足高速率与高连接需求。然而，频率复用导致严重的小区间干扰，尤其对小区边缘用户而言，这限制了通信性能。为克服这一挑战，我们研究了一种超对角可重构智能表面（BD-RIS）辅助的多小区多用户下行MIMO通信系统，其中部署BD-RIS以增强期望信号并抑制小区内与小区间干扰。我们构建了基站发射波束成形矩阵与BD-RIS反射矩阵的联合优化问题，以最大化所有用户的加权和速率，并受限于BD-RIS反射矩阵的严格酉约束以及基站的发射功率约束。为解决这一非凸且困难的优化问题，我们采用加权最小均方误差（WMMSE）方法将问题转化为等效可处理形式，并提出一种基于交替优化（AO）的高效算法，利用拉格朗日对偶理论与流形优化迭代更新发射波束成形与BD-RIS反射矩阵。数值结果证明了所提设计相对于多种基准方案的优越性，并为多小区系统中BD-RIS的部署策略提供了实用的实践见解。