Imperial College London

Department of Electrical and Electronic Engineering

Rate-Splitting Multiple Access for

Non-Terrestrial Communication and
Sensing Networks

Longfei Yin

Supervised by Prof. Bruno Clerckx

Submitted in part fulfilment of the requirements for the degree of

Doctor of Philosophy of Imperial College London and
the Diploma of Imperial College London, 2023
 Declaration of Originality

I hereby declare that the content of this thesis is my own research work. The main
parts of this thesis have been published in related conferences and journals. Where
other sources of information have been used, they have been acknowledged.

1
2
 Copyright Declaration

The copyright of this thesis rests with the author. Unless otherwise indicated, its
contents are licensed under a Creative Commons Attribution-Non Commercial 4.0
International Licence (CC BY-NC). Under this licence, you may copy and redistribute
the material in any medium or format. You may also create and distribute modified
versions of the work. This is on the condition that: you credit the author and do
not use it, or any derivative works, for a commercial purpose. When reusing or
sharing this work, ensure you make the licence terms clear to others by naming the
licence and linking to the licence text. Where a work has been adapted, you should
indicate that the work has been changed and describe those changes. Please seek
permission from the copyright holder for uses of this work that are not included in
this licence or permitted under UK Copyright Law.

3
4
 Abstract

Rate-splitting multiple access (RSMA) has emerged as a powerful and flexible

non-orthogonal transmission, multiple access (MA) and interference management
scheme for future wireless networks. This thesis is concerned with the application of
RSMA to non-terrestrial communication and sensing networks. Various scenarios
and algorithms are presented and evaluated.

First, we investigate a novel multigroup/multibeam multicast beamforming strategy

based on RSMA in both terrestrial multigroup multicast and multibeam satellite
systems with imperfect channel state information at the transmitter (CSIT). The
max-min fairness (MMF)-degree of freedom (DoF) of RSMA is derived and shown
to provide gains compared with the conventional strategy. The MMF beamforming
optimization problem is formulated and solved using the weighted minimum mean
square error (WMMSE) algorithm. Physical layer design and link-level simulations
are also investigated. RSMA is demonstrated to be very promising for multigroup
multicast and multibeam satellite systems taking into account CSIT uncertainty
and practical challenges in multibeam satellite systems.

Next, we extend the scope of research from multibeam satellite systems to satellite-
terrestrial integrated networks (STINs). Two RSMA-based STIN schemes are
investigated, namely the coordinated scheme relying on CSI sharing and the co-
operative scheme relying on CSI and data sharing. Joint beamforming algorithms
are proposed based on the successive convex approximation (SCA) approach to
optimize the beamforming to achieve MMF amongst all users. The effectiveness and
robustness of the proposed RSMA schemes for STINs are demonstrated.

Finally, we consider RSMA for a multi-antenna integrated sensing and communi-

cations (ISAC) system, which simultaneously serves multiple communication users
and estimates the parameters of a moving target. Simulation results demonstrate
that RSMA is beneficial to both terrestrial and multibeam satellite ISAC systems by
evaluating the trade-off between communication MMF rate and sensing Cramér-Rao
bound (CRB).

5
6
 Acknowledgements

First and foremost, I would like to express my sincere gratitude and appreciation
to my supervisor, Prof. Bruno Clerckx for his constant support, guidance and
continuous encouragement during my PhD study. I am profoundly grateful for his
insightful comments and inspiring suggestions which set me on the right path from
the very beginning and have made this research journey wonderful and fruitful. His
great enthusiasm, rigorous attitude, professional skills, wide knowledge and kind
personality will always inspire me in the future.

I am also truly thankful to my colleagues and friends in the Communications and

Signal Processing group at the Department of Electrical and Electronic Engineering
for the precious memories and wonderful friendships they have provided me during
all this time and have enriched my life.

Finally, I am profoundly grateful to my parents for their boundless love that gives
me the confidence to tackle all difficulties. They have always provided me with
constant support and encouragement and supported my decisions. An infinite thank
you for your endless love and endless support.

7
8
 Abbreviations

4G fourth generation

5G fifth generation

6G sixth generation

ADMM alternating direction method of multipliers

AMC adaptive modulation and coding

AWGN additive white Gaussian noise

B5G beyond fifth generation

BC broadcast channel

BS base station

CCP convex-concave procedure

CDMA code division multiple access

CRB Cramér-Rao bound

CSIR channel state information at the receiver

CSIT channel state information at the transmitter

CUs cellular users

DoA direction of arrival

DoD direction of departure

DoF degree of freedom

EE energy efficiency

eURLLC extremely ultra reliable and low-latency communication

9
FDMA frequency division multiple access

FeMBB further-enhanced mobile broadband

FIM Fisher information matrix

GEO geostationary orbit

GW gateway

IoT Internet-of-Things

ISAC integrated sensing and communications

LEO low Earth orbit

LLS link-level simulation

LMI linear matrix inequality

LOS line-of-sight

MA multiple access

MFR minimum fairness rate

MIMO multiple-input multiple-output

MISO multiple-input single-output

MMF max-min fairness

MMSE minimum mean square error

MSE mean square error

MU multiuser

MU-LP multiuser linear precoding

NOMA non-orthogonal multiple access

NTN non-terrestrial network

10
OFDMA orthogonal frequency division multiple access

OMA orthogonal multiple access

PHY physical layer

QAM quadrature amplitude modulation

QoS quality-of-service

QPSK quadrature-phase-shift-keying

RAN radio access network

RCS radar cross-section

RSMA rate-splitting multiple access

RF radio frequency

RMSE root mean square error

SAA sample average approximation

SAGIN space-air-ground integrated network

SC superposition coding

SCA successive convex approximation

SDMA space-division multiple access

SDP semi-definite programming

SDR semi-definite relaxation

SIC successive interference cancellation

SINR signal-to-interference-noise ratio

SISO single-input single-output

SNR signal-to-noise ratio

11
SOCP second-order cone program

STIN satellite-terrestrial integrated network

SUs satellite users

SWIPT simultaneous wireless information and power transfer

TDMA time division multiple access

UAV unmanned aerial vehicle

ULA uniform linear array

umMTC ultra massive machine type communication

UPA uniform planar array

V2X vehicle-to-everything

VoD video on demand

WIPT wireless information and power transfer

WMMSE weighted minimum mean square error

WSR weighted sum-rate

ZFBF zero-forcing beamforming

12
Contents

Declaration of Originality 1

Copyright Declaration 3

Abstract 5

Acknowledgements 7

Abbreviations 9

1 Introduction 23

1.1 Toward Rate-Splitting Multiple Access . . . . . . . . . . . . . . . . 24

1.2 Motivation and Organization . . . . . . . . . . . . . . . . . . . . . . 27

1.3 List of Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . 30

1.4 Publications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31

1.5 Notation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32

13
14 CONTENTS

2 Background 34

2.1 Fundamentals of Downlink RSMA . . . . . . . . . . . . . . . . . . . 34

2.2 Multibeam Satellite Systems . . . . . . . . . . . . . . . . . . . . . . 38

2.3 Satellite-Terrestrial Integrated Networks . . . . . . . . . . . . . . . 40

2.4 Integrated Sensing and Communications . . . . . . . . . . . . . . . 43

3 RSMA for Multigroup Multicast and Multibeam Satellite Systems 45

3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46

3.2 System Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48

3.2.1 Transceiver Scheme . . . . . . . . . . . . . . . . . . . . . . . 49

3.2.2 CSIT Uncertainty and Scaling . . . . . . . . . . . . . . . . . 52

3.3 Max-Min Fair DoF Analysis . . . . . . . . . . . . . . . . . . . . . . 52

3.3.1 Max-Min Fair DoF of SDMA . . . . . . . . . . . . . . . . . 53

3.3.2 Max-Min Fair DoF of RSMA . . . . . . . . . . . . . . . . . 59

3.4 Max-Min Fair Problem Formulation . . . . . . . . . . . . . . . . . . 68

3.5 The WMMSE approach . . . . . . . . . . . . . . . . . . . . . . . . 70

3.5.1 Rate-WMMSE Relationship . . . . . . . . . . . . . . . . . . 71

3.5.2 Alternating Optimization Algorithm . . . . . . . . . . . . . 74

3.6 Simulation Results and Analysis . . . . . . . . . . . . . . . . . . . . 76

3.6.1 Performance Over Rayleigh Channels . . . . . . . . . . . . . 77

3.6.2 Application to Multibeam Satellite Systems . . . . . . . . . 81

CONTENTS 15

3.6.3 Link-Level Simulations . . . . . . . . . . . . . . . . . . . . . 85

3.7 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91

4 RSMA for Satellite-Terrestrial Integrated Networks 92

4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93

4.2 System Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95

4.2.1 Channel Model . . . . . . . . . . . . . . . . . . . . . . . . . 96

4.2.2 Coordinated scheme and Cooperative Scheme . . . . . . . . 98

4.3 Proposed Joint Beamforming Scheme . . . . . . . . . . . . . . . . . 104

4.3.1 Joint Beamforming Design for Coordinated STIN . . . . . . 105

4.3.2 Joint Beamforming Design for Cooperative STIN . . . . . . 111

4.4 Robust Joint Beamforming Scheme . . . . . . . . . . . . . . . . . . 112

4.5 Simulation Results and Analysis . . . . . . . . . . . . . . . . . . . . 119

4.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127

5 RSMA for Integrated Sensing and Communication Systems 128

5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129

5.2 System Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129

5.2.1 Sensing Model and Metric . . . . . . . . . . . . . . . . . . . 131

5.2.2 Communication Model and Metric . . . . . . . . . . . . . . 133

5.3 ISAC Beamforming Optimization . . . . . . . . . . . . . . . . . . . 135

 5.4 Simulation Results and Analysis . . . . . . . . . . . . . . . . . . . . 140

5.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144

6 Conclusion 145

6.1 Summary of Thesis Achievements . . . . . . . . . . . . . . . . . . . 145

6.2 Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147

A Transceiver modules 151

References 154

16
List of Tables

3.1 Achievable MMF-DoF of different strategies . . . . . . . . . . . . . 67

3.2 Simulation parameters [Chapter 3] . . . . . . . . . . . . . . . . . . 82

4.1 Simulation parameters [Chapter 4] . . . . . . . . . . . . . . . . . . 120

17
18
List of Figures

2.1 Transceiver architecture of K-user downlink RSMA. . . . . . . . . . 35

3.1 MMF rate performance. Nt = 6 antennas, K = 6 users, M = 3

groups, G1 , G2 , G3 = 1, 2, 3 users. . . . . . . . . . . . . . . . . . . . 78

3.2 MMF rate performance. Nt = 4 antennas, K = 6 users, M = 3

groups, G1 , G2 , G3 = 1, 2, 3 users. . . . . . . . . . . . . . . . . . . . 78

3.3 MMF rate performance. Nt = 4 antennas, K = 6 users, M = 3

groups, G1 , G2 , G3 = 2, 2, 2 users. . . . . . . . . . . . . . . . . . . . 80

3.4 Architecture of multibeam satellite systems. . . . . . . . . . . . . . 81

3.5 MMF rate versus per-feed available power. Nt = 7 antennas, K = 14

users, ρ = 2 users. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83

3.6 MMF rate versus CSIT error scalling factor α. Nt = 7 antennas,

ρ = 2, 4, 6 users, P/Nt = 80 W. . . . . . . . . . . . . . . . . . . . . 84

3.7 MMF rate constrained by PAC/ TPC. Nt = 7 antennas, K = 14

users, ρ = 2 users, imperfect CSIT: α = 0.8. . . . . . . . . . . . . . 85

3.8 MMF rate versus per-feed available power. Nt = 7 antennas, K = 14

users, imperfect CSIT: α = 0.6, hot spot G = [8, 1, 1, 1, 1, 1, 1]. . . . 86

19
20 LIST OF FIGURES

3.9 Transceiver architecture of RSMA multigroup multicast. . . . . . . 87

3.10 MMF throughput versus SNR, α = 0.8, Nt = 6 antennas, K = 6

users, 2 users per group. . . . . . . . . . . . . . . . . . . . . . . . . 88

3.11 MMF throughput versus SNR, α = 0.6, Nt = 6 antennas, K = 6

users, 2 users per group. . . . . . . . . . . . . . . . . . . . . . . . . 88

3.12 MMF throughput versus SNR, α = 0.8, Nt = 4 antennas, K = 6

users, 2 users per group. . . . . . . . . . . . . . . . . . . . . . . . . 89

3.13 MMF throughput versus SNR, α = 0.6, Nt = 4 antennas, K = 6

users, 2 users per group. . . . . . . . . . . . . . . . . . . . . . . . . 89

3.14 MMF throughput versus per-feed available power, α = 0.8, Nt = 7

antennas, K = 14 users, 2 users per group. . . . . . . . . . . . . . 90

4.1 Model of a satellite-terrestrial integrated network. . . . . . . . . . . 96

4.2 Geometry of uniform planar array employed at the BS. . . . . . . . 97

4.3 MMF rate versus Pt with different Ps and Ns . Nt = 16, Kt = 4,

Ks = ρNs , ρ = 2. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121

4.4 MMF rate versus Pt with different Ks and Kt . Nt = 16, Ns = 3, ,

Ps = 120W. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121

4.5 MMF rate versus Pt with different transmission strategies. Nt = 16,

Kt = 4, Ns = 3, Ks = 6, Ps = 120W. . . . . . . . . . . . . . . . . . 123

4.6 MMF rate versus Pt for different transmission strategies. Nt = 4,

Kt = 4, Ns = 3, Ks = 6, Ps = 120W. . . . . . . . . . . . . . . . . . 123
4.7 MMF rate versus Pt with different satellite phase uncertainties. RSMA
is adopted at the transmitters. Nt = 16, Kt = 4, Ns = 3, Ks = 6,
Ps = 120W. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126

4.8 MMF rate versus Pt with different satellite phase uncertainties. SDMA
is adopted at the transmitters. Nt = 16, Kt = 4, Ns = 3, Ks = 6,
Ps = 120W. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126

5.1 Model of an RSMA-assisted ISAC system. . . . . . . . . . . . . . . 130

5.2 MFR versus RCRB in a terrestrial ISAC system, (a) θ (◦ ), (b) αR ,

(c) αI , (d) FD . Nt = 8, Nr = 9, K = 4, L = 1024, SNRradar = −20 dB.141

5.3 Target estimation performance in a terrestrial ISAC system, (a) θ (◦ ),

(b) αR , (c) αI , (d) FD . Nt = 8, Nr = 9, K = 4, L = 1024. . . . . . 142

5.4 MFR versus RCRB in a satellite ISAC system, (a) θ (◦ ), (b) αR , (c)
αI , (d) FD . Nt = 8, Nr = 9, K = 16, L = 1024, SNRradar = −20 dB. 143

21
22
Chapter 1

Introduction

With the rapid development of wireless communications over the past few decades,
the next-generation wireless networks, e.g., beyond fifth generation (B5G) and
sixth generation (6G) have attracted widespread attention from both academia
and industry. It is envisioned that B5G/6G will enable Internet to Everything,
and will cope with the increasing demands for high throughput, reliability, hetero-
geneity of quality-of-service (QoS), and massive connectivity to satisfy the require-
ments of further-enhanced mobile broadband (FeMBB), extremely ultra reliable
and low-latency communication (eURLLC), ultra massive machine type communi-
cation (umMTC) and new services such as integrated sensing and communications,
integrated satellite-terrestrial, and extended reality [1]. To accommodate these
requirements of next-generation wireless networks, multiple access (MA) techniques
have become increasingly imperative to make better use of wireless resources and
manage interference more efficiently.

23
24 Chapter 1. Introduction

1.1 Toward Rate-Splitting Multiple Access

The past decades have witnessed the evolution of MA schemes. The previous
generations of wireless networks rely on orthogonal multiple access (OMA) which
allocates orthogonal radio resources to users to alleviate multi-user interference, such
as using frequency division multiple access (FDMA), time division multiple access
(TDMA), code division multiple access (CDMA) or orthogonal frequency division
multiple access (OFDMA). The choice of orthogonal radio resource allocation is
motivated by avoiding multiuser interference and high transceiver complexity [2].
However, such an approach leads to inefficient use of radio resources. In fourth
generation (4G) and fifth generation (5G), multiple-input multiple-output (MIMO)
processing plays a pivotal role in wireless systems, and MA techniques are adopted in
conjunction with multiuser (MU)-MIMO to achieve higher throughput by exploiting
the spatial dimension resources.

The utilization of spatial domain and multi-antenna processing opens the door for
space-division multiple access (SDMA), a well-established MA technique based on
multiuser linear precoding (MU-LP). MU-LP is an efficient precoding strategy1 for
the multi-antenna broadcast channel (BC), which relies on linear precoding (also
called beamforming) at the transmitter, and treats multiuser interference as noise at
the receivers. It is able to achieve near-capacity performance when perfect channel
state information at the transmitter (CSIT) is assumed and the user channels are
nearly orthogonal with similar channel strengths or similar long-term signal-to-noise
ratio (SNR) [2]. Through SDMA, multiple users are served in a non-orthogonal
manner in the same time-frequency domain and the interference can be significantly
mitigated by spatial beamforming. An alternative interpretation is that SDMA relies

1
In this thesis, only channel-level precoding strategies are considered. These strategies exploit
the knowledge of CSIT to design precoders to be applied to multiple data streams, thus suppressing
interference. Note that symbol-level precoding uses the knowledge of both symbols of users and
CSIT to exploit, rather than suppress constructive interference [3, 4].
1.1. Toward Rate-Splitting Multiple Access 25

on a transmit-side interference cancellation strategy. It has received considerable

attention in the past decade and has become the basic principle behind numerous 4G
and 5G techniques. However, its limitations are as follows. First, when the system
is overloaded, i.e., when the number of served users is larger than the number of
transmit antennas, the multiuser interference cannot be successfully suppressed, thus
leading to significant multiplexing gain and rate loss. Second, SDMA is sensitive to
the channel strength and orthogonality. Schedulers are expected to pair users with
nearly orthogonal channels and relatively similar channel strengths. Therefore, the
complexity of scheduling and user pairing according to the user channel conditions
is not practical when conducting an exhaustive search process [5]. Scheduling
algorithms with low complexity are required. Third, SDMA highly relies on the
availability of accurate CSIT. The beamforming and interference nulling ability
heavily depend on the CSIT accuracy. Applying SDMA designed for perfect CSIT
in the presence of imperfect CSIT can result in residual multiuser interference caused
by the imprecise beamforming at the transmitter. However, the CSIT is always
imperfect in practice due to channel estimation errors, pilot contamination, limited
and quantized feedback accuracy, delay, mobility, inaccurate calibration of radio
frequency (RF) chains, etc [6].

Another non-orthogonal MA scheme which superposes users in the same time-

frequency resource is known as power-domain non-orthogonal multiple access (NOMA).2
NOMA relies on superposition coding (SC) at the transmit side and successive in-
terference cancellation (SIC) at the receive side. SC-SIC has been studied for
decades and is well known to achieve the capacity region of the single-input single-
output (SISO) BC [7]. Through NOMA, at least one user is forced to fully decode
the messages of the other co-scheduled users, and then remove them from its ob-
servation before decoding its own message. Interference is therefore removed. An

2
In this thesis, we focus only on power-domain NOMA and simply use NOMA to represent
power-domain NOMA.
26 Chapter 1. Introduction

alternative interpretation is that NOMA relies on a receiver-side interference cancel-

lation strategy. Due to the benefits of multi-antenna over single-antenna systems,
multi-antenna NOMA has been studied in a great number of literature in the past
few years. Similar to the benefit of using SC-SIC in SISO BC, multi-antenna NOMA
is very effective to cope with an overloaded deployment especially when the channels
are closely aligned, and channel disparity can further promote the advantage of
NOMA. However, the limitations of multi-antenna NOMA are as follows. First,
different from the degraded SISO BC where the users can be ordered based on their
channel strengths, the multi-antenna BC is non-degraded and the users cannot be
ordered based on channel strengths. The precoders and decoding orders need to
be jointly optimized. As the number of users increases, the number of decoding
orders increases exponentially. Second, a great number of SICs are conducted at
the receivers, which results in high receiver complexity. The number of SIC layers
increases as the user number grows. Accordingly, multi-antenna NOMA can impose
a significant computational burden on both the transmitter and the receivers. Third,
the spatial domains cannot be efficiently used, and there exists a multiplexing gain
loss. As analyzed in [8], for a multiple-input single-output (MISO) BC, the sum
multiplexing gain achieved by multi-antenna NOMA is reduced to unity, which is
the same as the multiplexing gain of OMA/single user MISO transmission. Fourth,
the performance of NOMA is sensitive to channel strength and orthogonality. When
the user channels are not aligned or with similar channel strengths, the performance
degrades.

Indeed, SDMA and NOMA can be seen as two extreme interference management
strategies, namely fully treating interference as noise and fully decoding interference.
To overcome the limitations of both strategies and take full advantage of their
benefits, rate-splitting multiple access (RSMA) has emerged as a promising and
powerful non-orthogonal transmission, interference management and MA scheme
for future multi-antenna wireless networks owning to its capability to enhance
1.2. Motivation and Organization 27

the system performance in a wide range of network loads, user deployments and
CSIT qualities. In [9], RSMA has been analytically demonstrated to generalize
several existing MA techniques, namely SDMA, NOMA, OMA and physical-layer
multicasting. RSMA relies on linearly preceded rate-splitting at the transmitter,
and SIC at the receivers. The key behind the flexibility and robust manner of
RSMA is to split user messages into common and private parts such that each of
these parts can be decoded flexibly at one or multiple receivers. Through SIC, users
sequentially decode the intended common streams (and therefore decode part of the
interference). The private streams are only decoded by their corresponding users.
This framework enables the capability of RSMA to partially decode the interference
and partially treat interference as noise. Alternatively, RSMA can be interpreted
as a smart combination of transmit-side and receive-side interference cancellation
strategy, where the contribution of the common parts and the power allocated
to the common and private parts can be adjusted flexibly [1]. This departs from
the transmit-side-only and receive-side-only interference management strategies,
e.g., SDMA and NOMA respectively. As a consequence, RSMA has the flexibility
to cope with various interference levels and user deployment scenarios. RSMA is
very robust to channel disparity, channel orthogonality and network loads [5]. It is
demonstrated to provide benefits in terms of multiplexing gain, system spectral and
energy efficiency with both perfect CSIT and imperfect CSIT [1, 2].

1.2 Motivation and Organization

With the explosive growth of data traffic and high demand for wireless connectivity
in B5G/6G, existing cellular infrastructures may no longer provide ubiquitous and
high-capacity global coverage to rural and remote areas [10]. Thereby, non-terrestrial
network (NTN) is envisioned to provide heterogeneous services and seamless network
28 Chapter 1. Introduction

coverage for everyone and everything by complementing and extending terrestrial

networks. The roles of NTNs lie in enhancing the availability in unserved (e.g.,
deserts, oceans, forests) or underserved areas (e.g., rural areas), enabling service
reliability by providing service continuity for Internet-of-Things (IoT) devices or
passengers on board moving platforms, and offering an infrastructure resilient to
natural disasters on the ground.

In this thesis, amongst the NTN platforms spanning from satellite-based and airborne-
based platforms, we particularly focus on the multibeam satellite systems that have
received considerable attention in recent years due to the full frequency reuse
across multiple narrow spot beams towards higher throughput [11,12]. The available
spectrum is aggressively reused, and thus inter-beam interference increases. Moreover,
by combining the advantages of both satellite and terrestrial networks, the satellite-
terrestrial integrated network (STIN) architecture shows great potential to find
a new development path toward ubiquitous wireless networks [13]. The satellite
sub-network shares the same frequency band as the terrestrial sub-network, and
severe interference in and between the subnetworks is induced. Hence, analogous to
terrestrial networks, it is deemed necessary to explore efficient MA strategies.

In addition to the demand for high-quality wireless connectivity, a common theme

for the future trend is that sensing will play a more significant role than ever
before and the demand for robust and accurate sensing capability increases [14].
Sharing of the frequency bands between radar sensors and communication systems
has received considerable attention from both industry and academia, therefore
motivating the research on integrated sensing and communications (ISAC) systems.
ISAC systems can simultaneously perform wireless communications and remote
sensing. Both functionalities are combined via shared use of the spectrum, the
hardware platform and a joint signal processing framework. Such design leads
to a trade-off between communication and radar performance and also calls for
1.2. Motivation and Organization 29

flexible and robust MA strategies. Driven by the appealing benefits of RSMA

in multi-antenna wireless communications, the main objective of this thesis is to
investigate the application of RSMA to the aforementioned scenarios and enabling
technologies in non-terrestrial communication and sensing networks, namely the
multigroup multicast and multibeam satellite systems, STIN and ISAC systems,
which are envisioned to play key roles in next-generation wireless networks.

The remainder of this thesis is organised as follows:

In Chapter 2, we review the fundamentals and related works of this thesis by

introducing the principles of downlink RSMA and the state-of-the-art works on
multigroup multicast and multibeam satellite systems, STIN and ISAC systems.

In Chapter 3, we investigate the application of RSMA for multigroup/multibeam

multicast beamforming in the presence of imperfect CSIT. The effectiveness of
RSMA is demonstrated in both cellular multigroup multicast and multibeam satellite
systems to manage interference, taking into account various practical challenges.
Physical (PHY) layer design and link-level simulations are also investigated. RSMA
is demonstrated to be a very promising MA strategy for practical implementation
in numerous application areas.

In Chapter 4, we introduce RSMA to STIN considering either perfect CSIT or

imperfect CSIT with satellite channel phase uncertainties at the gateway (GW). Two
RSMA-based STIN schemes are presented. Simulation results show the superiority
of the proposed RSMA-based STIN to manage the interference in and between the
satellite and terrestrial sub-networks.

In Chapter 5, we investigate the application of RSMA for ISAC systems, where the
ISAC platform has a dual capability to simultaneously communicate with downlink
users and probe detection signals to a moving target. Through RSMA-assisted ISAC
beamforming design, RSMA is shown to be very promising for both terrestrial and
30 Chapter 1. Introduction

satellite ISAC systems to manage the multiuser/inter-beam interference as well as

performing the radar functionality.

Finally, in Chapter 6, we conclude the thesis by summarising the achievements and

discussing possible directions for future works.

1.3 List of Contributions

The novel contributions of this thesis are listed as follows:

1. Max-min fairness (MMF)-degree of freedom (DoF) of RSMA is analyzed

in multigroup multicast with imperfect CSIT. RSMA is shown to provide
MMF-DoF gains in both underloaded and overloaded systems.

2. An RSMA-based MMF multigroup multicast beamforming optimization prob-

lem with imperfect CSIT is formulated to investigate whether the MMF-DoF
gain translates into MMF rate gain. A weighted minimum mean square
error (WMMSE) algorithm is developed to solve the problem.

3. The RSMA-based multigroup multicast framework and algorithm are applied

to a multibeam satellite setup. A novel RSMA-based multibeam multicast
beamforming scheme is therefore studied.

4. The RSMA transmitter and receiver architecture and link-level simulation

(LLS) platform are designed by considering finite length polar coding, finite
alphabet modulation, adaptive modulation and coding (AMC) algorithm, etc.

5. The MMF rate/throughput performance of the proposed RSMA-based multi-

group/multibeam multicast is compared with conventional strategies in both
cellular and multibeam satellite systems.
1.4. Publications 31

6. A multiuser downlink RSMA-based STIN is presented, where the GW operates

as a control center to implement centralized processing. Two integration levels
of RSMA-based STIN are proposed, namely the coordinated scheme, and the
cooperative scheme.

7. MMF RSMA-based STIN beamforming optimization problems are formulated

considering both perfect CSIT and imperfect CSIT with satellite channel phase
uncertainty. Iterative algorithms based on SCA are proposed respectively to
solve the optimization problems.

8. The MMF rate performance of the proposed RSMA-based STIN is compared

with several conventional baseline strategies.

9. A general RSMA-assisted ISAC system is presented, where the antenna array

is shared by a co-located monostatic MIMO radar system and a multiuser
communication system.

10. An RSMA-assisted ISAC beamforming optimization problem is formulated to

investigate the trade-off between the radar and communication performance.
An iterative algorithm based on SCA is proposed to solve the problem.

11. The performance of the proposed RSMA-assisted ISAC beamforming is com-

pared with conventional baseline strategies considering both terrestrial and
multibeam satellite ISAC systems.

1.4 Publications

The material presented in this thesis has led to the following publications:

1. L. Yin and B. Clerckx, ”Rate-Splitting Multiple Access for Multigroup Multi-

cast and Multibeam Satellite Systems,” in IEEE Transactions on Communi-
32 Chapter 1. Introduction

cations, vol. 69, no. 2, pp. 976-990, Feb. 2021. [Chapter 3]

2. L. Yin and B. Clerckx, ”Rate-Splitting Multiple Access for Multibeam Satellite

Communications,” in IEEE International Conference on Communications
Workshops (ICC Workshops), 2020, pp. 1-6. [Chapter 3]

3. L. Yin, O. Dizdar and B. Clerckx, ”Rate-Splitting Multiple Access for Multi-

group Multicast Cellular and Satellite Communications: PHY Layer Design
and Link-Level Simulations,” in IEEE International Conference on Communi-
cations Workshops (ICC Workshops), 2021, pp. 1-6. [Chapter 3]

4. L. Yin and B. Clerckx, ”Rate-Splitting Multiple Access for Satellite-Terrestrial

Integrated Networks: Benefits of Coordination and Cooperation,” in IEEE
Transactions on Wireless Communications, vol. 22, no. 1, pp. 317-332, Jan.
2023. [Chapter 4]

5. L. Yin, Y. Mao, O. Dizdar and B. Clerckx, ”Rate-Splitting Multiple Access

for 6G—Part II: Interplay With Integrated Sensing and Communications,” in
IEEE Communications Letters, vol. 26, no. 10, pp. 2237-2241, Oct. 2022.
[Chapter 5]

6. L. Yin and B. Clerckx, ”Rate-Splitting Multiple Access for Dual-Functional

Radar-Communication Satellite Systems,” in IEEE Wireless Communications
and Networking Conference (WCNC), 2022, pp. 1-6. [Chapter 5]

1.5 Notation

The following notation is used throughout the thesis. Boldface uppercase, bold-
face lowercase and standard letters denote matrices, column vectors, and scalars
respectively. The N × N identity matrix is denoted by IN , where N is the size
of the identity matrix. R and C denote the real and complex domains. E (·) is
1.5. Notation 33

the expectation of a random variable. The real part of a complex number x is

represented by R (x). The operators (·)T and (·)H denote the transpose and the
Hermitian transpose respectively. tr () denotes the trace operator. A ⪰ 0 means
that the symmetric matrix A is positive semidefinite. diag (a) is a diagonal matrix
with diagonal entries given by the elements of a. On the other hand, diag (A) is a
vector with entries given by the diagonal elements of A. |·| denotes the absolute
value or the size of a set. ∥·∥ denotes the Euclidean norm.
Chapter 2

Background

In this chapter, the background knowledge and state-of-the-art works covered in this
thesis are presented, including the fundamentals of downlink RSMA, multigroup
multicast and multibeam satellite systems, satellite-terrestrial integrated networks
and integrated sensing and communications.

2.1 Fundamentals of Downlink RSMA

We consider a MISO BC system, where K single-antenna users are simultaneously

served by a base station (BS) equipped with Nt transmit antennas. As shown in Fig.
2.1, the message intended for user-k, k ∈ K = {1, · · · , K} denoted as Wk is split
into a common part Wc,k and a private part Wp,k . The common parts are combined
into a common message Wc = {Wc,1 , · · · , Wc,K }, which is then encoded into a single
common stream Wc → sc . The private messages are encoded into corresponding
private streams Wp,k → sk , ∀k ∈ K.

By defining pc ∈ CNt ×1 and pk ∈ CNt ×1 , ∀k ∈ K as the linear precoders/beamforming

34
2.1. Fundamentals of Downlink RSMA 35

Figure 2.1: Transceiver architecture of K-user downlink RSMA.

vectors, the transmit signal x ∈ CNt ×1 can be written as

K
X
x = Ps = pc sc + pk sk , (2.1)
k=1

where P = [pc , p1 , · · · , pK ] denotes the beamforming matrix. The vector of symbol

streams to be transmitted is s = [sc , s1 , · · · , sK ]T ∈ C(K+1)×1 . We assume E ssH =



I, thus the sum transmit power constraint at the transmitter is tr PPH ≤ P ,
where P is the transmit power budget. The received signal at user-k is given by

K
X
yk = hH
k x + nk = hH
k pc s c + hH
k pk sk + nk , (2.2)
k=1

where hk ∈ CNt ×1 denotes the channel vector between the transmitter and the k-th
2


user. nk ∼ CN 0, σn,k is the receiver additive white Gaussian noise (AWGN) of
2 2 2
zero mean and variance σn,k . It is assumed that σn,1 , · · · , σn,K = σn2 .

At the receiver side, each user sequentially decodes the common stream and the
intended private stream to recover its message. User-k first decodes the common
stream by treating the interference from all private streams as noise. Hence, the
36 Chapter 2. Background

signal-to-interference-noise ratio (SINR) of decoding sc at user-k is expressed as

2
hH
k pc
γc,k = P 2 . (2.3)
H 2
i∈K |hk pi | + σn

After successfully decoding and removing the common stream using SIC1 , user-k
decodes its own private stream by treating the private streams of other users as
noise. By considering perfect SIC, the SINR of decoding sk at user-k is expressed as

2
hH
k pk
γk = P 2 . (2.4)
H 2
i∈K,i̸=k |hk pi | + σn

Under the assumption of Gaussian signalling and infinite block length, the achievable
rates for decoding the common and private streams at user-k are respectively
Rc,k = log2 (1 + γc,k ) and Rk = log2 (1 + γk ). To ensure the common stream sc is
successfully decoded by all users, its rate cannot exceed Rc = mink∈K Rc,k . Since sc
contains messages Wc,1 , · · · , Wc,K of the K users, let Ck denote the portion of rate
P
Rc allocated to user-k for Wc,K . Then, we have Rc = k∈K Ck . As a consequence,
the overall achievable rate of user-k is writtn as Rk,tot = Ck + Rk .

The beamforming matrix P = [pc , p1 , · · · , pK ] can be designed using low-complexity

methods, such as zero-forcing beamforming (ZFBF) for the private streams and
multicast precoders (e.g., dominant singular vector of concatenated channel matrix)
for the common stream. Alternatively, the precoders can be optimized via different
objectives, e.g., maximizing the weighted sum-rate (WSR) [5, 16], maximizing the
energy efficiency (EE) [17], etc.

It should be noted that in this section, the fundamentals of downlink RSMA is

introduced based on one-layer RSMA, the simplest and most practical RSMA
implementation [2], and will be utilized throughout this thesis. Interested readers

1
Throughout this thesis, perfect CSIR is assumed, where the common stream can be removed
perfectly by SIC. For imperfect CSIR, please see [15].
2.1. Fundamentals of Downlink RSMA 37

are referred to [2] for a more comprehensive study on the other forms of RSMA.
One-layer RSMA requires only one layer of SIC at each receiver. User grouping
and ordering are not required since each user decodes the common stream before
decoding its private stream. Compared with the generalized RSMA elaborated in [5],
which involves multiple common streams and requires multiple SIC layers at the
receivers, the encoding complexity, scheduling complexity and receiver complexity
are reduced tremendously. Results in [5] show that the low complexity one-layer
RSMA has a comparable rate performance to the generalized RSMA. The advantage
of complexity reduction becomes more significant when the user number increases.
Thanks to the inherent message splitting capability which is not featured in any
other MA schemes, RSMA allows to:

1) partially decode interference and partially treat interference as noise (hence its
efficiency, flexibility, reliability, and resilience),

2) reconcile the two extreme strategies of interference management and multiple MA

schemes into a single framework (hence its generality/universality),

3) achieve optimal DoF in practical scenarios subject to imperfect CSIT [1].

In the literature, the benefits achieved by RSMA have been investigated in a wide
range of multi-antenna scenarios, namely multiuser unicast transmission with perfect
CSIT [5,9,18,19], imperfect CSIT [16,20–25], multigroup multicast transmission [26–
29], as well as superimposed unicast and multicast transmission [17], etc. According
to the analysis and simulations, [5] shows that RSMA is more robust to the influencing
factors such as channel disparity, channel orthogonality, network load, and quality
of CSIT. For imperfect CSIT, the sum-DoF and MMF-DoF of underloaded MU-
MISO system are studied in [16] and [21]. RSMA is demonstrated to further
exploit spatial dimensions. The superior performance of RSMA can also be seen
in massive MIMO systems with residual transceiver hardware impairments [30],
38 Chapter 2. Background

mmWave communications [31] and simultaneous wireless information and power

transfer (SWIPT) networks [17], etc. The performance gains of RSMA are not just
limited to the assumption of Gaussian signalling and infinite block lengths, but are
realized for practical systems as well in throughput performance through link-level
simulations [32, 33].

2.2 Multibeam Satellite Systems

Satellite communications, appealing for its ubiquitous coverage, will play a key
role in the next generation of wireless communications [34]. It not only provides
connectivity in unserved areas but also decongests dense terrestrial networks. In
recent years, multibeam satellite communication systems have received considerable
research attention due to the full frequency reuse across multiple narrow spot beams
towards higher throughput [11, 12]. Multibeam satellites are equipped with multiple
antenna feeds and serve multiple user groups within multiple co-channel beams. Since
the available spectrum is aggressively reused, interference management techniques
become particularly important. Based on state-of-the-art technologies in DVB-
S2X [35], each spot beam of the satellite serves more than one user simultaneously
by transmitting a single coded frame. Multiple users within the same beam share
the same precoding vector. Since different beams illuminate different groups of
users [36], this promising multibeam multicasting follows the physical layer (PHY)
multigroup multicast transmission.

Multicast transmission is at first considered in [37] with a single-group setup. Then,

the problem is extended to multigroup multicast in [38] where the beamforming
design is investigated in two optimization perspectives, namely the QoS-constrained
transmit power minimization (QoS problem) and the power-constrained max-min
fairness (MMF) problem. Both formulations are shown to be NP-hard, containing the
2.2. Multibeam Satellite Systems 39

multiuser unicast and the single-group multicast as extreme cases. The combination
of semi-definite relaxation (SDR) and Gaussian randomization, together with the
bisection search algorithm are elaborated to generate feasible approximate solutions.
Alternatively, a convex-concave procedure (CCP) [39] algorithm is demonstrated to
provide better performance. However, its complexity increases dramatically as the
problem size grows. In [40], a low-complexity algorithm for multigroup multicast
beamforming based on alternating direction method of multipliers (ADMM) together
with CCP is proposed for large-scale wireless systems. Moreover, the multigroup
multicast beamforming is extended to many other scenarios, including the per-
antenna power constraint addressed in [41], Cloud-radio access network (RAN)
with wireless backhaul [42], coordinated beamforming in multi-cell networks [43],
cache aided networks [44] and massive MIMO [45]. Since one practical application
of multigroup multicast is found in multibeam satellite communication systems,
in the literature of multibeam satellite systems, a generic iterative algorithm is
proposed in [46] to design the precoding and power allocation alternatively in a
TDM scheme considering a single user per beam. Then, multibeam multicast is
considered. [35] proposes a frame-based precoding problem for multibeam multicast
satellites. Optimization of the system sum rate is considered under individual
power constraints via an alternating projection technique with an SDR procedure,
which is adequate for small to medium-coverage areas. In [47], a two-stage low
complex beamforming design for multibeam multicast satellite systems is proposed.
The first stage minimizes inter-beam interference, while the second stage enhances
intra-beam SINR. [36] studies the sum rate maximization problem in multigateway
multibeam satellite systems considering feeder link interference. Leakage-based
minimum mean square error (MMSE) and successive convex approximation (SCA)-
ADMM algorithms are used to compute beamforming vectors locally with limited
coordination.

All aforementioned works rely on the conventional multigroup/multibeam multicast

40 Chapter 2. Background

linear precoding (denoted as SDMA in this thesis). Each user decodes its desired
stream while treating all the interference streams as noise. The advantage of this
conventional scheme lies in exploiting the spatial degrees of freedom provided by
multiple antennas using low-complexity transmitter-receiver architecture. However,
its effectiveness severely depends on the network load and the quality of CSIT. Since
the precoders are designed based on the channel knowledge, CSIT inaccuracy can
result in an inter-group interference problem which is detrimental to the system
performance. Another limitation is that the SDMA is able to eliminate inter-group
interference only when the number of transmit antennas is sufficient. Otherwise,
it fails to do so in overloaded systems [26]. For example, rate saturation occurs in
overloaded systems. Departing from SDMA, the employment of RSMA in multi-
group multicast beamforming is at first proposed in [26]. The key of RSMA-based
multigroup multicast beamforming is to divide each group-intended message into a
common part and a private part. An RSMA-based MMF problem was formulated
and solved by the WMMSE approach [48]. The superiority of RSMA with perfect
CSIT is shown in overloaded multigroup multicast systems.

2.3 Satellite-Terrestrial Integrated Networks

In recent years, due to the explosive growth of wireless applications and multimedia
services, STIN has gained a tremendous amount of attention in both academia and
industry as it can provide ubiquitous coverage and convey rich multimedia services,
e.g., video on demand (VoD) streaming and TV broadcasting, etc. to users in both
densely and sparsely populated areas [49]. The integration of terrestrial and satellite
networks is of great potential in achieving geographic coverage, especially for remote
areas where no terrestrial BS infrastructure can be employed [50, 51]. It is envisaged
that the C-band (4 − 8 GHz) and S-band (2 − 4 GHz) can be shared between the
2.3. Satellite-Terrestrial Integrated Networks 41

terrestrial and satellite networks. In addition, Ka band from 20 GHz to 40 GHz

is foreseen to be the most promising candidate radio band for the next generation
terrestrial cellular networks, and part of this band has already been allocated to the
satellite networks [52].

A number of research efforts have investigated STIN systems. A coexistence frame-

work of the satellite and terrestrial network is presented in [53] with the satellite
link as primary and the terrestrial link as secondary. Transmit beamforming tech-
niques are studied to maximize the SINR towards terrestrial users and minimize
the interference towards satellite users. [54] considers a time division cooperative
STIN, where a weighted MMF problem was formulated to jointly optimize the
beamforming of BSs and the satellite. A multicast beamforming STIN system
is investigated in [55] with the aim to maximize the sum of user minimum ratio
under constraints of backhaul links and QoS. The authors generally assume the
satellite channels as Rician channels. The effects of satellite antenna gain, path loss
and atmospheric attenuation can be taken into account to model more practical
satellite channels so as to evaluate the system performance more accurately. In
this regard, [52] investigates a joint beamforming scheme for secure communication
of STIN operating in mmWave frequencies. [56] focuses on the joint optimization
for wireless information and power transfer (WIPT) technique in STINs. In [57],
the cache-enabled low Earth orbit (LEO) satellite network is introduced, and the
scheme of STIN is proposed to enable an energy-efficient RAN by offloading traffic
from BSs through satellite broadcast transmission.

The above works consider conventional linear precoding and assume perfect CSIT.
Each user decodes its desired stream while treating all the other interference streams
as noise. The spatial degrees of freedom provided by multiple antennas are exploited,
however, the effectiveness of beamforming design relies on the accuracy of CSIT
significantly. In the real satellite communication environment, one practical issue is
42 Chapter 2. Background

that accurate CSI is very difficult to acquire at the GW because of the long-distance
propagation delay and device mobility. Thus, robust design in the presence of
imperfect CSIT has been widely studied in the literature [58–63]. [58–60] assume the
satellite channel uncertainty as additive estimation error located in a bounded error
region. Robust beamforming is designed based on the optimization of the worst-case
situation. Yet, due to the special characteristics of satellite channels, the channel
magnitude does not vary significantly due to the fact that the channel propagation
is dominated by the line-of-sight component. The phase variations constitute the
major source of channel uncertainty [11]. Therefore, in [61–63], beamforming is
studied when considering constant channel amplitudes within the coherence time
interval and independent time-varying phase components. Considering the phase-
blind scenario, the achievable rate performance of RSMA in an multiuser MISO
network is investigated in [64]. Apart from the difficulties in acquiring perfect CSIT,
another consideration is the frame-based structure of multibeam satellite standards
such as DVB-S2X [65]. Each spot beam of the satellite serves more than one user
simultaneously by transmitting a single coded frame. Multiple users within the same
beam share the same beamforming vector. Such multibeam multicast transmission
is a promising solution for the rapidly growing content-centric applications including
video streaming, advertisements, large-scale system updates and localized services,
etc.

More recently, the use of RSMA in multibeam satellite and integrated satellite
systems has been investigated. [25] studies RSMA in a two-beam satellite system
adopting TDM in each beam. [66] focuses on the sum rate optimization and low
complexity RSMA precoding design by decoupling the design of common stream and
private streams. [67, 68] propose a RSMA-based multibeam multicast beamforming
scheme and formulate an MMF problem with different CSIT qualities. In [69],
RSMA is proven to be promising for multigateway multibeam satellite systems
with feeder link interference. [70] considers a satellite and aerial integrated network
2.4. Integrated Sensing and Communications 43

comprising a satellite and an unmanned aerial vehicle (UAV). The satellite employs
multicast transmission, while the UAV uses RSMA to improve spectral efficiency.
In [71], a secure beamforming scheme for STIN is presented, where the satellite
serves one earth station (ES) with K eavesdroppers (Eves). RSMA is employed
at the BS to achieve higher spectral efficiency. A robust beamforming scheme is
proposed to maximize the secrecy energy efficiency of the ES considering Euclidean
norm bounded channel uncertainty.

2.4 Integrated Sensing and Communications

ISAC has been envisioned as a key technique for future 6G wireless networks to fulfil
the increasing demands on high-quality wireless connectivity as well as accurate
and robust sensing capability [14]. ISAC merges wireless communications and
remote sensing into a single system, where both functionalities are combined via
shared use of the spectrum, the hardware platform, and a joint signal processing
framework. ISAC systems are typically categorized into three types: radar-centric
design, communication-centric design, and joint beamforming design [72]. This
thesis will only focus on the joint beamforming design of ISAC rather than relying
on existing radar or communication waveforms [73, 74].

ISAC has been considered in several promising terrestrial applications, including

autonomous vehicles, human activity monitoring, indoor positioning, etc [75–77].
In [78], a novel framework is proposed for the transmit beamforming of the joint
multi-antenna radar-communication (RadCom) system. The precoders are designed
to formulate an appropriate desired radar beampattern, while guaranteeing the SINR
requirements of the communication users. The authors in [79] propose the joint
waveform design such that the multiuser interference is minimized while formulating
a desired radar beampattern. [72] investigates the joint waveform design with
44 Chapter 2. Background

emphasis on optimizing the target estimation performance, measured by Cramér-

Rao bound (CRB) considering both point and extended target scenarios.

Inspired by the advantages of RSMA in spectral efficiency, energy efficiency, user

fairness, reliability, and QoS, performance enhancements in a wide range of network
loads (underloaded and overloaded) and channel conditions, etc, the interplay
between RSMA and ISAC systems is proposed in [80, 81], which demonstrates the
benefits of an RSMA-assisted ISAC system with the objective of jointly maximising
WSR and minimising mean square error (MSE) of beampattern approximation
considering the per-antenna power constraint. As a step further, RSMA-assisted
ISAC is studied in [82] considering partial CSIT and mobility of communication
users as a practical application. RSMA is shown to better manage the interference
and improve the trade-off between WSR and MSE of beampattern approximation
compared with other MA strategies such as SDMA and NOMA. The design of
RSMA-assisted ISAC with low resolution digital-to-analog converter (DAC) units is
introduced in [83], where RSMA is shown to achieve improved energy efficiency by
employing a smaller number of RF chains, owing to its generalized structure and
improved interference management capabilities.
Chapter 3

RSMA for Multigroup Multicast

and Multibeam Satellite Systems

This chapter is concerned with RSMA and its beamforming design problem to
achieve MMF among multiple co-channel multicast groups with imperfect CSIT.
Contrary to the conventional SDMA for multigroup multicast that relies on linear
precoding and fully treating any residual interference as noise, we consider a novel
multigroup multicast beamforming strategy based on RSMA. We characterize the
MMF-DoF achieved by RSMA and SDMA in multigroup multicast with imperfect
CSIT and demonstrate the benefits of RSMA for both underloaded and overloaded
scenarios. Motivated by the DoF analysis, we then formulate a generic transmit power
constrained optimization problem to achieve MMF rate performance. PHY layer
design and link-level simulations are also investigated. The superiority of RSMA-
based multigroup multicast beamforming compared with conventional schemes is
demonstrated via simulations in both terrestrial multigroup multicast and multibeam
satellite systems. In particular, due to the characteristics and challenges of multibeam
satellite systems, the proposed RSMA-based strategy is shown promising to manage
its inter-beam interference.

45
46 Chapter 3. RSMA for Multigroup Multicast and Multibeam Satellite Systems

3.1 Introduction

With the proliferation of mobile data and multimedia traffic, demands for massive
connectivity and content-centric services are continuously rising. Examples include
audio/video streaming, advertisements, large-scale system updates, localized services
and downloads, etc. Spurred by such requirements, wireless multicasting has
attracted widespread research attention. It is a promising solution to deliver the
same message to a group of recipients. In a more general scenario, which is known
as multigroup multicasting, distinct contents are simultaneously transmitted to
multiple co-channel multicast groups. Since the available spectrum is aggressively
reused towards spectrum efficient and high throughput wireless communications,
advanced interference mitigation techniques are of particular importance.

In this chapter, motivated by exploring the benefits of RSMA for multigroup multicast
beamforming, we consider both underloaded and overloaded regimes with imperfect
CSIT and its application to multibeam satellite systems. The main contributions
are as follows:

• First, the MMF-DoF of RSMA and SDMA in multigroup multicast with

imperfect CSIT is characterized. The MMF-DoF, also known as the MMF
multiplexing gain, indicates the maximum multiplexing gain that can be
simultaneously achieved by all multicast groups. It reflects the pre-log factor of
MMF-rate at high SNR. This is the first work on DoF analysis for multigroup
multicast in the presence of imperfect CSIT. In [26], MMF-DoF gains of RSMA
with perfect CSIT are only observed in overloaded systems. In this chapter
with an imperfect CSIT setting, RSMA is shown to provide MMF-DoF gains
in both underloaded and overloaded systems. Through residual interference
and group partitioning analysis, RSMA is shown to be more flexible than
SDMA to overcome the residual interference caused by imperfect CSIT. By
3.1. Introduction 47

adjusting the common stream and private streams, we can determine how
much interference to be decoded and how much to be treated as noise. Due to
the existence of a common part, RSMA provides extra gains and avoids the
saturating performance at the high SNR regime.

• Second, motivated by the benefits of RSMA from a DoF perspective, an MMF

beamforming optimization problem is formulated to investigate whether the
DoF gain translates into rate gain. This is the first work on the optimization
of RSMA-based multigroup multicast with imperfect CSIT. Solving the MMF
problem with imperfect CSIT via sample average approximation (SAA) and
WMMSE is for the first time studied. The optimum MMF Ergodic rate can
be obtained by optimizing the defined short-term MMF Average rate over a
long sequence of channel estimates. The formulated problem is general enough
to cope with flexible power constraints, namely a total power constraint (TPC)
and per-antenna power constraints (PAC). Through simulation results, the
DoF benefits of RSMA translate into rate benefits at finite SNR and RSMA
is shown to outperform SDMA in a wide range of setups. All the simulation
results are inline with the derived theoretical MMF-DoFs results. Considering
imperfect CSIT, we show that RSMA for multigroup multicast brings rate
gains compared with SDMA in both underloaded and overloaded scenarios.
This contrasts with the perfect CSIT setting of [26], where RSMA is shown to
provide gains in the overloaded scenarios only.

• Third, the proposed RSMA framework is applied to a multibeam satellite setup

and the results confirm the significant performance gains over conventional
schemes. Based on state-of-the-art technologies in DVB-S2X, each spot beam of
the satellite serves more than one user simultaneously by transmitting a single
coded frame. This multibeam multicast scheme follows the PHY multigroup
multicast transmission. Different from [84], which studies RSMA in a two-
48 Chapter 3. RSMA for Multigroup Multicast and Multibeam Satellite Systems

beam satellite system adopting TDM scheme in each beam, and [85] which
focuses on the sum rate optimization and low-complexity RSMA beamforming
assuming perfect CSIT, we consider a novel RSMA-based multibeam multicast
beamforming in this chapter and formulate a per-feed power constrained MMF
problem. RSMA is shown very promising for multibeam satellite systems to
manage inter-beam interference, taking into account practical challenges such
as CSIT uncertainty, per-feed transmit power constraints, hot spots, uneven
user distribution per beam, and overloaded regimes. Simulation results confirm
the significant performance gains over conventional techniques.

• Fourth, the RSMA transmitter and receiver architecture, PHY layer and
LLS platform are designed by considering finite length polar coding, finite
alphabet modulation, AMC algorithm, etc. LLS results verify the effectiveness
of RSMA-based multigroup multicast for practical implementation.

3.2 System Model

We consider a multigroup multicasting downlink MISO system. The transmitter is

equipped with Nt antennas, serving K single-antenna users which are grouped into
M (1 ≤ M ≤ K) multicast groups. The users within each group desire the same
multicast message. The messages are independent amongst different groups. Let
Gm denote the set of users belonging to the m-th group, for all m ∈ M = {1 · · · M }.
The size of group-m is Gm = |Gm |. We assume that each user belongs to only one
group, thus Gi ∩ Gj = ∅, for all i, j ∈ M, i ̸= j. Let K = {1 · · · K} denote the set
of all users, i.e., ∪m∈M Gm = K. In this model, the signal received at user-k writes
Nt ×1
as yk = hH
k x + nk , ∀k ∈ K, where x ∈ C is the transmitted signal, hk ∈ CNt ×1
is the channel vector between the transmitter and the k-th user. H ≜ [h1 , · · · , hK ]
2


is the composite channel. nk ∼ CN 0, σn,k represents the AWGN at user-k, which
3.2. System Model 49

is independent and identically distributed (i.i.d) across users with zero mean and
2
variance σn,k . Without loss of generality, unit noise variances are assumed, i.e.,
2
σn,k = σn2 = 1.

3.2.1 Transceiver Scheme

The application of RSMA to multigroup multicasting is described as follows. We

assume that there are M messages W1 , · · · , WM intended to users in G1 , G2 , · · · , GM
respectively. Each message is split into a common part and a private part, i.e.,
Wm → {Wm,c , Wm,p }. All the common parts are packed together and encoded
into a common stream shared by all groups, i.e., {W1,c · · · WM,c } → sc , while the
private parts are encoded into private streams for each group independently, i.e.,
Wm,p → sm . As a consequence, the vector of symbol streams to be transmitted is
s = [sc , s1 , · · · , sM ]T ∈ C(M +1)×1 , where E ssH = IM +1 . Data streams are then


mapped to transmit antennas through a beamforming matrix P = [pc , p1 , · · · pM ] ∈

CNt ×(M +1) . This yields a transmit signal x ∈ CNt ×1 given by

M
X
x = Ps = pc sc + pm sm , (3.1)
m=1

where pc ∈ CNt ×1 is the common precoder, and pm ∈ CNt ×1 is the m-th group’s
precoder. Moreover, flexible transmit power constraints are considered in this work,
including a total power constraint and per-antenna power constraints. Since the
average power of transmit symbols are normalized to be one, the expression of a
general transmit power constraint writes as

M
X
pH
c Dl pc + pH
m Dl pm ≤ Pl , l = 1 · · · L, (3.2)
m=1

where L is the number of power constraints. Pl is the l-th power limit, and Dl is a
diagonal shaping matrix changing among different demands. In particular, when the
50 Chapter 3. RSMA for Multigroup Multicast and Multibeam Satellite Systems

focus is on a total transmit power constraint, let L = 1, Dl = I and Pl = P > 0, from

which P equals to the transmit SNR. However, in some practical implementations,
using individual amplifiers per-antenna causes a lack of flexibility in sharing energy
resources. Such a scenario is typically found in multibeam satellite communications
because flexible on-board payloads are costly and complex to implement. Per-
antenna available power constraints are taken into account by setting L = Nt , and
Pl = P/Nt . The matrix Dl becomes a zero matrix except its l-th diagonal element
equaling 1. Then, we define µ : K → M as mapping a user to its corresponding
group. The signal received at user-k can be expanded as

M
X
yk = hH
k pc sc + hH
k pµ(k) sµ(k) + hH
k pj sj + nk , (3.3)
j=1,j̸=µ(k)

where µ (k) is the group index of user-k. Each user at first decodes the common
stream sc and treats M private streams as noise. The SINR of decoding sc at user-k
is
2
hH
k pc
γc,k = 2 PM 2
. (3.4)
hH
k pµ(k) + j=1,j̸=µ(k) |hH 2
k pj | + σn

Its corresponding achievable rate writes as Rc,k = log2 (1 + γc,k ). To guarantee that
each user is capable of decoding sc , we define a common rate Rc at which sc is
communicated
Rc ≜ min Rc,k . (3.5)
k∈K

PM
Note that sc is shared among groups such that Rc ≜ m=1 Cm , where Cm cor-
responds to group-m’s portion of common rate. After the common stream sc is
decoded and removed through SIC, each user then decodes its desired private stream
by treating all the other interference streams as noise. The SINR of decoding sµ(k)
at user-k is given by
2
hH
k pµ(k)
γk = PM 2 . (3.6)
j=1,j̸=µ(k) |hH 2
k pj | + σn
3.2. System Model 51

Its corresponding achievable rate is Rk = log2 (1 + γk ). In terms of group-m, the

multicast information sm should be decoded by all users in Gm . Thus, the shared
information rate rm is determined by the weakest user in Gm and defined as

rm = min Ri . (3.7)
i∈Gm

The m-th group-rate is composed of Cm and rm , and writes as

RS
rg,m = Cm + rm = Cm + min Ri . (3.8)
i∈Gm

In addition, the conventional linear precoding (SDMA) for multigroup multicast is

revisited. Unlike RSMA, information intended for each group is encoded directly to a
single stream, i.e., Wm → sm , ∀m ∈ {1 · · · M }, rather than splitting into a common
part and private part. The symbol vector to be transmitted is s = [s1 , · · · , sM ]T ∈
CM ×1 , where E ssH = I . At the receiver side, each user decodes its desired stream


and treats all the interference streams as noise. Following the same multicast logic
as (3.7), the m-th group rate writes as

SDM A
rg,m = rm = min Ri . (3.9)
i∈Gm

Through the description above, RSMA is a more general scheme1 which encompasses
SDMA as a special case by allocating all transmit power to the private streams.

Remark 3.1: The encoding complexity and receiver complexity of RSMA are slightly
higher than SDMA. For one-layer RSMA in a M -group multigoup multicast MISO
BC, M +1 streams need to be encoded in contrast to M streams for SDMA. One-layer
RSMA requires one SIC at each user while SDMA does not require any SIC.

1
RSMA is also a more general framework that encompasses NOMA as a special case [5, 9, 17, 26].
Since NOMA leads to a waste of spatial resources and multiplexing gain/DoF (and therefore
rate loss) in multi-antenna settings at the additional expense of large receiver complexity, as
demonstrated extensively in [5, 9], we do not compare with NOMA in this work.
52 Chapter 3. RSMA for Multigroup Multicast and Multibeam Satellite Systems

3.2.2 CSIT Uncertainty and Scaling

Imperfect CSIT is considered in this work while the channel state information
at the receiver (CSIR) is assumed to be perfect2 . To model CSIT uncertainty,
 
channel matrix H is denoted as the sum of a channel estimate H b ≜ h b1, · · · , h
bK
 
and a CSIT error H e ≜ h e1, · · · , h
e K , i.e. H = H b + H. e CSIT uncertainty can


be characterized by a conditional density fH|H b H | H [16]. Taking each channel
b
e k 2 is allowed to decay
2

vector separately, the CSIT error variance σe,k ≜ Ehe k h
as O (P −αk ) [16, 21, 87, 88], where αk ∈ [0, ∞) is the scaling factor which quantifies
CSIT quality of the k-th user. Equal scaling factors among users are assumed for
simplicity in this model, i.e., αk = α. For a finite non-zero α, CSIT uncertainty
decays as P grows, (e.g., by increasing the number of feedback bits). In extreme
cases, α = 0 corresponds to a non-scaling CSIT, (e.g., with a fixed number of
feedback bits). α → ∞ represents perfect CSIT, (e.g., with infinite number of
feedback bits). The scaling factor is truncated such that α ∈ [0, 1] in this context
since α = 1 corresponds to perfect CSIT in the DoF sense [16, 21].

3.3 Max-Min Fair DoF Analysis

The MMF-DoFs of RSMA and SDMA are investigated in this section to characterize
the performance of both schemes. The MMF-DoF, also named MMF multiplexing
gain or symmetric multiplexing gain, corresponds to the maximum multiplexing gain
that can be simultaneously achieved across multicast groups. It reflects the pre-log
factor of MMF-rate at high SNR. The larger MMF-DoF is, the faster MMF-rate
increases with SNR. One would therefore like to use communication schemes with
the largest possible DoF. Motivated by mitigating interference at receivers, the
beamforming used in this section is sufficient from the DoF perspective since DoF
2
For imperfect CSIR, please see [15, 86].
3.3. Max-Min Fair DoF Analysis 53

can be roughly interpreted as the number of interference-free streams simultaneously

communicated in a single channel use [16, 26].

3.3.1 Max-Min Fair DoF of SDMA

Rk (P )
We start from SDMA, and define the k-th user-DoF as Dk ≜ limP →∞ log2 (P )
. The m-
SDM A (P )
rg,m
th group-DoF is given by dSDM
m
A
≜ limP →∞ log2 (P )
= mini∈Gm Di , and dSDM A ≜
minm∈M dSDM
m
A
is achieved by all groups. For a given beamforming matrix P =
[p1 , · · · pM ] ∈ CNt ×M , dSDM A represents the MMF-DoF. It interprets the maximum
fraction of an interference-free stream that can be simultaneously communicated
amongst groups. Since each user is equipped with only one antenna, we have

dSDM A ≤ dSDM
m
A
≤ Di ≤ 1, ∀ i ∈ Gm , m ∈ M. (3.10)

Proposition 3.1. The optimum MMF-DoF achieved by SDMA is given by





 α, Nt ≥ K − G1 + 1

α

d∗SDM A = , K − GM + 1 ≤ Nt < K − G1 + 1 (3.11)


 2

0, 1 ≤ Nt < K − GM + 1.


The achievability of Proposition 3.1 is discussed as follows by providing at least one

feasible beamforming matrix that achieves the DoF in (3.11). Next, results in (3.11)
are derived as tight upper-bounds from the converse, which completes the proof of
Proposition 3.1.

1) Achievability of Proposition 3.1:

To mitigate inter-group interference observed by each user, we aim to design the

b H pm = 0, ∀ m ∈ M, k ∈ K \ Gm . Define H
precoders such that h b m as the
k


b H , where
composite channel estimate of users in group-m, we have pm ∈ null H m
54 Chapter 3. RSMA for Multigroup Multicast and Multibeam Satellite Systems

b M ∈ CNt ×(K−Gm ) is a channel estimate matrix

 
H
bm ≜ Hb 1, · · · H b m+1 · · · , H
b m−1 , H

excluding H
b m . All the channel vectors are assumed to be independent. To satisfy



dim null Hb H ≥ 1, a minimum number of transmit antennas is required, as follows
m

Nt ≥ K − Gm + 1. (3.12)

(3.12) ensures sufficient Nt to place pm in the null space of its unintended groups.
Primary inter-group interference caused by the m-th precoder can be eliminated.
Without loss of generality, group sizes are assumed in ascending order: G1 ≤ G2 ≤
· · · ≤ GM . In an underloaded scenario, condition (3.12) has to hold for all m ∈ M,
and we rewrite it as
Nt ≥ K − G1 + 1. (3.13)

When (3.13) is satisfied, the system is underloaded. Considering equal power

P
allocation such that ∥p1 ∥2 = · · · = ∥pM ∥2 = M
, the received signal of user-k and
the scaling of received signal components are expressed by

O (P 1−α )
O(P ) z }| {
O (P 0 )
z }| { M
X
yk = hH eH z}|{
k p µ(k) s µ(k) +h k pj sj + nk . (3.14)
j=1,j̸=µ(k)

The second term is named as residual interference caused by imperfect CSIT. All the
b H PM
primary inter-group interference h k j=1,j̸=µ(k) pj sj has been eliminated. Since the

channel state does not depend on P , we have ∥hk ∥2 , ∥h

b k ∥2 = O (1). The residual

interference term scales as O (P 1−α ), with the CSIT error variance decaying as
O (P −α ). Note that when α = 1, the residual interference is reduced to the noise
level, and it corresponds to perfect CSIT from the DoF sense. When α ∈ [0, 1], γk
scales as O (P α ), from which Dk = α at each user. For all m ∈ M, dSDM
m
A
= α,
thus the MMF-DoF dSDM A = α. When Nt < K − G1 + 1, the system becomes
overloaded. If reducing the spatial dimensions to Nt < K − GM + 1, it is evident that
3.3. Max-Min Fair DoF Analysis 55

the inter-group interference caused by each precoder cannot be eliminated. Such

scenario is identified as fully-overloaded [26], and its MMF-DoF collapses to 0. Next,
we focus on the partially-overloaded in which K − GM + 1 ≤ Nt < K − G1 + 1. We
generally assume Nt = K − Gx + 1, where the group index x ∈ (1, M ]. Following the
logic of (3.12), primary inter-group interference caused by the [x, M ]-th group can


be nulled if the precoders are designed such that pm ∈ null H b H , ∀ m ∈ [x, M ]. In
m

addition, since Nt = K − Gx + 1 > (K − Gx ) − G1 + 1, the system excluding group-x

can be regarded as underloaded. Thus, we design pm ∈ null H b H , ∀ m ∈ M \ x to
m,x

remove inter-group interference among M\x. The beamforming directions described

above can be concluded as

b H , ∀ m ∈ [1, x)


null H

m,x
pm ∈ (3.15)
b H , ∀ m ∈ [x, M ] .
null H


m

An example of power allocation is

Pβ


 , ∀m ∈ M \ x
∥pm ∥2 = M − 1 (3.16)
P − P β , m ∈ x,



where β ∈ [0, 1] is a power partition factor. User-k’s received signal is given by


 O (P β−α )
 O (P β ) O (P 1−α )
z }| { O (P 0 )

 z }| {

 z }| { M
X
H eH e H px sx + z}|{

 hk pµ(k) sµ(k) + hk pj sj + h nk , ∀ k ∈ K \ Gx


 k
yk = j=1,j̸=µ(k),j̸=x

O (P β ) O (P β−α )

 O(P )
{ O (P 0 )



 z }| { z X }| { z X}|
 hH H eH

pi si + nk , ∀ k ∈ Gx .
z}|{
 k px sx + hk pj sj + h

 k
j∈[1,x) i∈(x,M ]

(3.17)

It is observed that Gx bear both residual interference and interference from groups
[1, x), while K \ Gx see only residual interference. γk at user k ∈ K \ Gx scales as
56 Chapter 3. RSMA for Multigroup Multicast and Multibeam Satellite Systems

O P β+α−1 , and γk at user k ∈ Gx scales as O P 1−β . Achieving max-min fair
DoF requires the same DoF amongst groups. By setting β = 1 − α2 , all users’ SINRs
α

scale as O P 2 . It turns out that dSDM
m
A
= α2 for all m ∈ M, and the MMF-DoF
α
dSDM A = 2
is achieved. Multiplexing gains are partially achieved. Importantly,
such partially-overloaded scenario does not exist when the group sizes are equal.

2) Converse of Proposition 3.1:

Proposition 3.1 is further shown as a tight upper-bound for any feasible SDMA
beamforming. Here, we generally assume the power allocation ∥p1 ∥2 , · · · , ∥pM ∥2
scale as O (P a1 ) , · · · , O (P aM ), where a1 , · · · , aM ∈ [0, 1] are power partition factors.
For each m ∈ M, Im ⊂ M is defined as a group set with precoding vectors
interfering with the m-th group, while Rm ⊂ M is defined as a group set with
precoding vectors that only cause residual interference to the m-th group. We define
am ≜ maxj∈Im aj , and am ≜ maxj∈Rm aj . Note that am = 0 for Im = ø, and am = 0
for Rm = ø. For each m ∈ M, there exists at least one user k ∈ Gm with SINR
+

n o
min (am −am )+ , (am −am +α)
scaling as O P , since the received signal can be generally
written as

O (P am ) O (P am −α)
O(P am )
z }| { z X }| { z X }| { O (P 0 )
yk = hH H eH z}|{
k pµ(k) sµ(k) + hk pj sj + h k pi si + nk . (3.18)
j∈Im i∈Rm

According to the definition, we obtain an upper-bound for the achievable group-DoF

n
+ o
+
dSDM
m
A
≤ min (am − am ) , am − am + α , (3.19)

where (·)+ ensures DoF non-negativity. The achievable MMF-DoF of SDMA satisfies
dSDM A ≤ dSDM
m
A
for all m ∈ M. Next, we aim to derive its tight upper-bound
d∗SDM A such that dSDM A ≤ d∗SDM A for any feasible SDMA-based beamforming in
different network load scenarios.
3.3. Max-Min Fair DoF Analysis 57

When the system is underloaded, it is obvious that Im = ø and Rm = M \ m for

all m ∈ M. Accordingly, we have am = 0 and am = maxj∈M\m aj . (3.19) can be
rewritten as
n
+ o
dSDM
m
A
≤ min a m , a m − max a j + α . (3.20)
j∈M\m

From (3.20), we assume am −maxj∈M\m aj +α > 0 because am −maxj∈M\m aj +α ≤ 0

limits d∗SDM A to 0. Then, (·)+ can be omitted. Since dSDM A is upper-bounded by
taking the average of any two group-DoFs, we have

SDM A dSDM
1
A
+ dSDM
2
A
d ≤ (3.21)
 2 
min a1 , a1 − maxj∈M\1 aj + α + min a2 , a2 − maxj∈M\2 aj + α
≤
2
(3.22)
a1 − maxj∈M\1 aj + α + a2 − maxj∈M\2 aj + α
≤ (3.23)
2
≤ α. (3.24)

(3.23) follows from the fact that point-wise minimum is upper-bounded by any element
in the set. (3.24) is obtained due to a1 ≤ maxj∈M\2 aj and a2 ≤ maxj∈M\1 aj .

Next, we focus on the partially-overloaded scenario. It is sufficient to show that

α
dSDM A ≤ 2
for Nt = K −G1 , as decreasing the number of antennas does not increase
DoF. Since Nt < K − G1 + 1, p1 leads to interference to at least one group. We
denote such group index as m1 . Thus, we have Im1 = 1 and Rm1 = M \ {1, m1 },
i.e., am1 = a1 and am1 = maxj∈M\{1,m1 } aj . Recalling (3.19), dSDM
m1
A
writes as

n
+ o
dSDM
m1
A
≤ min (am1 − a1 )+ , am1 − max aj + α . (3.25)
j∈M\{1,m1 }

For group-1, it is obvious that I1 = ø and R1 = M \ 1, i.e., a1 = 0 and a1 =

58 Chapter 3. RSMA for Multigroup Multicast and Multibeam Satellite Systems

maxj∈M\1 aj . Then, we have

n
+ o
dSDM
1
A
≤ min a 1 , a 1 − max a j + α . (3.26)
j∈M\1

By assuming am1 − a1 > 0 and a1 − maxj∈M\1 aj + α > 0, the group-DoF d∗SDM

m1
A

and d∗SDM
1
A
are not limited to 0. (·)+ can be omitted in both inequalities. Since
a1 − maxj∈M\1 aj + α > 0 leads to am1 − a1 < am1 − maxj∈M\{1,m1 } aj + α, (3.25)
can be rewritten as dSDM
m1
A
≤ am1 − a1 . Following the same logic as (3.21), dSDM A
is upper-bounded by taking the average of dSDM
1
A
and dSDM
m1
A

dSDM
1
A
+ dSDM
m1
A
dSDM A ≤ (3.27)
 2
min a1 , a1 − maxj∈M\1 aj + α + am1 − a1
≤ (3.28)
2
a1 − maxj∈M\1 aj + α + am1 − a1
≤ (3.29)
2
α
≤ . (3.30)
2

(3.29) is obtained because point-wise minimum is upper-bounded by any element in

the set. (3.30) is obtained due to am1 − maxj∈M\1 aj ≤ 0.

In a fully-overloaded scenario, it is sufficient to show that dSDM A is upper-bounded

by 0 for Nt = K − GM , as further decreasing Nt does not increase DoF. In this
case, we have Nt < K − Gm + 1 for all m ∈ M. Each pm causes interference to at
least one group. Here, we assume am2 = maxm∈M am . The index of group seeing
interference from pm2 is denoted by m3 . Thus, dSDM A is upper-bounded by

n
+ o
+
dSDM A ≤ dSDM
m3
A
≤ min (a m3 − a m2 ) , a m3 − am3 + α ≤ (am3 − am2 )+ = 0.
(3.31)
Combining the upper-bounds and achievability derived above, Proposition 3.1 is
proved. When α = 1, the results boil down to the Proposition 1 in [26] with perfect
CSIT.
3.3. Max-Min Fair DoF Analysis 59

Remark 3.2: The basic difference between perfect and imperfect CSIT scenarios while
analysing the DoF of SDMA is the existence of residual interference. For example,
when we consider perfect CSIT [26], Nt ≥ K − Gm + 1 ensures a sufficient number of
transmit antennas to place the m-th precoder in the null space of all of its unintended
groups. Inter-group interference caused by such precoder can be fully eliminated.
However, considering imperfect CSIT here, only primary inter-group interference
can be eliminated. At least one form of residual interference still exists.

From the above discussion, when the number of transmit antenna is greater than
K − G1 + 1, only residual interference will be seen by each user by controlling
the beamforming directions and power allocation. Otherwise, the system becomes
overloaded. Through beamforming and power control, the MMF-DoF does not collapse
to zero directly as in multi-user unicast or equal-group multigroup multicast systems.
When Nt drops below K − G1 + 1, M − 1 groups can be regarded as underloaded,
seeing only two forms of residual interference as given in the first equation of (3.17),
while the remaining one group’s received signal subspace is partially sacrificed. As a
α
result, an MMF-DoF of 2
is achieved through power control. When Nt drops below
K − GM + 1, each multicast group sees interference from all of its unintended groups.
The MMF-DoF drops to 0.

3.3.2 Max-Min Fair DoF of RSMA

RS (P )
rg,m
In RSMA scheme, the m-th group-DoF writes as dRS
m ≜ limP →∞ log2 (P )
= mini∈Gm Di +
Cm (P )
dc,m , where dc,m ≜ limP →∞ log2 (P )
is provided by the common rate portions. dRS ≜
minm∈M dRS
m is the MMF-DoF for a given beamforming matrix P = [pc , p1 , · · · pM ] ∈

CNt ×(M +1) . Thus, we have

dRS ≤ dRS
m ≤ Di + dc,m ≤ 1, ∀ i ∈ Gm , m ∈ M. (3.32)
60 Chapter 3. RSMA for Multigroup Multicast and Multibeam Satellite Systems

Proposition 3.2. The optimum MMF-DoF achieved by RSMA is given by


1−α

 +α, when Nt ≥ K − G1 + 1, 0 ≤ α ≤ 1


 M

1 1
d∗RS ≥ 1 + M − MR∗
, when 1 ≤ Nt < K − G1 + 1,
1 + M − MR∗
<α≤1 (3.33)


 ∗
α+ 1 − (1 + M − MR ) α , when 1

.

1 ≤ Nt < K − G1 + 1, 0 ≤ α ≤
M 1 + M − MR∗

Note that MR∗ is the maximum number of groups which can be regarded as un-
derloaded, and served by RSMA when the system is overloaded. The inequality
indicates that the results provided here are achievable, yet not necessarily optimum.

1) Achievability of Proposition 3.2:

bH ,
When the system is underloaded, i.e., Nt ≥ K − G1 + 1, we design pm ∈ null H m

which follows the same logic as SDMA. The direction of pc is chosen randomly.
Pδ
Consider the power allocation such that ∥p1 ∥2 = · · · = ∥pM ∥2 = M
, and ∥pc ∥2 =
P − P δ , where δ ∈ [0, 1] is a power partition factor. The received signal writes as

O (P δ−α )
O(P ) O (P δ )
O (P 0 )
z }| {
z }| { z }| { M
X
yk = hH H eH z}|{
p s
k c c + h k µ(k) µ(k) + hk
p s pj sj + nk . (3.34)
j=1,j̸=µ(k)

It can be observed that sc is firstly decoded at each user with SINR γc,k scaling as
O P 1−δ . The common stream can provide a DoF of 1 − δ. Since Rc = M

P
m=1 Cm ,
1−δ
sharing Rc equally amongst groups leads to max-min fairness, and dc,m = M
is
achieved by each group. After removing sc , each user then decodes sµ(k) with γk


scaling as O P min{α,δ} . For all k ∈ K, we have Dk = min {α, δ}. Therefore, the
1−δ
MMF-DoF dRS = minm∈M dRS
m = M
+ min {α, δ} can be achieved. By setting
1−α
δ = α, dRS reaches its maximum value at M
+ α.
3.3. Max-Min Fair DoF Analysis 61

Next, in overloaded scenarios, i.e., 1 ≤ Nt < K − G1 + 1, we consider a special

case of RSMA where groups are divided into two subsets, namely MR ⊆ M and
MC = M \ MR . Specifically, MR is a subset which can be treated as underloaded
and served by RSMA, while MC are the remaining groups and served by degraded
beamforming. Based on this mixed scheme, messages are split such that Wm →
{Wm,c , Wm,p } for all m ∈ MR , and Wm → {Wm,c } for all m ∈ MC . Such scheme
leads to ∥pm ∥2 = 0 for all m ∈ MC . The size of MR and MC are denoted by
MR = |MR | and MC = |MC | = M − MR respectively. To gain insight into the
subset partition, we define

 M
 X
K − G1 − Gj + 1, L ∈ {1, · · · , M − 1}


NL = j=L+1 (3.35)


K − G + 1, L = M.

1

According to (3.12), NL is the minimum number of transmitting antennas required

to regard groups {1, · · · , L} as underloaded while disregarding all the remaining
groups. Conversely, if Nt satisfies NL ≤ Nt < NL+1 , L is interpreted as the maximum
number of MR . We can define it as

M, Nt ≥ NM

MR∗ = (3.36)
L, NL ≤ Nt < NL+1 , ∀L ∈ {1, · · · , M − 1} .


 
H
For all m ∈ MR , beamforming directions are designed as pm ∈ null H m,M b .
{ C}

Pδ
pc ’s direction is set randomly. Consider the power allocation ∥pm ∥2 = M R
for all
m ∈ MR , and ∥pc ∥2 = P − P δ , where δ ∈ [0, 1].
62 Chapter 3. RSMA for Multigroup Multicast and Multibeam Satellite Systems

The received signal of user-k is written as


O (P δ ) O (P δ−α ) O (P 0 )

 O(P )

 z }| { z }| { z }|
X { z}|{
H H eH pj sj + σn2 ,




 hk pc sc + hk pµ(k) sµ(k) + h k ∀ k ∈ {Gm | m ∈ MR }

j∈MR \µ(k)
yk =
 O (P δ ) O (P 0 )

 O(P )

 z }| { z X }| { z}|{
H H
pj sj + σn2 , ∀ k ∈ {Gm | m ∈ MC } .

 h p s + hk
 k c c


j∈MR
(3.37)
Firstly, sc is decoded at each user by treating all the other streams as noise. γc,k is


observed to scale as O P 1−δ for k ∈ K. Thus, the common stream achieves a DoF
of 1 − δ. Since the common rate Rc = M
P
m=1 Cm is divided amongst MR and MC ,
P
we introduce a fraction z ∈ [0, 1] of the common rate such that m∈MR Cm = zRc ,
P z(1−δ)
and m∈MC Cm = (1 − z) Rc . This leads to dc,m = MR
for m ∈ MR and
(1−z)(1−δ)
dc,m = M −MR
for m ∈ MC . After removing sc through SIC, it can be seen that


γk scales as O P min{α,δ} in the first subset MR . Hence, we have Dk = min {α, δ}
for all k ∈ {Gm | m ∈ MR }. The group-DoF dRS
m is given by


z (1 − δ)

 + min {α, δ} , ∀ m ∈ MR
MR

dRS
m = (3.38)
(1 − z) (1 − δ)
, ∀ m ∈ MC .



M − MR

To achieve max-min fairness, equal group-DoFs between MR and MC are required.

On one hand, we assume δ ≥ α, and the equation can be written as

z (1 − δ) (1 − z) (1 − δ)
+α= . (3.39)
MR M − MR

Note that there are two variables δ and z on both sides of (3.39). Since the two
variables cannot be solved simultaneously, we fix one variable to maximize at least
one side of (3.39) while reserving the other variable on both sides. For example, let
3.3. Max-Min Fair DoF Analysis 63

δ = α in this case, and then calculate the remaining variable z according to

z (1 − α) (1 − z) (1 − α)
+α= . (3.40)
MR M − MR

[1−(1+M −MR )α]MR

z= (1−α)M
is obtained. Substitute it into arbitrary side of (3.40), and
1−(1+M −MR )α
the group-DoF dRS
m = α + M
for all m ∈ M is derived. Moreover, a
1
corresponding condition 0 ≤ α ≤ 1+M −MR
is obtained by considering 0 ≤ z =
[1−(1+M −MR )α]MR
(1−α)M
≤ 1. The MMF-DoF is achieved as dRS = minm∈M dRS
m = α +
1−(1+M −MR )
∗ α
1
M
, when 0 ≤ α ≤ ∗
1+M −MR
.

On the other hand, we assume δ < α. The equation in (3.39) is rewritten as

z (1 − δ) (1 − z) (1 − δ)
+δ = . (3.41)
MR M − MR

There are still two variables δ and z in (3.41). In this case, we can set z = 0 to
maximize the right side of (3.41) and calculate δ according to

1−δ
δ= . (3.42)
M − MR

1
By substituting the solution δ = 1+M −MR
into arbitrary side of (3.42), the group-DoF
1 1
dRS
m = 1+M −MR
for all m ∈ M is derived. Since δ = 1+M −MR
< α, we obtain the
1
corresponding condition 1+M −MR
< α ≤ 1 for this case.

Above all, the achievable MMF-DoF of RSMA is summarized in Proposition 3.2.

When α = 1, such result boils down to the achievability of Proposition 3 in [26]
with perfect CSIT. In overloaded scenarios, it is noteworthy that the dRS with
1
∗
1+M −MR
< α ≤ 1 is not a function of α and is the same as that achieved with perfect
1
CSIT. Thus, one can relax the CSIT quality up to ∗
1+M −MR
without affecting the
1
MMF-DoF. However, in the other case when 0 ≤ α ≤ ∗,
1+M −MR
dRS diminishes as
the CSIT quality reduces.
64 Chapter 3. RSMA for Multigroup Multicast and Multibeam Satellite Systems

2) Insight:

From (3.37), the interference seen by each user k ∈ {Gm | m ∈ MR } after SIC


scales as O P δ−α . As discussed above, we have two assumptions, namely δ ≥ α
and δ < α. When δ ≥ α, this residual interference cannot be ignored. By setting
the power partition factor δ → α, we can reduce it to the noise level and at the


same time increase γc,k which scales as O P 1−δ for all k ∈ K. To achieve max-min
fairness, the common rate factor z is then managed to obtain equal group-DoFs
1
among groups in MR and MC . 0 ≤ α ≤ ∗
1+M −MR
is derived as a corresponding
range of this case. The MMF-DoF reduces as α goes down. Otherwise, when δ < α,
such interference is always at the noise level. By setting z → 0, all the common rate
Rc contributes to Cm , for all m ∈ MC . The RSMA scheme used by MR boils down
to SDMA. Meanwhile, the group-DoFs of all m ∈ MC are maximized. Then, we
further manage the power partition factor δ to achieve max-min fairness amongst
1
all groups. ∗
1+M −MR
< α ≤ 1 is derived as the corresponding range. In this case,
changing α will no longer affect MMF-DoF because the interference seen by each
user k ∈ {Gm | m ∈ MR } after SIC is always at the noise level. The MMF-DoF
performance remains the same as that achieved with perfect CSIT. Such behaviour
is not observed in partially-overloaded SDMA. It can be observed in (3.17) that the
power of interference seen by each user k ∈ K \ Gx and k ∈ Gx scales as O (P 1−α )


and O P β respectively. α will always affect MMF-DoF as O (P 1−α ) cannot be
ignored unless considering perfect CSIT. To get more insight into the gains provided
by RSMA over SDMA, we substitute (3.36) into (3.33) and yield (3.43).

By comparing (3.43) with (3.11), we can see that the achievable MMF-DoF of RSMA
is always superior to d∗SDM A , and hence d∗RS ≥ d∗SDM A is guaranteed. The gain
1−α
of RSMA over SDMA is M
when the system is underloaded. Once Nt ≥ NM is
violated, the range of partially-overloaded SDMA K − GM + 1 ≤ Nt < K − G1 + 1,
(i.e., NM −1 + G1 ≤ Nt < NM ) locates within the range NM −1 ≤ Nt < NM . For any
3.3. Max-Min Fair DoF Analysis 65

1−α


 + α, when Nt ≥ NM


 M

 1 1
, when NM −1 ≤ Nt < NM , <α≤1


2 2



1 − 2α


 1

 α + , when NM −1 ≤ Nt < NM , 0 ≤ α ≤
M 2



.

..



d∗RS ≥ 1 1 (3.43)
 , when N2 ≤ Nt < N3 , <α≤1
M −1 M −1




α + 1 − (M − 1) α , 1


when N2 ≤ Nt < N3 , 0 ≤ α ≤




 M M −1
1 1



 ,
 when 1 ≤ Nt < N2 , <α≤1


 M M
1, 1


when 1 ≤ Nt < N2 , 0 ≤ α ≤

M M

0 ≤ α ≤ 1, the achievable MMF-DoF of RSMA is still greater than SDMA. Once Nt

drops below NM −1 +G1 , by taking Nt = NM −1 as an example, the number of transmit
antenna is not sufficient to eliminate any inter-group interference through SDMA
beamforming. d∗SDM A collapses to 0. For RSMA, d∗RS is kept by exploiting all the
MR∗ streams and transmitting the remaining stream through degraded beamforming.
This is carried on until RSMA reducing to a single-stream degraded beamforming. A
single DoF is split amongst all the groups. Therefore, d∗RS ≥ 1
M
> 0 is guaranteed.
For the particular case where all the group sizes are equal, (i.e., Gm = G, ∀m ∈ M),
there is no partially-overloaded scenario in (3.11). When Nt drops below K − G + 1,
d∗SDM A decreases from α to 0 directly. However, the expression of d∗RS remains the
1
same as (3.43), which is always greater than M
.

Remark 3.3: The obtained MMF-DoFs of different strategies are listed in Table 3.1,
where the first row represents underloaded and the others are the results of overloaded
systems. From the above discussion, the MMF-DoF analysis in the underloaded
regime is similar when considering RSMA and SDMA. Each user sees only residual
interference by managing the beamforming directions and power allocation. A gain
1−α
of M
is obtained by applying RSMA. Thus, we can conclude that in the presence of
66 Chapter 3. RSMA for Multigroup Multicast and Multibeam Satellite Systems

imperfect CSIT, there is an MMF-DoF gain of RSMA over SDMA when the system
is underloaded. This contrasts with perfect CSIT scenarios where both underloaded
SDMA and RSMA can achieve full MMF-DoF of 1. Overloaded RSMA is more
challenging since both residual interference and group partitioning method should be
considered. [26] considers a special case where the groups are partitioned into two
subsets, namely MD ⊆ M which are served using SDMA, and MC ⊆ M \ MD
served by degraded beamforming. The number of groups in MD is set as the maximum
number of groups that can be served by interference-free SDMA (i.e., achieving a
group-DoF of 1 each). However, in this work considering imperfect CSIT, SDMA
can no longer reach an MMF-DoF of 1. As shown in Table 3.1, the maximum
achievable MMF-DoF is α when the system is underloaded, while RSMA outperforms
SDMA slightly. Thus, we consider a different subset partitioning in this work where
the groups are divided into MR ⊆ M and MC ⊆ M \ MR . The number of groups
in MR is chosen as the maximum number of groups which can be served by RSMA
1−α
and achieve an MMF-DoF of M
+ α. MC is still served by degraded beamforming.
Accordingly, from the results summarised in Table 3.1, RSMA is shown to provide
MMF-DoF gains and outperform SDMA in overloaded systems.

All the discussions above motivate the use of RSMA from a DoF perspective.
However, DoF is an asymptotically high SNR metric. It remains to be seen whether
the DoF gain translates into rate gain. To that end, the design of RSMA for rate
maximization at finite SNR needs to be investigated. Beamforming schemes that
achieve Proposition 3.1 and Proposition 3.2 are not necessarily optimum from an
MMF-rate sense. Therefore, the beamforming directions, power allocation and rate
partition can be elaborated by formulating MMF-rate optimization problems as
we see in the next section. Importantly, the DoF analysis provides fundamental
grounds, helps to draw insights into the performance limits of various strategies and
guides the design of efficient strategies (rate-splitting in this case).
3.3. Max-Min Fair DoF Analysis 67

Table 3.1: Achievable MMF-DoF of different strategies

Perfect CSIT [16] Imperfect CSIT [this chapter]

Strategy SDMA RSMA SDMA RSMA

1−α
Nt ≥ NM 1 1 α M
+α

1 1

 , <α≤1

NM −1 + G1 ≤ Nt < NM 1 1 1 α 2 2
α + 1 − 2α , 0 ≤ α ≤ 1
2 2 2 
M 2

1 1

 , <α≤1

NM −1 ≤ Nt < NM −1 + G1 0 1
0 2 2
α + 1 − 2α , 0 ≤ α ≤ 1
2 
M 2

1 1

 , <α≤1

NM −2 ≤ Nt < NM −1 0 1
0 3 3
α + 1 − 3α , 0 ≤ α ≤ 1
3 
M 3
.. ..
. .
1 1
1 ≤ Nt < N2 0 M
0 M

1
The second line of this table (partially-overloaded scenario) does not exist when the
group sizes are equal. When Nt drops below K − G + 1, d∗SDM A decreases to 0 directly.
68 Chapter 3. RSMA for Multigroup Multicast and Multibeam Satellite Systems

3.4 Max-Min Fair Problem Formulation

Now, we formulate an optimization problem to design precoders to achieve MMF

among multiple co-channel multicast groups subject to a flexible power constraint
with imperfect CSIT. MMF Ergodic rate is the metric for both RSMA and SDMA.
It reflects long-term MMF rate performance over varying channel states. Given
a long sequence of channel estimates, the MMF Ergodic rate can be measured
by updating precoders based on each short-term MMF Average rate. For a given
channel estimate H,
b the Average rate is defined as the expected performance over

CSIT error distribution. The Average rates for the common and private streams of
user-k are short-term measures given by


Rc,k (H) b |H
b = E b Rc,k (H, H) b , (3.44)
H|H

Rk (H) b |H
b = E b Rk (H, H) b . (3.45)
H|H

Note that Average rates should not be confused with Ergodic rates. Ergodic rates
capture the long-term performance over all channel states, while Average rates
measure the short-term expected performance over CSIT error distribution for a
given channel state estimate. According to the law of total expectation and the
definition of Average rate, the Ergodic rates for the common and private streams of
user-k are expressed by

   
E{H,H} Rc,k (H, H) b |H
b = E b E b Rc,k (H, H) b = EH
b Rc,k (H) ,
b (3.46)
b H {H|H}
   
E{H,H} Rk (H, H) b |H
b = E b E b Rk (H, H) b b Rk (H) .
= EH b (3.47)
b H {H|H}

It turns out that measuring Ergodic rates is transformed into measuring Average
rates over the variation of H.
b Therefore, the MMF Ergodic rate maximization

problem is decomposed to an MMF Average rate maximization problem for each H.

b

b {FRS }, where FRS is

The MMF Ergodic rate of RSMA can be characterized by EH
3.4. Max-Min Fair Problem Formulation 69

the MMF Average rate maximization problem for a given channel estimate H.
b

FRS : max min C m + min Ri (3.48)
c,P m∈M i∈Gm
M
X
s.t. Rc,k ≥ C m, ∀k ∈ K (3.49)
m=1

C m ≥ 0, ∀m ∈ M (3.50)
M
X
pH
c Dl pc + pH
m Dl pm ≤ Pl , l = 1···L (3.51)
m=1

 
The average common rate vector c = C 1 , · · · , C M and the beamforming matrix
P = [pc , p1 , · · · pM ] are jointly optimized to achieve MMF performance. Since the
average common rate is defined by Rc = M
P
m=1 C m = mink∈K Rc,k , we use constraint

(3.49) to ensure that the common stream sc is decoded by each user. Constraint
(3.50) implies that each portion of Rc is non-negative and (3.51) is the transmit
power constraint.

Similarly, the corresponding SDMA-based MMF Average rate maximization problem

is formulated as

FSDM A : max min min Ri (3.52)
P m∈M i∈Gm
M
X
s.t. pH
m Dl pm ≤ Pl , l = 1···L (3.53)
m=1

where the beamforming matrix P = [p1 , · · · , pM ] is optimized to solve FSDM A . (3.53)

is the transmit power constraint. SDMA is a sub-scheme of RSMA by switching off
(i.e., allocating zero power to) the common stream. Solving FSDM A is a special case
of FRS by fixing c = 0 and ∥pc ∥2 = 0. We will focus on solving the RSMA-based
problem in the following discussion.

Sample average approximation (SAA) is then adopted to convert FRS into a

(S)
deterministic problem denoted by FRS . For a given channel estimate H
b and
70 Chapter 3. RSMA for Multigroup Multicast and Multibeam Satellite Systems

sample index set S ≜ {1, · · · , S}, we construct S channel samples denoted as


H(S) ≜ H(s) = Hb +He (s) | H,
b s ∈ S containing S i.i.d realizations drawn from a

b (H | H). These realizations are available

conditional distribution with density fH|H b

at the transmitter and can be used to approximate the Average rates experienced
by each user through sample average functions (SAFs). When S → ∞, according to
the strong law of large numbers, we have

S
(S) 1X
Rc,k = lim Rc,k = lim Rc,k (H(s) ), (3.54)
S→∞ S→∞ S
s=1
S
(S) 1X
Rk = lim Rk = lim Rk (H(s) ), (3.55)
S→∞ S→∞ S
s=1

where Rc,k (H(s) ) and Rk (H(s) ), s ∈ S are the common and private rates associated
with the s-th channel realization. Accordingly, the SAA problem can be written as

(S) (S)

FRS : max min C m + min Ri (3.56)
c,P m∈M i∈Gm
M
(S) X
s.t. Rc,k ≥ C m, ∀k ∈ K (3.57)
m=1

C m ≥ 0, ∀m ∈ M (3.58)
M
X
pH
c Dl pc + pH
m Dl pm ≤ Pl , l = 1···L (3.59)
m=1

(S)
Note that FRS is a non-convex optimization problem which is challenging to solve.
Next, we turn to solve the SAA problem using the WMMSE approach.

3.5 The WMMSE approach

The WMMSE approach, initially proposed in [48], is effective in solving problems

containing non-convex superimposed rate expressions, i.e., RSMA-based sum rate
maximization problems [16]. In this section, we further modify this approach so as
3.5. The WMMSE approach 71

to solve the formulated SAA problem to achieve max-min fairness in RSMA-based

multigroup multicast with imperfect CSIT. To begin with, the relationship between
rate and WMMSE is derived, enabling the formulation of the equivalent problem.

3.5.1 Rate-WMMSE Relationship

At the k-th user, we denote the estimate of the common stream sc by sbc,k = gc,k yk ,
where gc,k is a scalar equalizer. After sc is successfully decoded by all receivers and
removed from the received signal yk , the estimate of sµ(k) is obtained at user-k such
that sbµ(k) = gk (yk − hH
k pc sc ), where gk is the corresponding equalizer.

sc,k − sc,k |2 } and εk =

The common and private MSEs are defined as εc,k = E{|b
sµ(k) − sµ(k) |2 }. The expectations are taken over the distributions of the input
E{|b
signals and the noise. By substituting the signal expressions into the definitions, the
MSEs can be expressed by

εc,k = |gc,k |2 Tc,k − 2R{gc,k hH

k pc } + 1, (3.60)

εk = |gk |2 Tk − 2R{gk hH
k pµ(k) } + 1, (3.61)

where the k-th user’s received power is given by Tc,k = |hH 2 H 2

k pc | + |hk pµ(k) | +
PM H 2 2
j=1,j̸=µ(k) |hk pj | + σn . The power of observation after SIC writes as Tk =

Tc,k − |hH 2
k pc | . Furthermore, we define Ic,k as the interference plus noise portion in

Tc,k which is equal to Tk , and define Ik = Tk − |hH 2

k pµ(k) | as the interference plus
∂εc,k
noise portion in Tk . To minimize the MSEs over equalizers, we let ∂gc,k
= 0 and
∂εk
∂gk
= 0. This yields the optimum equalizers given by

M M SE −1 −1
gc,k = pH
c hk Tc,k and gkM M SE = pH
µ(k) hk Tk . (3.62)

By substituting (3.62) into (3.60) and (3.61), the MMSEs with optimum equalizers,
72 Chapter 3. RSMA for Multigroup Multicast and Multibeam Satellite Systems

i.e., the well-known MMSE equalizers are given by

−1
εM
c,k
M SE
= min εc,k = Tc,k Ic,k , (3.63)
gc,k

εM
k
M SE
= min εk = Tk−1 Ik . (3.64)
gk

It is evident that the SINRs can be expressed in the form of MMSEs such that

γc,k = (1/εM
c,k
M SE
) − 1 and γk = (1/εM
k
M SE
) − 1. (3.65)

The corresponding rate expressions write as

Rc,k = − log2 (εM

c,k
M SE
) and Rk = − log2 (εM
k
M SE
). (3.66)

Next, we introduce the augmented WMSEs from which the Rate-WMMSE rela-
tionship is derived. The common and private augmented WMSEs of user-k are
respectively defined as

ξc,k = uc,k εc,k − log2 (uc,k ) and ξk = uk εk − log2 (uk ), (3.67)

where uc,k and uk denote auxiliary positive weights. By substituting optimum

equalizers to WMSEs, we obtain

M M SE
ξc,k (gc,k ) = min ξc,k = uc,k εM
c,k
M SE
− log2 (uc,k ), (3.68)
gc,k

ξk (gkM M SE ) = min ξc,k = uk εM

k
M SE
− log2 (uk ). (3.69)
gk

∂ξc,k (gc,k
M M SE
) ∂ξk (gkM M SE )
Moreover, let ∂uc,k
= 0 and ∂uk
k
= 0 to minimize the WMSEs over
both equalizers and weights. This yields the optimum MMSE weights

M SE −1 M SE −1
uM
c,k
M SE
= (εM
c,k ) and uM
k
M SE
= (εM
k ) . (3.70)
3.5. The WMMSE approach 73

We substitute (3.70) into (3.68), (3.69), hence leading to the Rate-WMMSEs rela-
tionship
M M SE
ξc,k = min ξc,k = 1 + log2 (εM
c,k
M SE
) = 1 − Rc,k , (3.71)
gc,k ,uc,k

M M SE
ξc,k = min ξc,k = 1 + log2 (εM
k
M SE
) = 1 − Rk . (3.72)
gc,k ,uc,k

With respect to imperfect CSIT, a deterministic SAF version of the Rate-WMMSE

relationship is constructed such that

M M SE(S) (S) (S)

ξ c,k = min ξ c,k = 1 − Rc,k , (3.73)
gc,k ,uc,k

M M SE(S) (S) (S)

ξk = min ξ k = 1 − Rk . (3.74)
gk ,uk

This relationship holds for the whole set of stationary points [16]. For a given channel
M M SE(S) M M SE(S)
estimate, ξ c,k and ξ k represent the Average WMMSEs. We have
M M SE(S) PS M M SE(s) M M SE(S) M M SE(s) M M SE(s)
1
= S1 Ss=1 ξk
P
ξ c,k = S s=1 ξc,k and ξ k , where ξc,k
M M SE(s)
and ξk are associated with the s-th realization in H(S) . The sets of optimum
M M SE M M SE(s) M M SE(s)
equalizers are defined as gc,k = {gc,k | s ∈ S} and gkM M SE = {gk |
s ∈ S}. Following the same manner, the sets of optimum weights are uM
c,k
M SE
=
M M SE(s) M M SE(s)
{uc,k | s ∈ S} and uM
k
M SE
= {uk | s ∈ S}. Each optimum element in
these sets is associated with the s-th realization in H(S) . From the perspective of
each user, the composite optimum equalizers and composite optimum weights are
respectively
GM M SE = gc,k
 M M SE M M SE
, gk |k∈K , (3.75)

UM M SE = uM
 M SE M M SE
c,k , uk |k∈K . (3.76)

Note that the WMSEs are convex in each of their corresponding variables (e.g.,
equalizers, weights or precoding matrix) when fixing the other two. This block-
wise convexity, preserved under superimposed expressions, together with the Rate-
WMMSE relationship is the key to WMMSE approach [26]. Now, we can transform
74 Chapter 3. RSMA for Multigroup Multicast and Multibeam Satellite Systems

(S)
FRS into an equivalent WWMSE problem.

(S)
WRS : max rg (3.77)
c,P,G,U,rg ,r

s.t. C m + rm ≥ rg , ∀m ∈ M (3.78)
(S)
1 − ξi ≥ rm , ∀i ∈ Gm , ∀m ∈ M (3.79)
M
(S) X
1− ξ c,k ≥ C m, ∀k ∈ K (3.80)
m=1

C m ≥ 0, ∀m ∈ M (3.81)
M
X
pH
c Dl pc + pH
m Dl pm ≤ Pl , l = 1···L (3.82)
m=1

where rg and r = [r1 , · · · , rM ] are auxiliary variables. Furthermore, if the solution

(S)
(P∗ , G∗ , U∗ , r∗g , r∗ , c∗ ) satisfies the KKT optimality conditions of WRS , (P∗ , c∗ ) will
(S)
satisfy the KKT optimality conditions of FRS (P ). Considering the block-wise
convexity property, we use an Alternating Optimization (AO) algorithm illustrated
(S)
below to solve WRS .

3.5.2 Alternating Optimization Algorithm

Each iteration of the AO algorithm is composed of two steps.

1) Updating G and U:

During the n-th iteration, all the equalizers and weights are updated accord-


ing to a given beamforming matrix such that G = GM M SE P[n−1] and U =


UM M SE P[n−1] , where P[n−1] is the given beamforming matrix obtained from the
previous iteration. To facilitate the P updating problem in the next step, we intro-
duce several expressions calculated by updated G and U [16] to express the Average
3.5. The WMMSE approach 75

WMSEs.

(s) (s) (s) 2 (s) (s) (s) 2

tc,k = uc,k gc,k and tk = uk gc,k (3.83)
(s) (s) (s) (s)H (s) (s) (s) (s)H
Ψc,k = tc,k hk hk and Ψk = tk hk hk (3.84)
(s) (s) (s) (s)H (s) (s) (s) (s)H
fc,k = uc,k hk gc,k and fk = uk hk gk (3.85)
(s) (s)
(s) (s)

vc,k = log2 uc,k and vk = log2 uk . (3.86)

Therefore, by taking the averages over S realizations, the corresponding SAFs are
(S) (S) (S) (S) (S) (S) (S) (S)
tc,k , tk , Ψc,k , Ψk , f c,k , f k , v c,k , v k , from which leads to the Average WMSEs
coupled with updated G and U.

M
(S) (S) X (S) 2 (S)
 (S)H (S) (S)
ξ c,k = pH
c Ψc,k pc + pH
m Ψc,k pm + σn tc,k − 2R f c,k pc + uc,k − v c,k , (3.87)
m=1
M
(S) X (S) 2 (S)
 (S)H (S) (S)
ξk = pH
m Ψk pm + σn tk − 2R f k pµ(k) + uk − v k . (3.88)
m=1

2) Updating P:

In this step, we fix G, U, and update P together with all the auxiliary variables.
By substituting the Average WMSEs coupled with updated G and U into W, the
(S)[n]
problem of updating P based on updated G and U is formulated in W . This is
a convex optimization problem which can be solved using interior-point methods.
The steps are summarized in Algorithm 1.

(S)[n]
WRS max rg (3.89)
c,P,rg ,r

s.t. C m + rm ≥ rg , ∀m ∈ M (3.90)
M
X (S) (S)
1 − rm ≥ pH 2
m Ψk pm + σn tk
m=1
 (S)H (S) (S)
− 2R f k pµ(k) + uk − v k , ∀i ∈ Gm , ∀m ∈ M (3.91)
76 Chapter 3. RSMA for Multigroup Multicast and Multibeam Satellite Systems

M M
X (S) X (S)
1− C m ≥ pH
c Ψc,k pc + pH
m Ψc,k pm
m=1 m=1
(S)  (S)H (S) (S)
+ σn2 tc,k − 2R f c,k pc + uc,k − v c,k , ∀k ∈ K (3.92)

C m ≥ 0, ∀m ∈ M (3.93)
M
X
pH
c Dl pc + pH
m Dl pm ≤ Pl , l = 1···L (3.94)
m=1

Through the AO algorithm, variables in the equivalent WMMSE problem are

optimized iteratively in an alternating manner. The proposed algorithm is guaranteed
to converge as the objective function is bounded above for the given power constraints.
The objective function rg increases until convergence as the iteration process goes
on.

Algorithm 1 Alternating Optimization

(S)[n]
Initialize: n ← 0, P, WRS ←0
(S)[n] (S)[n−1]
while WRS − WRS < ε do
[n−1]
n ← n + 1, P ←
P
M M SE [n−1]
G←G P
(3.75)
U ← UM M SE P[n−1] (3.76)
(S) (S) (S) (S) (S) (S) (S) (S) (S) (S)
update tc,k , tk , Ψc,k , Ψk , f c,k , f k , v c,k , v k , uc,k , uc,k (3.83)-(3.86)
(S)[n]
P ← Solve arg WRS
end while

3.6 Simulation Results and Analysis

In this section, the performance of the proposed RSMA-assisted multigroup multicast

beamforming strategy is evaluated through simulation results by considering the
scenarios of both Rayleigh fading channels (representative of cellular terrestrial
systems) and multibeam satellite systems. Additionally, we evaluate the throughput
performance by link-level simulations.
3.6. Simulation Results and Analysis 77

3.6.1 Performance Over Rayleigh Channels

The performance of RSMA and SDMA are both evaluated over Rayleigh fading
channels (representative of conventional cellular terrestrial systems) when considering
a total transmit power constraint. During simulation, entries of H are independently
drawn from CN (0, 1). Following the CSIT uncertainty model, entries of H
e are also

i.i.d complex Gaussian drawn from CN (0, σe2 ), where σe2 = Nt−1 σe,k
2
= P −α . Herein,
we evaluate the MMF Ergodic rate by averaging over 100 channel estimates. For
b = H − H,
each given channel estimate H e its corresponding MMF Average rate is

approximated by SAA method and the sample size S is set to be 1000. H(S) is the
set of conditional realizations available at the transmitter. The s-th conditional
realization in H(S) is given by H(s) = H e (s) , where H
b +H e (s) follows the above CSIT

error distribution.

We firstly consider an underloaded system with Nt = 6 transmit antennas, G = 3

groups and K = 6 users. The group sizes are respectively G1 = 1, G2 = 2, G3 = 3.
Fig. 3.1 presents the MMF Ergodic rate of RSMA and SDMA versus an increasing
SNR under various CSIT qualities. For perfect CSIT, beaming an interference-free
stream to each group simultaneously is possible since the system is underloaded.
Both RSMA and SDMA achieve full MMF-DoF and the performance of such two
schemes are nearly identical. However, RSMA shows a little improvement in the
rate sense compared with SDMA due to its more flexible architecture. For imperfect
CSIT, the superiority of RSMA over SDMA becomes more evident. It can be
observed in Fig. 3.1 that the MMF-DoF disparity between RSMA and SDMA
gradually appears as the CSIT uncertainty increases. The MMF-DoFs of SDMA
1−α
and RSMA in Fig. 3.1 are respectively α and M
+ α, which follow the results in
Table 3.1. This implies that the common stream of RSMA can provide a DoF gain
1−α
of M
and consequently MMF rate gains in underloaded regimes.

In Fig. 3.2, we reduce the number of transmit antennas to 4 and the system becomes
78 Chapter 3. RSMA for Multigroup Multicast and Multibeam Satellite Systems

11
RSMA, perfect
10 SDMA, perfect
RSMA, imperfect 0.8
9
SDMA, imperfect 0.8
8 RSMA, imperfect 0.6
SDMA, imperfect 0.6
MMF Rate (bps/Hz)

7 RSMA, imperfect 0.4

SDMA, imperfect 0.4
6 RSMA, imperfect 0.2
SDMA, imperfect 0.2
5

0
5 10 15 20 25 30 35
SNR(dB)

Figure 3.1: MMF rate performance. Nt = 6 antennas, K = 6 users, M = 3 groups,

G1 , G2 , G3 = 1, 2, 3 users.

7
RSMA, perfect
SDMA, perfect
6 RSMA, imperfect 0.8
SDMA, imperfect 0.8
RSMA, imperfect 0.6
5 SDMA, imperfect 0.6
MMF Rate (bps/Hz)

RSMA, imperfect 0.4

SDMA, imperfect 0.4
4 RSMA, imperfect 0.2
SDMA, imperfect 0.2

0
5 10 15 20 25 30 35
SNR(dB)

Figure 3.2: MMF rate performance. Nt = 4 antennas, K = 6 users, M = 3 groups,

G1 , G2 , G3 = 1, 2, 3 users.
3.6. Simulation Results and Analysis 79

partially-overloaded (K − G3 + 1 ≤ Nt < K − G1 + 1). When considering perfect

CSIT, RSMA and SDMA achieve identical MMF-DoFs at 12 . It follows the perfect
CSIT results in Table 3.1. Meanwhile, it also follows the results of imperfect CSIT by
setting α = 1. Multiplexing gains are partially achieved. A small rate gap between
the two schemes is observed although their MMF-DoFs are equal. Next, it comes to
imperfect CSIT. We can see that the merit of RSMA over SDMA becomes more
obvious compared with the underloaded regime. From Fig. 3.2, the MMF-DoFs
α
of SDMA (blue curves) are approximately 2
, which match the theoretical result
in (3.11). CSIT imperfectness can affect the system’s performance significantly.
Considering RSMA, we have MR∗ = 2 as a result of N2 ≤ Nt < N3 in this specific
setup. Substituting MR∗ = 2 and M = 3 into (3.33) or the overloaded results in
Table 3.1, we obtain

1

 , 0.5 < α ≤ 1

d∗RS ≥ 2 (3.95)
α + 1 − 2α , 0 ≤ α ≤ 0.5.

3

In addition, we have d∗SDM A = α2 . Such DoF performance is exhibited in Fig. 3.2.

All simulation results are inline with the theoretical MMF-DoFs in Table 3.1. Due
1
to the benefits of RSMA, the system is able to maintain its MMF-DoFs at 2
for all
0.5 < α ≤ 1 in this example. When 0 ≤ α ≤ 0.5, the MMF-DoFs decrease slightly
1−2α α
to α + 3
, which is still greater than the 2
achieved by SDMA. Compared with
the underloaded scenario in Fig. 3.1, the gaps between RSMA (red curves) and
SDMA (blue curves) increase. In other words, the superiority of RSMA over SDMA
becomes more apparent when the system is partially-overloaded.

Furthermore, we keep the same setting as in Fig. 3.2 but change the group sizes to
be symmetric, i.e., G1 = 2, G2 = 2, G3 = 2. It is noted that the system at present
becomes fully-overloaded (1 ≤ Nt < K − G3 + 1). As illustrated in Fig. 3.3, RSMA
outperforms SDMA to a great extent in both perfect CSIT and imperfect CSIT
80 Chapter 3. RSMA for Multigroup Multicast and Multibeam Satellite Systems

7
RSMA, perfect
SDMA, perfect
6 RSMA, imperfect 0.8
SDMA, imperfect 0.8
RSMA, imperfect 0.6
5
MMF Rate (bps/Hz) SDMA, imperfect 0.6
RSMA, imperfect 0.4
SDMA, imperfect 0.4
4
RSMA, imperfect 0.2
SDMA, imperfect 0.2
3

0
5 10 15 20 25 30 35
SNR(dB)

Figure 3.3: MMF rate performance. Nt = 4 antennas, K = 6 users, M = 3 groups,

G1 , G2 , G3 = 2, 2, 2 users.

scenarios. RSMA maintains the same MMF-DoFs as in Fig. 3.2. However, all the
multiplexing gains of SDMA are sacrificed and collapse to 0. The corresponding
MMF rate performance of SDMA gradually saturates as SNR grows, thus resulting
in severe rate limitation.

Through the simulation results over Rayleigh fading channels, it is demonstrated

that RSMA-based multigroup multicast beamforming is more robust to CSIT
imperfectness than the conventional SDMA scheme. RSMA is able to further exploit
spatial multiplexing gains and achieve higher MMF rate performance in various
setups. In particular, RSMA provides significant gains compared with SDMA in
overloaded regimes with imperfect CSIT.

Above all, the gains of RSMA for multigroup multicast in the presence of imperfect
CSIT are shown via simulations in both underloaded and overloaded deployments.
This contrasts with [26] where gains in the presence of perfect CSIT were demon-
strated primarily in the overloaded scenarios.
3.6. Simulation Results and Analysis 81

Figure 3.4: Architecture of multibeam satellite systems.

3.6.2 Application to Multibeam Satellite Systems

In order to show the versatility of RSMA, the application of RSMA-based multigroup

multicast beamforming to multibeam satellite systems is addressed in this section.
Here, we focus on a Ka-band multibeam satellite system with multiple single-antenna
terrestrial users served by a geostationary orbit (GEO) satellite as shown in Fig.
3.4. A single gateway is employed in this system, and the feeder link between the
gateway and the satellite is assumed to be noiseless. Let Nt denote the number
of antenna feeds. The array fed reflector can transform Nt feed signals into M
transmitted signals (i.e., one signal per beam) to be radiated over the multibeam
coverage area [89]. Considering single feed per beam architecture which is popular
in modern satellites such as Eutelsat Ka-Sat [34, 36], only one feed is required to
generate one beam (i.e., Nt = M ). Since the multibeam satellite system is in practice
user overloaded, we assume that ρ (ρ > 1) users are served simultaneously by each
beam. Users per beam are uniformly distributed within the satellite coverage area.
Ideally, the user selection and beamforming can be jointly designed. However, this
is out of the scope of this thesis and can be explored in future work. K = ρNt is
the total number of users.
82 Chapter 3. RSMA for Multigroup Multicast and Multibeam Satellite Systems

Table 3.2: Simulation parameters [Chapter 3]

Parameter Value
Frequency band (carrier frequency) Ka (20 GHz)
Satellite height 35786 km (GEO)
User link bandwidth 500 MHz
3 dB angle 0.4◦
Maximum beam gain 52 dBi
User terminal antenna gain 41.7 dBi
System noise temperature 517 K

1) Multibeam Satellite Channel :

The main difference between satellite and terrestrial communications lies in the
channel characteristics including free space loss, radiation pattern and atmospheric
fading. The satellite channel H ∈ CNt ×K is a matrix composed of receive antenna
gain, free space loss and satellite multibeam antenna gain. Its (n, k)-th entry can be
modeled as p
GR Gn,k
Hn,k = (3.96)
4π dλk
p
κTsys Bw

where GR is the user terminal antenna gain, dk is the distance between user-k and
the satellite, λ is the carrier wavelength, κ is the Boltzmann constant, Tsys is the
receiving system noise temperature and Bw denotes the user link bandwidth. Gn,k is
the multibeam antenna gain from the n-th feed to the k-th user. It mainly depends
on the satellite antenna radiation pattern and user locations.

2) Performance Over Satellite Channels:

Then, we evaluate the application of RSMA in multibeam satellite communications.

Results of MMF problems are obtained by averaging 100 satellite channel realizations.
Since non-flexible on-board payloads prevent power sharing between beams, per-feed
power constraints are adopted. System parameters are summarized in Table 3.2.
Fig. 3.5 shows the curves of MMF rates among Nt = 7 beams versus an increasing
per-feed available transmit power. We assume two users per beam, i.e., ρ = 2.
3.6. Simulation Results and Analysis 83

4 RSMA, perfect SDMA, imperfect 0.6

SDMA, perfect 4-color perfect
RSMA, imperfect 0.8 4-color 0.8
3.5
SDMA, imperfect 0.8 4-color 0.6
RSMA, imperfect 0.6
3
MMF Rate(bps/Hz)

2.5

1.5

0.5
20 40 60 80 100 120
Per antenna feed power(W)

Figure 3.5: MMF rate versus per-feed available power. Nt = 7 antennas, K = 14

users, ρ = 2 users.

For perfect CSIT, RSMA achieves around 25% gains over SDMA. For imperfect
CSIT, RSMA is seen to outperform SDMA with 31% and 44% gains respectively
when α = 0.8 and α = 0.6. Accordingly, the advantage of employing RSMA in
multigroup multicast beamforming is still observed in multibeam satellite systems.
Through partially decoding the interference and partially treating the interference
as noise, RSMA is more robust to the CSIT uncertainty and overloaded regime than
SDMA. Such benefit of RSMA exactly tackles the challenges of multibeam satellite
communications. The conventional 4-colour scheme performs the worst compared
with full frequency reuse schemes.

Here, we set the per-feed available transmit power to be 80 Watts. As CSIT error
scaling factor drops, the MMF rate gap between RSMA and SDMA increases
gradually, which implies the gains of our proposed RSMA scheme become more
and more apparent as the CSIT quality decreases. In addition, the impact of user
number per frame is also studied. Since all the users within a beam share the same
beamforming vector, the beam rate is determined by the user with the lowest SINR.
84 Chapter 3. RSMA for Multigroup Multicast and Multibeam Satellite Systems

RSMA, rho=2
SDMA, rho=2
2.5 RSMA, rho=4
SDMA, rho=4
RSMA, rho=6
MMF Rate (bps/Hz) 2 SDMA, rho=6

1.5

0.5

0
perfect 0.8 0.6 0.4 0.2
CSIT error scalling factor

Figure 3.6: MMF rate versus CSIT error scalling factor α. Nt = 7 antennas,
ρ = 2, 4, 6 users, P/Nt = 80 W.

Considering ρ = 2, 4, 6 users per frame, it is clear that increasing the number of

users per frame results in system performance degradation for both RSMA and
SDMA.

Moreover, the impact of different transmit power constraints is studied. Based on the
fair per-antenna power constraint assumption, each transmit antenna cannot radiate
a power more than P/Nt . Compared with the total transmit power constraint, the
existence of per-antenna power constraint will inevitably restrict the flexibility of
beamforming design. Taking imperfect CSIT with α = 0.8 as an example, Fig. 3.7
respectively shows the MMF rates when considering total power constraint and
per-antenna power constraint. It is noticed that the practical per-antenna power
constraint reduces MMF rate performance slightly in both RSMA and SDMA.

Finally, we consider a hot spot user configuration rather than the uniform user
configuration. In Fig. 3.8, the performance of a hot spot configuration, (e.g., with 8
users in the central beam and 1 user each in the other beams) is compared with the
above uniform setting. We can observe that the MMF rate improvement provided by
3.6. Simulation Results and Analysis 85

3 RSMA,PAC
SDMA,PAC
RSMA,TPC
SDMA,TPC
2.5
MMF Rate (bps/Hz)

1.5

20 40 60 80 100 120
Per antenna feed power(W)

Figure 3.7: MMF rate constrained by PAC/ TPC. Nt = 7 antennas, K = 14 users,

ρ = 2 users, imperfect CSIT: α = 0.8.

RSMA is more obvious than SDMA, which means that RSMA is better at managing
interference in such a hot spot scenario. Specifically, for perfect CSIT, RSMA
outperforms SDMA with 42% gains. For imperfect CSIT, RSMA achieves higher
gains at around 54%.

3.6.3 Link-Level Simulations

In this section, by leveraging the results of the MMF optimization problem with
assumptions of Gaussian inputs and infinite block length, we further investigate the
RSMA PHY layer design for multigroup multicast with finite length polar coding,
finite alphabet modulation and an AMC algorithm. In [90], the uncoded link-level
performance of RSMA-based multiuser MISO systems is investigated. With channel
coding taken into consideration, [32] designs the basic transmitter and receiver
architecture for RMSA in a MISO BC with two single-antenna users. Here, we
use the same transceiver architecture as [32] and conduct LLS to show explicit
throughput gain of RSMA multigroup multicast in both cellular and multibeam
86 Chapter 3. RSMA for Multigroup Multicast and Multibeam Satellite Systems

4.5 RSMA, perfect, uniform RSMA, imperfect, uniform

SDMA, perfect, uniform SDMA, imperfect, uniform
RSMA, perfect, hot spot RSMA, imperfect, hot spot
4 SDMA, perfect, hot spot SDMA, imperfect, hot spot

3.5
MMF Rate(bps/Hz)

2.5

1.5

0.5
20 40 60 80 100 120
Per antenna feed power(W)

Figure 3.8: MMF rate versus per-feed available power. Nt = 7 antennas, K = 14

users, imperfect CSIT: α = 0.6, hot spot G = [8, 1, 1, 1, 1, 1, 1].

satellite systems.

The transmitter and receiver architecture for RSMA multigroup multicast is depicted
in Fig. 3.9. We use finite alphabet modulation symbols carrying codewords from
finite-length polar code codebooks as channel inputs. The overall framework follows
the architecture in [32] where a two-user MISO BC system is considered. For more
detailed explanations of each module, please refer to Appendix A.

Thus, we can demonstrate the performance improvements achieved by RSMA over

SDMA for multigroup multicast by LLS results and compare the obtained throughput
levels with the Shannon bounds obtained in the previous sections. The PHY-layer
design follows the architecture described in Fig. 3.9. Appropriate modulation
schemes and coding rates are selected by the AMC algorithm.

In LLS, we define throughput as the number of bits which can be transmitted

correctly at a single channel use. All MMF throughput levels are obtained by
averaging over 100 Monte-Carlo realizations. The number of channel uses in the l-th
3.6. Simulation Results and Analysis 87

Figure 3.9: Transceiver architecture of RSMA multigroup multicast.

(l)
Monte-Carol realization is denoted by S (l) . Ds,k denotes the number of successfully
recovered information bits by user-k for all k ∈ K. Thus, the MMF throughput can
be written as P (l)
mink∈K l Ds,k
MMF Throughput [bps/Hz] = P . (3.97)
l S (l)

Without loss of generality, we assume S (l) = 256 for all l = 1, · · · , 100 Monte-Carlo
realizations. The maximum code rate is set as β = 0.9.

First, we consider a cellular terrestrial multigroup multicast system with K = 6

users equally divided into M = 3 multicast groups. Independent and identically
distributed Rayleigh fading channels are adopted. When the number of transmit
antenna Nt = 6, the system is underloaded. Fig. 3.10 and Fig. 3.11 respectively
show the Shannon bounds and throughput levels achieved by RSMA and SDMA
with imperfect CSIT α = 0.8 and α = 0.6. It can be clearly observed that RSMA
has a significant LLS throughput gain over SDMA. The trend of throughput levels is
consistent with that of Shannon bounds. The performance improvements achieved
by RSMA compared with SDMA are demonstrated in the PHY-layer design and
LLS platform. Moreover, as the CSIT error scaling factor drops from 0.8 to 0.6, the
CSIT uncertainty increases, thus leading to lower throughput values.
88 Chapter 3. RSMA for Multigroup Multicast and Multibeam Satellite Systems

8 RSMA, Shannon Bound

RSMA, Link-Level
SDMA, Shannon Bound
7
MMF Throughput (bps/Hz)
SDMA, Link-Level

0
5 10 15 20 25 30 35
SNR (dB)

Figure 3.10: MMF throughput versus SNR, α = 0.8, Nt = 6 antennas, K = 6 users,

2 users per group.

10

9
RSMA, Shannon Bound
8 RSMA, Link-Level
SDMA, Shannon Bound
MMF Throughput (bps/Hz)

7 SDMA, Link-Level

0
5 10 15 20 25 30 35
SNR (dB)

Figure 3.11: MMF throughput versus SNR, α = 0.6, Nt = 6 antennas, K = 6 users,

2 users per group.
3.6. Simulation Results and Analysis 89

RSMA, Shannon Bound

6 RSMA, Link-Level
SDMA, Shannon Bound
SDMA, Link-Level
MMF Throughput (bps/Hz)
5

0
5 10 15 20 25 30 35
SNR (dB)

Figure 3.12: MMF throughput versus SNR, α = 0.8, Nt = 4 antennas, K = 6 users,

2 users per group.

RSMA, Shannon Bound

6 RSMA, Link-Level
SDMA, Shannon Bound
MMF Throughput (bps/Hz)

SDMA, Link-Level
5

0
5 10 15 20 25 30 35
SNR (dB)

Figure 3.13: MMF throughput versus SNR, α = 0.6, Nt = 4 antennas, K = 6 users,

2 users per group.
90 Chapter 3. RSMA for Multigroup Multicast and Multibeam Satellite Systems

3.5

RSMA, Shannon Bound

3 RSMA, Link-Level
SDMA, Shannon Bound
SDMA, Link-Level

MMF Throughput (bps/Hz) 2.5

1.5

0.5

20 40 60 80 100 120
Per antenna feed power (W)

Figure 3.14: MMF throughput versus per-feed available power, α = 0.8, Nt = 7

antennas, K = 14 users, 2 users per group.

Next, Fig. 3.12 and Fig. 3.13 depict the Shannon bounds and throughput levels
when the number of transmit antenna Nt is 4. Now the system becomes overloaded,
and all multiplexing gains of SDMA are sacrificed and collapse to 0 [68]. The curve
of SDMA Shannon bound gradually saturates as SNR grows. The rate gain of
RSMA over SDMA is more obvious. By LLS, the MMF throughput levels of both
RSMA and SDMA follow the trend of Shannon bounds with comparable gaps. The
throughput of RSMA outperforms SDMA significantly in the presence of considered
imperfect CSIT α = 0.8 and α = 0.6.

Finally, we consider the same multibeam satellite system as discussed in Section

3.6.2, where a GEO satellite equipped with Nt = 7 antennas serves K = 14 single-
antenna users simultaneously. Single feed per beam architecture is used such that
K
only one feed is required to generate one beam (i.e., Nt = M ). ρ = M
= 2 users are
served simultaneously by each beam. Fig. 3.14 illustrates the Shannon bounds and
throughput levels achieved by RSMA and SDMA versus an increasing per-antenna
transmit power budget with imperfect CSIT α = 0.8 We can still observe the
3.7. Summary 91

matching trends of the Shannon bounds and throughput curves in this satellite
setup. The effectiveness of using RSMA in multibeam satellite systems compared
with conventional SDMA is demonstrated by LLS.

3.7 Summary

In this chapter, we focus on the application of RSMA for multigroup multicast

beamforming in the presence of imperfect CSIT. Through MMF-DoF analysis,
RSMA is shown to provide gains in both underloaded and overloaded systems
compared with the conventional SDMA. A generic MMF optimization problem is
formulated and solved by developing a modified WMMSE approach together with
an AO algorithm. The effectiveness of adopting RSMA for multigroup multicast
and multibeam satellite communications is evaluated through simulations in a wide
range of setups, taking into account CSIT uncertainty and practical challenges.
Additionally, the RSMA transmitter and receiver architecture and LLS platform are
designed. According to numerical link-level results, we can conclude that RSMA
is very promising for practical implementation to tackle the challenges of modern
communication systems in numerous application areas.
Chapter 4

RSMA for Satellite-Terrestrial

Integrated Networks

In this chapter, we investigate the joint beamforming design problem to achieve

max-min rate fairness in a STIN where the satellite provides wide coverage to
multibeam multicast satellite users (SUs), and the terrestrial BS serves multiple
cellular users (CUs) in a densely populated area. Both the satellite and BS operate
in the same frequency band. We present two RSMA-based STIN schemes, namely
the coordinated scheme relying on CSI sharing and the cooperative scheme relying
on CSI and data sharing. The objective is to maximize the minimum fairness rate
amongst all SUs and CUs subject to transmit power constraints at the satellite
and the BS. A joint beamforming algorithm is proposed to reformulate the original
problem into an approximately equivalent convex one, which can be iteratively solved.
Moreover, an expectation-based robust joint beamforming algorithm is proposed
against the practical environment when the satellite channel phase uncertainties
are considered. Simulation results demonstrate the effectiveness and robustness of
the proposed RSMA schemes for STIN and exhibit significant performance gains
compared with various baseline strategies.

92
4.1. Introduction 93

4.1 Introduction

The concept of STIN has been proposed in the literature [91–93]. The satellite sub-
network shares the same frequency band with the terrestrial sub-network through
dynamic spectrum access technology to enhance spectrum utilization, thereby achiev-
ing higher spectrum efficiency and throughput. However, aggressive frequency reuse
can induce severe interference within and between the sub-networks. In this chap-
ter, we will concentrate on RSMA-based joint beamforming schemes to efficiently
mitigate the interference of STIN.

Motivated by the benefits of RSMA presented in Chapter 3, in this chapter, we

further investigate the application of RSMA into STIN to manage the interference
within and between both sub-networks. Practical challenges are considered, such as
the per-feed transmit power constraints, CSIT uncertainty, and multibeam multicast
transmission due to the existing satellite communication standards [65]. The main
contributions of this chapter are summarized as follows.

• First, we present a multiuser downlink framework for the integrated network

where the satellite exploits multibeam multicast communication to serve SUs,
while the terrestrial BS employs uniform planar array (UPA) and serves cellular
users (CUs) in a densely populated area. We take into account multibeam
satellite characteristics, including the array pattern, path loss and rain atten-
uation, thus building a more realistic channel model to evaluate the system
performance. The GW operates as a control center to implement centralized
processing and control the whole network. Based on such framework, the joint
beamforming design arises so that the satellite and terrestrial sub-system can
share the same radio spectrum resources and cooperate with each other. RSMA
is used at both the satellite and the BS to mitigate the interference including
inter-beam interference, intra-cell interference and interference between the two
94 Chapter 4. RSMA for Satellite-Terrestrial Integrated Networks

sub-systems. We investigate two scenarios of RSMA-based STIN, namely the

coordinated scheme, and the cooperative scheme. For the coordinated scheme,
the satellite and BS exchange CSI of both direct and interfering links at the
GW, and coordinate beamforming to manage the interference. For the cooper-
ative scheme, the satellite and BS exchange both CSI and data at the GW.
All propagation links (including interfering ones) are exploited to carry useful
data upon appropriate beamforming. This differs from the prior RSMA-based
STIN paper [71], where RSMA is utilized only at the terrestrial sub-system,
and the benefits of coordination and cooperation are not investigated.

• Second, for both coordinated scheme and cooperative scheme, we respectively

formulate optimization problems to maximize the minimum fairness rate of
the RSMA-based STIN amongst all users subject to the constraint of per-feed
transmit power at the satellite and the constraint of sum transmit power
at the BS. Such problems upgrade the application of RSMA to multibeam
satellite communications and terrestrial networks to a more general case,
therefore leading to a joint beamforming design so that the two sub-systems
can cooperate with each other. This is the first work on the joint beamforming
design of RSMA-based coordinated STIN and cooperative STIN. Since the
original optimization problem is non-convex, we apply the SCA to reformulate
the original problem into an approximately equivalent convex one, which
belongs to a second-order cone program (SOCP) and can be solved iteratively.
The cooperative scheme is shown to outperform the coordinated scheme due
to data exchange between the satellite and BS at the GW. Multiple baseline
strategies are considered, including SDMA, NOMA, a two-step beamforming
and fractional frequency reuse. Simulation results demonstrate the superiority
of the proposed RSMA-based cooperative scheme and coordinated scheme
compared with the baseline strategies.
4.2. System Model 95

• Third, since it is in general very challenging to acquire accurate satellite CSI

at the GW due to the round-trip delay and device mobility, we develop an
expectation-based robust beamforming design against satellite channel phase
uncertainty. For both RSMA-based coordinated STIN and cooperative STIN,
non-convex MMF problems are formulated. To tackle the non-convexity of the
robust design, a novel iterative algorithm is proposed using SCA combining with
a penalty function. Simulation results verify the effectiveness and robustness
of the proposed RSMA schemes for STIN.

4.2 System Model

As illustrated in Fig. 4.1, we consider a STIN system employing full frequency

reuse, where all SUs and CUs operate in the same frequency band. A GEO satellite
is equipped with an array-fed reflector antenna. It provides services to SUs that
lack terrestrial access in sparsely populated or remote areas. By assuming a single
feed per beam architecture, the array-fed reflector antenna comprises a feed array
with Ns feeds and generates Ns adjacent beams. Within the multibeam coverage
Ks
area, we assume Ks SUs, and ρ = Ns
users in each beam. Since the SUs of each
beam are served simultaneously by transmitting a single coded frame following DVB-
S2X, the GEO satellite implements multibeam multicast transmission. Meanwhile,
the terrestrial BS1 equipped with Nt -antenna UPA serves densely populated areas
in the same frequency band. Kt ≤ Nt unicast CUs are assumed. User mobility
is not considered in this work. Spectrum sharing is able to improve spectrum
efficiency, which also leads to interference in and between the terrestrial and satellite
sub-networks. As shown in Fig. 4.1, the GW acts as a control center to collect
and manage various kinds of information, implement centralized processing and
1
In this chapter, a unique BS is considered. The setting of multiple BSs is not considered here
and is left for future studies.
96 Chapter 4. RSMA for Satellite-Terrestrial Integrated Networks

Figure 4.1: Model of a satellite-terrestrial integrated network.

control the whole STIN. Optimal resource allocation and interference management
on the satellite and BS can be jointly implemented at the GW2 to improve system
performance.

4.2.1 Channel Model

As illustrated in Fig. 4.2, we assume UPA at the BS with dimension Nt = N1 × N2 .

N1 and N2 are respectively the number of array elements uniformly employed along
the X-axis and the Z-axis. The distances between adjacent array elements are
identical, thus d1 = d2 = d. Due to the characteristic of radio wave propagation
at high-frequency bands, the terrestrial channels can be expressed by a model
consisting of L scatters. Each scatter contributes to a single propagation path.
Mathematically,the downlink channel between the BS and CU kt is given by [96]

r L
1X
hkt = αk ,l aUPA (θkt ,l , φkt ,l ) , (4.1)
L l=1 t

2
Complete CSI of the STIN system is required at the GW, leading to significant CSI feedback
overhead. To reduce the feedback overhead in STIN systems, several techniques can be used
including e.g., compressed sensing, codebook-based feedback, spatially correlated feedback and CSI
prediction using Kalman filtering or deep learning-based methods [94, 95]. However, this problem
exceeds the scope of this thesis.
4.2. System Model 97

Figure 4.2: Geometry of uniform planar array employed at the BS.

where αkt ,l is the complex channel gain of the l-th path. Each αkt ,l is assumed to
follow independent and identical distribution (i.i.d) CN (0, 1). By denoting θkt ,l and
φkt ,l as the azimuth and elevation angles of the l-th path, the vector aUPA (θkt ,l , φkt ,l )
can be expressed as a function of the Cartesian coordinates of the transmit arrays
as follows

T T
 
j 2π r̄ ,··· ,r̄Nt ]
λ [ 1 [cos θkt ,l cos φkt ,l , sin θkt ,l cos φkt ,l , cos φkt ,l ]
aUPA (θkt ,l , φkt ,l ) = e . (4.2)

where [r̄1 , · · · , r̄Nt ] ∈ R3×Nt have columns representing the Cartesian coordinates
of the UPA array elements. The terrestrial channel matrix between the BS and all
CUs is denoted by H = [h1 , · · · , hKt ] ∈ CNt ×Kt .

Considering the free space loss, radiation pattern and rain attenuation of satellite
channels, the downlink channel from the satellite to SU-ks can be modelled the same
as in Section 3.6.2.

The satellite channel matrix between the satellite and all SUs is denoted by F =
[f1 , · · · , fKs ] ∈ CNs ×Ks . Similarly, when we consider ns ∈ {1, · · · , Ns } and kt ∈
{1, · · · , Kt }, the interfering satellite channel matrix between the satellite and all
CUs is denoted by Z = [z1 , · · · , zKt ] ∈ CNs ×Kt .
98 Chapter 4. RSMA for Satellite-Terrestrial Integrated Networks

4.2.2 Coordinated scheme and Cooperative Scheme

We consider two levels of integration between the satellite and terrestrial BS.

1) Coordinated Scheme:

First, we consider the basic level of integration where the CSI of both direct
and interfering links of the whole network is available at the GW, while data is
not exchanged between the satellite and BS at the GW. We call such scheme a
coordinated scheme. It allows the satellite and BS to coordinate power allocation
and beamforming directions to suppress interference. Different multiple access
strategies can be exploited at the satellite and BS, such as RSMA, SDMA, NOMA,
etc. Here, we elaborate on the scenario where RSMA3 is used at both the BS
and satellite. To that end, the unicast messages W1 , · · · , WKt intended to CUs
indexed by Kt = {1, · · · , Kt } are split into common parts and private parts, i.e.,
Wkt → {Wc,kt , Wp,kt } , ∀kt ∈ Kt . All common parts are combined into Wc and
encoded into a common stream sc to be decoded by all CUs. All private parts are
independently encoded into private streams s1 , · · · , sKt . The vector of BS streams
s = [sc , s1 , · · · , sKt ]T ∈ C(Kt +1)×1 is therefore created, and we suppose it obeying

E ssH = I. For the satellite, multicast messages M1 , · · · MNs are intended to the
beams indexed by Ns = {1, · · · , Ns }. Each message Mns , ∀ns ∈ Ns is split into a
common part Mc,ns and a private part Mp,ns . All common parts are combined as
Mc and encoded into mc , while all private parts are independently encoded into
m1 , · · · , mNs . The vector of satellite streams m = [mc , m1 , · · · , mNs ]T ∈ C(Ns +1)×1

is obtained, and we assume it satisfying E mmH = I. Both s and m are linearly
precoded. The transmitted signals at the satellite and BS are respectively

Ns
X Kt
X
xsat = wc mc + wns mns and xbs = pc sc + pkt skt , (4.3)
ns =1 kt =1

3
RSMA has been shown analytically as a general multiple access strategy, which boils down to
SDMA and NOMA when allocating powers to the different types of message streams [9].
4.2. System Model 99

where W = [wc , w1 , · · · , wNs ] ∈ CNs ×(Ns +1) and P = [pc , p1 , · · · , pKt ] ∈ CNt ×(Kt +1)
are defined as the beamforming matrices at the satellite and BS. mc and sc are
superimposed on top of the private signals. Even though power-sharing mechanisms
among beams can be implemented by using, e.g., multi-port amplifiers [97], the
deployment of satellite payloads allowing flexible power allocation will require costly
and complex radio-frequency designs. Thus, a per-feed transmit power constraint
Ps
is considered, which is given by (WWH )ns ,ns ≤ Ns
, ∀ns ∈ Ns . The sum transmit
power constraint of BS is given by tr(PPH ) ≤ Pt . Based on the channel models
defined above, the received signal at each SU-ks writes as

Ns
X
yksat
s
= fkHs wc mc + fkHs wi mi + nsat
ks . (4.4)
i=1

Since we assume all SUs are located outside the BS service area, each SU sees
multibeam interference and no interference from the BS. The received signal at each
CU-kt writes as

Kt
X Ns
X
ykbst = hH
kt pc sc + hH
kt pj s j + zH
kt wc mc + zH
kt wi mi + nbs
kt . (4.5)
j=1 i=1

Each CU suffers from intra-cell interference and from satellite interference. z1 , · · · , zKt
represent satellite interfering channels. nsat bs
ks and nkt are the AWGN with zero mean

and variance σksat2

s
and σkbs2
t
respectively. For both SUs and CUs, the common stream
is firstly decoded while treating the other interference as noise. The SINRs of
decoding the common stream at SU-ks and CU-kt are given by

2
sat fkHs wc
γc,k s
= PN 2 , (4.6)
s
i=1 fkHs wi + σksat2
s
2
bs hH
kt pc
γc,k t
= PK 2 2 PNs 2 . (4.7)
t
j=1 hH
kt pj + zH
k t wc + i=1 zH bs2
kt wi + σkt

sat
Given perfect CSIT, the achievable rate of the common streams are Rc,k s
= log2 (1 +
100 Chapter 4. RSMA for Satellite-Terrestrial Integrated Networks

sat bs bs
γc,k s
) and Rc,k t
= log2 (1 + γc,k t
). To guarantee that each SU is capable of decoding
mc , and each CU is capable of decoding sc , they must be transmitted at rates that
do not exceed

Ns
X Kt
X
Rcsat sat
Cnsat Rcbs bs
Ckbst ,
 
= min Rc,k s
= s
and = min Rc,k t
= (4.8)
ks ∈Ks kt ∈Kt
ns =1 kt =1

where Cnsat
s
is the portion of the common part of the ns -th beam’s message. Ckbst
is the portion of the common part of the kt -th CU’s message. After the common
stream is re-encoded, precoded and subtracted from the received signal through
SIC, each user then decodes its desired private stream. We define µ : Ks → Ns as
mapping a SU to its corresponding beam. The SINRs of decoding mµ(ks ) at SU-ks
and decoding skt at CU-kt are given by

2
fkHs wµ(ks )
γksat
s
= PN 2 , (4.9)
s
i=1,i̸=µ(ks ) fkHs wi + σksat2
s
2
hH
kt pkt
γkbst = PK 2 2 PNs 2 . (4.10)
t
j=1,j̸=kt hH
kt pj + zH
k t wc + i=1 zH bs2
kt wi + σkt

The achievable rates of the private streams are respectively Rksat

s
= log2 (1 + γksat
s
)
and Rkbst = log2 (1 + γkbst ). Thus, the achievable rates of the ns -th beam and kt -th CU
respectively write as

sat
Rtot,n s
= Cnsat
s
+ min Risat bs
and Rtot,k t
= Ckbst + Rkbst , (4.11)
i∈Gns

where Gns denotes the set of SUs belonging to the ns -th beam.

2) Cooperative Scheme

Second, we consider a higher level of integration, i.e., cooperative scheme where

both CSI and data are exchanged between the satellite and BS at the GW. In
this scenario, all downlink messages W1 , · · · , WKt intended to CUs, and multicast
4.2. System Model 101

messages M1 , · · · , MNs intended to SUs are transmitted at both the satellite and BS.
All propagation links (including interfering ones) are exploited to carry useful data
upon appropriate beamforming. We still consider RSMA to manage interference in
this cooperative STIN, including inter-beam interference, intra-cell interference and
interference between the satellite and terrestrial sub-networks. Each message is split
into a common part and a private part. All common parts are encoded together into
a super common stream shared by all users in the system. As a result, the symbol
stream to be transmitted is given by ś = [śc , ḿ1 , · · · , ḿNs , ś1 , · · · , śKt ]T ∈ C Ns +Kt +1 .
Throughout this work, we use “´” to differentiate notations in the cooperative scheme
and the above coordinated scheme. The transmitted signals at the satellite writes as

Ns
X Kt
X
sat
x́ = ẃc śc + ẃisat ḿi + ẃjbs śj , (4.12)
i=1 j=1

 
where Ẃ = ẃc , ẃ1sat , · · · , ẃN
sat
s
, ẃ1bs , · · · , ẃK
bs
t
is the beamforming matrix, and the
superscripts of ẃisat and ẃjbs are used to differentiate the precoder of satellite data
and BS data. The per-feed transmit power constraint writes as (ẂẂH )ns ,ns ≤
Ps
Ns
, ∀ns ∈ Ns . Similarly, the transmitted signal at the BS writes as

Ns
X Kt
X
bs
x́ = ṕc śc + ṕsat
i ḿi + ṕbs
j śj , (4.13)
i=1 j=1

 
where Ṕ = ṕc , ṕsat sat bs bs
1 , · · · , ṕNs , ṕ1 , · · · , ṕKt is the beamforming matrix, and the sum

transmit power constraint of the BS is tr(ṔṔH ) ≤ Pt . Accordingly, the received

signal at SU-ks is given by

Ns
X Kt
X
ýksat
s
= fkHs ẃc śc + fkHs ẃisat ḿi + fkHs ẃjbs śj + ńsat
ks . (4.14)
i=1 j=1
102 Chapter 4. RSMA for Satellite-Terrestrial Integrated Networks

The received signal at CU-kt is given by

Kt
X Ns
X
ýkbst = hH H
kt ṕc śc + hkt ṕbs H
j śj + hkt ṕsat
i ḿi
j=1 i=1
Ns
X Kt
X
+ zH
kt ẃc śc + zH
kt ẃisat ḿi + zH
kt ẃjbs śj + ńbs
kt . (4.15)
i=1 j=1

To simplify (4.15), aggregate channels and aggregate beamforming vectors are defined
by

H H
gk t = z H ∈ C(Ns +Nt )×1 , ∀kt ∈ Kt ,
 
kt , hkt (4.16)
H
vc = wc∗H , p∗H ∈ C(Ns +Nt )×1 ,

c (4.17)
 satH satH H
vnsat
s
= ẃns , ṕns ∈ C(Ns +Nt )×1 , ∀ns ∈ Ns , (4.18)
H
vkbst = ẃkbsH , ṕbsH ∈ C(Ns +Nt )×1 , ∀kt ∈ Kt .

t kt (4.19)

The received signal at CU-kt can be rewritten as

Kt
X Ns
X
ýkbst = gkHt vc śc + gkHt vjbs śj + gkHt visat ḿi + ńbs
kt . (4.20)
j=1 i=1

Satellite interfering links are exploited to carry terrestrial data so as to improve

the performance of STIN. The aggregate beamforming vectors are collected into a
matrix

V = [vc , v1sat , · · · , vN
sat
s
, v1bs , · · · , vK
bs
t
] ∈ C(Ns +Nt )×(Ns +Kt +1) , (4.21)

which can also be denoted by V = [ẂH , ṔH ]H . For both SUs and CUs, the common
stream is firstly decoded and removed from the received signal through SIC. The
4.2. System Model 103

SINRs of decoding śc at the ks -th SU and the kt -th CU are respectively

2
sat fkHs ẃc
γ́c,k = PN 2 P t H bs 2 , (4.22)
+ K
s s
i=1 fkHs ẃisat f
j=1 ks ẃ j + σ sat2
ks
2
bs gkHt vc
γ́c,k = PK 2 P s H sat 2 . (4.23)
+ N
t t
j=1 gkHt vjbs i=1 g v
kt i + σ bs2
kt

sat sat bs
The corresponding achievable rates are Ŕc,k s
= log2 (1 + γ́c,k s
) and Ŕc,k t
= log2 (1 +
bs
γ́c,k t
). Since śc is decoded by all users in the system, we define the common rate as

Ns
X Kt
X
sat bs
Ćnsat Ćkbst .

Ŕc = min Ŕc,k s
, Ŕc,k t
= s
+ (4.24)
ks ∈Ks ,kt ∈Kt
ns =1 kt =1

Note that śc is shared amongst all satellite beams and CUs. Ćnsat
s
and Ćkbst respectively
correspond to the beam-ns ’s and CU-kt ’s portion of common rate. After removing śc
using SIC, each user then decodes its desired private stream. The SINRs of decoding
private streams are

2
fkHs ẃµ(k
sat
s)
γ́ksat = PN , (4.25)
H sat 2
s PKt H bs 2
s
i=1,i̸=µ(ks ) fks ẃi + j=1 fks ẃj + σksat2
s
2
gkHt vkbst
γ́kbst = PK 2 P s H sat 2 . (4.26)
t
j=1,j̸=kt gkHt vjbs + N i=1 g v
kt i + σ bs2
kt

Ŕksat
s
= log2 (1 + γ́ksat
s
) and Ŕkbst = log2 (1 + γ́kbst ) are the achievable rates of the private
streams. Thus, the achievable rates of the ns -th beam and kt -th CU respectively
write as

sat
Ŕtot,n s
= Ćnsat
s
+ min Ŕisat bs
and Ŕtot,k t
= Ćkbst + Ŕkbst . (4.27)
i∈Gns

From the above expressions, we can regard the satellite and BS working together as
a super “BS” but subject to their respective power constraints to serve the CUs and
SUs. The super common stream contains parts of the unicast messages intended
104 Chapter 4. RSMA for Satellite-Terrestrial Integrated Networks

to the CUs, and parts of the multicast messages intended to the SUs. At each
user side, the super common stream is at first decoded and then removed through
SIC. Accordingly, the interference is partially decoded. Each user then decodes its
private stream and treats the remaining interference as noise. Such scheme has the
capability to better manage interference including not only inter-beam interference,
intra-cell interference, but also interference between the satellite and terrestrial
sub-networks.

Remark 4.1: With the assumption of Gaussian signalling and infinite block length,
there is no decoding error in SIC. Decoding errors in SIC would only occur if we depart
from Shannon assumptions and assume finite constellations and finite block lengths.
We consider one-layer RSMA for either coordinated scheme and cooperative scheme.
Only one layer of SIC is required at each terminal. The receiver complexity does
not depend on the number of served users. The generalized RSMA and hierarchical
RSMA described in [5] is able to provide more room for achievable rate enhancements
at the expense of more layers of SIC at receivers. However, its implementation can
be complex due to the large number of SIC layers and common messages involved.
The receiver complexity of generalized RSMA and hierarchical RSMA increases with
the number of served users. Moreover, ordering and grouping are not required in
this one-layer RSMA architecture since all users decode the common stream before
decoding their private streams. Both scheduling complexity and receiver complexity
are reduced tremendously. Readers are referred to [5] and [98] for more details on
complexity issues.

4.3 Proposed Joint Beamforming Scheme

In this section, the problem of interest is to design a joint beamforming scheme

to maximize the minimum fairness rate amongst all unicast CUs and multibeam
4.3. Proposed Joint Beamforming Scheme 105

multicast SUs subject to transmit power constraints. We respectively consider the

scenarios of RSMA-based coordinated STIN and cooperative STIN with perfect CSI
at the GW.

4.3.1 Joint Beamforming Design for Coordinated STIN

For RSMA-based coordinated STIN, the optimization problem to maximize the

minimum fairness rate can be formulated as

bs sat

P1 : max min Rtot,kt
, Rtot,ns
(4.28)
W,P,csat ,cbs ns ∈Ns ,kt ∈Kt
Kt
X
bs
s.t. Rc,k t
≥ Cjbs , ∀kt ∈ Kt (4.29)
j=1

Ckbst ≥ 0, ∀kt ∈ Kt (4.30)

tr(PPH ) ≤ Pt (4.31)
Ns
X
sat
Rc,k s
≥ Cjsat , ∀ks ∈ Ks (4.32)
j=1

Cnsat
s
≥ 0, ∀ns ∈ Ns (4.33)
Ps
(WWH )ns ,ns ≤ , ∀ns ∈ Ns (4.34)
Ns

where csat = [C1sat , · · · , CNsats ]T , cbs = [C1bs , · · · , CK

bs T
t
] are the vectors of common rate
portions. (4.29) guarantees that the common stream sc can be decoded by all CUs.
(4.31) is the sum transmit power constraint of the BS. Similarly, (4.32) ensures the
common stream mc to be decoded by all SUs. (4.34) represents the per-feed transmit
power constraint of the satellite. (4.30) and (4.33) guarantee the non-negativity of
all common rate portions.

Note that the formulated problem is non-convex, we exploit an SCA-based method

to convexify the non-convex constraints and approximate the non-convex problem
106 Chapter 4. RSMA for Satellite-Terrestrial Integrated Networks

to a convex one. First, we introduce an equivalent reformulation of P1 , which is

E1 : max q (4.35)
W,P,csat ,cbs ,q,r,α

s.t. Ckbst + αkt ≥ q, ∀kt ∈ Kt (4.36)

Rkbst ≥ αkt , ∀kt ∈ Kt (4.37)

Cnsat
s
+ rks ≥ q, ∀ks ∈ Gns (4.38)

Rksat
s
≥ rks , ∀ks ∈ Ks (4.39)

(4.29) − (4.34)

where q, α = [α1 , · · · , αKt ]T , r = [r1 , · · · , rKs ]T are introduced auxiliary variables.

To deal with the non-convexity of (4.29), (4.32), (4.37), (4.39), we further introduce
new auxiliary variables a = [a1 , · · · , aKt ]T , ac = [ac,1 , · · · , ac,Kt ]T , b = [b1 , · · · , bKs ]T
and bc = [bc,1 , · · · , bc,Ks ]T . The problem E1 can be rewritten as

S1 : max q (4.40)
q,W,P,csat ,cbs ,r,α,a,ac ,b,bc

s.t. log (1 + akt ) ≥ αkt log 2, ∀kt ∈ Kt (4.41)

γkbst ≥ akt , ∀kt ∈ Kt (4.42)

log (1 + bks ) ≥ rks log 2, ∀ks ∈ Ks (4.43)

γksat
s
≥ bks , ∀ks ∈ Ks (4.44)
Kt
X
log (1 + ac,kt ) ≥ Cjbs log 2, ∀kt ∈ Kt (4.45)
j=1

bs
γc,k t
≥ ac,kt , ∀kt ∈ Kt (4.46)
Ns
X
log (1 + bc,ks ) ≥ Cjsat log 2, ∀ks ∈ Ks (4.47)
j=1

sat
γc,k s
≥ bc,ks , ∀ks ∈ Ks (4.48)

(4.30), (4.31), (4.33), (4.34), (4.36), (4.38)

4.3. Proposed Joint Beamforming Scheme 107

where (4.41) - (4.48) are obtained by extracting the SINRs from the rate expressions
Rkbst , Rksat
s
bs
, Rc,k t
sat
, Rc,k s
in (4.29), (4.32), (4.37), (4.39) of Problem E1 . Since the con-
straints of S1 hold with equality at optimality, the equivalence between P1 and S1
can be guaranteed. Now, the non-convexity of S1 comes from (4.42), (4.44), (4.46)
and (4.48) which contain SINR expressions. (4.42) can be expanded as

Kt Ns 2
X 2 2
X 2 hH pkt
hH
kt pj + zH
kt wc + zH
kt wi + σkbs2
t
≤ kt , (4.49)
j=1,j̸=kt i=1
akt

where the right-hand side quadratic-over-linear function is convex. We approximate

it with its lower bound, which is obtained by the first-order Taylor approximation
[n] [n]

around the point pkt , akt . Then, we have

2 [n]H
[n]H [n]

hH
kt pkt 2R pkt hkt hH
kt pkt pkt hkt hHkt pkt
≥ [n]
− [n]
2
akt
akt akt ak t
[n] [n]

≜ fb1 pkt , akt ; pkt , akt (4.50)

where n represents the n-th SCA iteration. Replacing the linear approximation
[n] [n]

fb1 pkt , akt ; pkt , akt with the right-hand side of (4.49) yields

Kt Ns
2 2 2
X X
hH zH zH bs2 b1 pkt , akt ; p[n] , a[n] ≤ 0.


kt pj + k t wc + kt w i + σ kt − f kt kt
j=1,j̸=kt i=1

(4.51)

Similarly, the constraint (4.44) can be expanded as

Ns 2
X 2 f H wµ(ks )
fkHs wi + σksat2
s
≤ ks . (4.52)
bk s
i=1,i̸=µ(ks )
108 Chapter 4. RSMA for Satellite-Terrestrial Integrated Networks

[n] [n]

We approximate its right-hand side around the point wµ(ks ) , bks , and obtain

2 [n]H
[n]H [n]

fkHs wµ(ks ) 2R wµ(ks ) fks fkHs wµ(ks ) wµ(ks ) fks fkHs wµ(ks )
≥ [n]
− [n]
2
bk s
bk s bk s bks
[n] [n]

≜ fb2 wµ(ks ) , bks ; wµ(ks ) , bks . (4.53)

[n] [n]

Replacing fb2 wµ(ks ) , bks ; wµ(ks ) , bks with the right-hand side of (4.52) yields

Ns
2
X [n] [n]

fkHs wi + σksat2
s
− fb2 wµ(ks ) , bks ; wµ(ks ) , bks ≤ 0. (4.54)
i=1,i̸=µ(ks )

Following the same logic, (4.46) and (4.48) are respectively approximated by

Kt Ns
2 2 2 2
X X
hH
kt pkt + hH
kt pj + zH
k t wc + zH bs2
kt wi + σkt
j=1,j̸=kt i=1

[n]
− fb3 pc , ac,kt ; p[n]


c , ac,kt ≤ 0, (4.55)
Ns
2 2
X [n]

fkHs wµ(ks ) + fkHs wi + σksat2
s
− fb4 wc , bc,ks ; wc[n] , bc,ks ≤ 0, (4.56)
i=1,i̸=µ(ks )

[n] [n]
[n] [n]

where fb3 pc , ac,kt ; pc , ac,kt and fb4 wc , bc,ks ; wc , bc,ks are linear lower bound ex-
pressions given by

[n]H
[n]H [n]

[n]
2R pc hkt hH
kt pc pc hkt hH kt pc
fb3 pc , ac,kt ; p[n]
c , ac,kt ≜ [n]
− [n] 2
ac,kt , (4.57)
ac,kt ac,kt
[n]H
[n]H [n]
[n]
2R wc fks fkHs wc wc fks fkHs wc
fb4 wc , bc,ks ; wc[n] , bc,ks ≜ [n]
− [n]
2
bc,ks . (4.58)
bc,ks bc,ks

Although (4.41), (4.43), (4.45) and (4.47) are convex constraints, which are solvable
through the CVX toolbox in Matlab, the log terms belong to generalized nonlinear
convex program with high computational complexity. Aiming at more efficient
implementation, [99] approximates the log constraints to a set of SOC constraints,
which introduce a great number of slack variables and result in an increase of
4.3. Proposed Joint Beamforming Scheme 109

per-iteration complexity. Here, we use the property that x log (1 + x) is convex

as in [100], and approximate (4.41), (4.43), (4.45), (4.47) without additional slack
variables. Since akt ≥ 0, the constraint (4.41) can be rewritten as

akt log (1 + akt ) ≥ akt αkt log 2. (4.59)

Its left-hand side is convex, so we compute the first-order Taylor approximation of

[n]
akt log (1 + akt ) around the point akt as

h a[n] i
[n] [n]
[n]
kt [n]

akt log (1 + akt ) ≥ akt log 1 + akt + akt − akt [n]
+ log 1 + akt
1 + akt
[n] [n]
= akt vkt − ukt , (4.60)

[n] [n] [n]

where vkt and ukt are expressions of akt given by

[n] [n]
2
[n] akt [n]
[n] akt
vkt = [n]
+ log 1 + akt and ukt = [n]
. (4.61)
ak t + 1 ak t + 1

[n] [n]
Now, (4.59) can be rewritten by akt vkt − ukt ≥ akt αkt log 2, which is SOC repre-
sentable [101] as

q 
 [n] [n] [n]
akt + αkt log 2 − vkt 2 ukt ≤ akt − αkt log 2 + vkt . (4.62)
2

Similarly, the constraint (4.43), (4.45), (4.47) can be replaced by

q 
 [n] [n] [n]
bks + rks log 2 − v̄ks 2 ūks ≤ bks − rks log 2 + v̄ks , (4.63)
2
Kt
X q Kt
X
[n] [n]  [n]
Cjbs log 2 − vc,kt Cjbs log 2 + vc,kt ,

ac,kt + 2 uc,kt ≤ ac,kt − (4.64)
2
j=1 j=1
Ns
X q Ns
X
[n] [n]  [n]
Cjsat Cjsat log 2 + v̄c,ks .

bc,ks + log 2 − v̄c,ks 2 ūc,ks ≤ bc,ks − (4.65)
2
j=1 j=1
110 Chapter 4. RSMA for Satellite-Terrestrial Integrated Networks

[n] [n] [n] [n] [n] [n]

The expressions of v̄ks , ūks , vc,kt , uc,kt , v̄c,ks , ūc,ks are respectively

[n] [n]
2
[n] bk s [n]
[n] bk s
v̄ks = [n]
+ log 1 + bks and ūks = [n]
,
bks + 1 bks + 1
[n] [n]
2
[n] ac,kt [n]
[n] ac,kt
vc,kt = [n]
+ log 1 + ac,kt and uc,kt = [n]
,
ac,kt + 1 ac,kt + 1
[n] [n]
2
[n] bc,ks [n]
[n] bc,ks
v̄c,ks = [n]
+ log 1 + bc,ks and ūc,ks = [n]
. (4.66)
bc,ks + 1 bc,ks + 1

By replacing the constraints (45)-(52) with (55), (58), (59), (60), (66)-(69), we
obtain

A1 : max q (4.67)
q,W,P,csat ,cbs ,r,α,a,ac ,b,bc

s.t. (4.51), (4.55), (4.62), (4.64), ∀kt ∈ Kt

(4.54), (4.56), (4.63), (4.65), ∀ks ∈ Ks

(4.30), (4.31), (4.33), (4.34), (4.36), (4.38)

The n-th iteration of the problem A1 belongs to SOCP and can be efficiently solved
by the standard solvers in CVX. In each iteration, the problem defined around the
solution of the previous iteration is solved. Variables are updated iteratively until a
stopping criterion is satisfied. We summarize the procedure of this RSMA-based joint
beamforming scheme in Algorithm 2. ε is the tolerance value. The optimal solution
of Problem A1 at iteration-n is a feasible solution of the problem at iteration-(n + 1).
As a consequence, the objective variable q increases monotonically. It is bounded
above by the transmit power constraints. The proposed Algorithm 2 is guaranteed to
converge while the global optimality of the achieved solution can not be guaranteed.
The solution of the proposed SCA-based algorithm converges to the set of KKT
points (which is also known as the stationary points) of problem P1 [102].
4.3. Proposed Joint Beamforming Scheme 111

Algorithm 2 Proposed Joint Beamforming Scheme

[n] [n]
Initialize: n ← 0, W[n] , P[n] , a[n] , ac , b[n] , bc , q [n] ;
repeat
[n] [n]

Solve the problem A1 at W[n] , P[n] , a[n] , ac , b[n] , bc to get


the optimal solution W̆, P̆, ă, ăc , b̆, b̆c , q̆ ;
n ← n + 1;
[n] [n]
Update W[n] ← W̆, P[n] ← P̆, a[n] ← ă, ac ← ăc , b[n] ← b̆, bc ← b̆c , q [n] ←
q̆;
until q [n] − q [n−1] < ε ;

4.3.2 Joint Beamforming Design for Cooperative STIN

When RSMA-based cooperative STIN is considered, the optimization problem of

max-min fairness rate among all users is given by

Ŕkbst , Ŕksat

P2 : max min s
(4.68)
Ẃ,Ṕ,ć ns ∈Ns ,kt ∈Kt
Kt
X Ns
X
bs
s.t. Ŕc,k t
≥ Ćjbs + Ćjsat , ∀kt ∈ Kt (4.69)
j=1 j=1

Ćkbst ≥ 0 ∀kt ∈ Kt (4.70)

tr ṔṔH ≤ Pt


(4.71)
Kt
X Ns
X
sat
Ŕc,k s
≥ Ćjbs + Ćjsat , ∀ks ∈ Ks (4.72)
j=1 j=1

Ćnsat
s
≥ 0, ∀ns ∈ Ns (4.73)
Ps
ẂẂH ns ,ns ≤


, ∀ns ∈ Ns (4.74)
Ns

where ć = [Ć1sat , · · · , ĆNsats , Ć1bs , · · · , ĆK

bs T
t
] is the vector of all common rate portions.
(4.69) and (4.72) guarantee that the common stream of the whole system śc can be
decoded by all SUs and CUs. (4.70) and (4.73) ensure non-negativity of each element
in ć. (4.71) and (4.74) are respectively the sum transmit power constraint of the
BS and per-feed transmit power constraints of the satellite. The formulated MMF
problem for cooperative STIN is also non-convex. Note that the main difference
112 Chapter 4. RSMA for Satellite-Terrestrial Integrated Networks

between P1 and P2 lies in the transmit data information sharing in P2 . One super
common stream is transmitted at both the satellite and BS instead of transmitting
individual common streams. The achievable rate expressions and beamforming
matrices of cooperative STIN have been given in Section 4.2.2. We can still use the
SCA-based algorithm to solve P2 . Here, we omit the detailed problem transformation
and optimization framework, which follow the same procedure as that for P1 .

4.4 Robust Joint Beamforming Scheme

Here, we further investigate the beamforming design for RSMA-based coordinated

STIN and cooperative STIN considering satellite channel phase uncertainty. The
CSIT of terrestrial channels is assumed to be perfect. From the satellite channel
model, we can observe that the amplitudes of the channel vector components are
determined by some constant coefficients during the coherence time interval, including
the free space loss, satellite antenna gain and rain attenuation [62]. However, the
satellite channel phases vary rapidly due to a series of time-varying factors, such
as the use of different local oscillators (LO) on-board, the rain, cloud and gaseous
absorption, and the use of low-noise block (LNB) at receivers [11, 62]. Therefore,
within a coherence time interval, the phase of the channel vector from the satellite
to SU-ks at time instant t1 can be modeled as

ϕks (t1 ) = ϕks (t0 ) + eks , (4.75)

where ϕks (t0 ) represents the phase vector, which is estimated at the previous time
instant t0 and fed back to the GW. eks = [eks ,1 , eks ,2 , · · · , eks ,Ns ]T is the phase
uncertainty following the distribution eks ∼ N (0, δ 2 I), with i.i.d Gaussian random
entries. For ease of notation, we can generally indicate ϕks (t1 ) and ϕks (t0 ) by ϕks
and ϕ
bks respectively. Since we assume blueconstant channel amplitudes within the
4.4. Robust Joint Beamforming Scheme 113

coherence time interval, the channel vector from the satellite to SU-ks is written as

fks ⊙ xks = diag b
fks = b fks xks , (4.76)

where xks = exp {jeks } is a random vector. We further assume that the channel

fks and the correlation matrix of xks denoted by Xks = E xks xH
estimate b ks are
 ′
known at the GW [62]. For the interfering channels, by defining ykt = exp jekt
T
and e′kt = e′kt ,1 , e′kt ,2 , · · · , e′kt ,Ns following e′kt ∼ N (0, δ 2 I), the channel vector


from the satellite to CU-kt write as

zkt ⊙ ykt = diag b
zkt = b zkt ykt , (4.77)


zkt and the correlation matrix Ykt = E ykt ykHt
where the channel estimate b are
available at the GW. Hence, we concentrate on the expectation-based robust beam-
forming design. The MMF optimization problem for RSMA-based coordinated
STIN considering satellite phase uncertainty remains the same as P1 in Section
4.3.1, By introducing auxiliary variables q, α = [α1 , · · · , αKt ]T , r = [r1 , · · · , rKs ]T ,
W = {Wc , W1 , · · · , WNs } and P = {Pc , P1 , · · · , PKt }, the original P1 can be
equivalently transformed into semi-definite programming (SDP) form with rank-one
constraints

D1 : max q (4.78)
W,P,csat ,cbs ,q,r,α
Kt
X
s.t. tr (Pc ) + tr (Pkt ) ≤ Pt (4.79)
kt =1
Ns
 X  Ps
Wc + Wi ns ,ns ≤ , ns ∈ Ns (4.80)
i=1
Ns

Wc ⪰ 0, Wns ⪰ 0, ∀ns ∈ Ns (4.81)

Pc ⪰ 0, Pkt ⪰ 0, ∀kt ∈ Kt (4.82)

114 Chapter 4. RSMA for Satellite-Terrestrial Integrated Networks

rank (Wc ) = 1, rank (Wns ) = 1, ∀ns ∈ Ns (4.83)

rank (Pc ) = 1, rank (Pkt ) = 1, ∀kt ∈ Kt (4.84)

(4.29), (4.30), (4.32), (4.33), (4.36) − (4.39)

 Ns  Kt
where Wc = wc wcH , Wns = wns wnHs ns =1
, Pc = p c p H H
c , Pkt = pkt pkt kt =1
. (4.79)
and (4.80) are transmit power constraints. All the rate expressions in this section
are redefined by the Ergodic form R ≜ E {log2 (1 + SINR)}, as the metric of average
robust design. By taking (4.39) as an example, Rksat
s
can be approximated by

Rksat = E log2 1 + γksat



s s

E tr Fks Wµ(ks ) + N
 
P s sat2 
i=1,i̸=µ(ks ) E {tr (Fks Wi )} + σks
≈ log2 PNs sat2
i=1,i̸=µ(ks ) E {tr (Fks Wi )} + σks

tr Fks Wµ(ks ) + N

P s
sat2 
i=1,i̸=µ(ks ) tr Fks Wi + σks

= log2 PNs
sat2
. (4.85)
i=1,i̸=µ(ks ) tr Fks Wi + σks

Note that (4.85) is very tight and has been verified to be theoretically accu-



rate in [103]. Specifically, Fks = diag b fks xks xH bH
ks diag fks . Fks = E {Fks } =



diag b fkHs is defined as the channel correlation matrix, which captures
fks Xks diag b
the expectation over the distribution of phase uncertainty. Based on the approxi-
mated rate expressions, D1 can be rewritten as F1 .

F1 : max q (4.86)
W,P,csat ,cbs ,q,r,α,η,ξ

s.t. ηkbst − ξkbst ≥ αkt log 2, ∀kt ∈ Kt (4.87)

Kt
ηkbs
X
e t ≤ tr (Hkt Pkt ) + tr (Hkt Pj ) +
j=1,j̸=kt
Ns
X
tr Zkt Wi + σkbs2



tr Zkt Wc + t
, ∀kt ∈ Kt (4.88)
i=1
4.4. Robust Joint Beamforming Scheme 115

Kt Ns
bs X
X
eξkt ≥ tr Zkt Wi + σkbs2


tr (Hkt Pj ) + tr Zkt Wc + t
, ∀kt ∈ Kt (4.89)
j=1,j̸=kt i=1

ηksat
s
− ξksat
s
≥ rks log 2, ∀ks ∈ Ks (4.90)
Ns
ηksat
X
tr Fks Wi + σksat2



e s ≤ tr Fks Wµ(ks ) + s
, ∀ks ∈ Ks (4.91)
i=1,i̸=µ(ks )
Ns
sat
X
eξks ≥ tr Fks Wi + σksat2


s
, ∀ks ∈ Ks (4.92)
i=1,i̸=µ(ks )
Kt
X
bs bs
ηc,k t
− ξc,k t
≥ Cjbs log 2, ∀kt ∈ Kt (4.93)
j=1
Kt
bs X
eηc,kt ≤ tr (Hkt Pc ) + tr (Hkt Pj ) +
j=1
Ns
X
tr Zkt Wi + σkbs2



tr Zkt Wc + t
, ∀kt ∈ Kt (4.94)
i=1
Kt Ns
bs
ξc,k
X X
tr Zkt Wi + σkbs2



e t ≥ tr (Hkt Pj ) + tr Zkt Wc + t
, ∀kt ∈ Kt (4.95)
j=1 i=1
Ns
X
sat sat
ηc,k s
− ξc,k s
≥ Cjsat log 2, ∀ks ∈ Ks (4.96)
j=1
Ns
sat
X
ηc,k
tr Fks Wi + σksat2



e s ≤ tr Fks Wc + s
, ∀ks ∈ Ks (4.97)
i=1
Ns
sat
X
ξc,k
tr Fks Wi + σksat2


e s ≥ s
, ∀ks ∈ Ks (4.98)
i=1

(4.30), (4.33), (4.36), (4.38), (4.79) − (4.84)

where η and ξ are the sets of introduced slack variables. The constraints (4.87)-(4.89),
(4.90)-(4.92), (4.93)-(4.95), and (4.96)-(4.98) are respectively the expansions of the
rate constraints (4.37), (4.39), (4.29) and (4.32). Note that (4.89), (4.92), (4.95) and
(4.98) are non-convex with convex left-hand sides, which can be approximated by
the first-order Taylor approximation. Hence, we obtain these approximated linear
116 Chapter 4. RSMA for Satellite-Terrestrial Integrated Networks

constraints

Kt Ns
X X bs[n]
bs[n]
tr Zkt Wi + σkbs2 ≤ eξkt ξkbst − ξkt + 1 ,




tr (Hkt Pj ) + tr Zkt Wc + t
j=1,j̸=kt i=1

(4.99)
Ns
X sat[n]
sat[n]
tr Fks Wi + σksat2 ≤ eξks ξksat



s s
− ξks + 1 , (4.100)
i=1,i̸=µ(ks )
Kt Ns
X X bs[n]
ξc,k bs[n]
tr Zkt Wi + σkbs2 bs




tr (Hkt Pj ) + tr Zkt Wc + t
≤ e t ξc,kt − ξc,kt + 1 ,
j=1 i=1

(4.101)
Ns
X sat[n]
ξc,ks sat[n]
tr Fks Wi + σksat2 sat



s
≤ e ξc,k s
− ξc,k s
+ 1 . (4.102)
i=1

where n represents the n-th SCA iteration. The constraints (4.89), (4.92), (4.95)
and (4.98) belong to generalized nonlinear convex program with high computational
complexity. Following the same method introduced in the previous Section, they
can be represented in linear and SOC forms given by

Kt
X Ns
X
tbs tr Zkt Wi + σkbs2



kt ≤ tr (Hkt Pkt ) + tr (Hkt Pj ) + tr Zkt Wc + t
,
j=1,j̸=kt i=1

(4.103)
q
bs bs bs[n] bs[n] bs[n]
≤ tbs bs



tkt + ηkt − log(tkt ) + 1 2 tkt kt − ηkt + log(tkt ) + 1 , (4.104)
2
Ns
X
tsat tr Fks Wi + σksat2



ks ≤ tr Fks Wµ(ks ) + s
, (4.105)
i=1,i̸=µ(ks )
q
sat[n] sat[n] sat[n]
tsat ηksat ≤ tsat sat



ks + s
− log(tks ) +1 2 tks ks − ηks + log(tks )+1 ,
2

(4.106)
Kt
X Ns
X
tbs tr Zkt Wi + σkbs2



c,kt ≤ tr (Hkt Pc ) + tr (Hkt Pj ) + tr Zkt Wc + t
, (4.107)
j=1 i=1
q
bs[n] bs[n] bs[n]
tbs bs
≤ tbs bs



c,kt + ηc,kt − log(tc,kt ) + 1 2 tc,kt c,kt − ηc,kt + log(tc,kt ) + 1 ,
2

(4.108)
4.4. Robust Joint Beamforming Scheme 117

Ns

X
tsat tr Fks Wi + σksat2


c,ks ≤ tr F ks W c + s
, (4.109)
i=1
q
sat[n] sat[n] sat[n]
tsat sat
≤ tsat sat



c,ks + ηc,k s
− log(tc,ks ) +1 2 tc,ks c,ks − ηc,ks + log(tc,ks ) + 1 .
2

(4.110)

Since rank-one implies only one nonzero eigenvalue, the non-convex constraints
(4.83) and (4.84) can be rewritten by

tr (Wc ) − λmax (Wc ) = 0, tr (Wns ) − λmax (Wns ) = 0, ∀ns ∈ Ns , (4.111)

tr (Pc ) − λmax (Pc ) = 0, tr (Pkt ) − λmax (Pkt ) = 0, ∀kt ∈ Kt , (4.112)

where λmax (X) denotes the maximum eigenvalue of X ⪰ 0. Then, we build a penalty
function to insert these constraints into the objective function (4.86) and obtain

 Ns
X
max q − β [tr (Wc ) − λmax (Wc )] + [tr (Wns ) − λmax (Wns )]
W,P,csat ,cbs ,q,r,α,η,ξ
ns =1
Kt
X 
+ [tr (Pc ) − λmax (Pc )] + [tr (Pkt ) − λmax (Pkt )] . (4.113)
kt =1

β is a proper penalty factor to guarantee the penalty function as small as possible.

(4.113) is nonconcave due to the existence of the penalty function. To tackle this
issue, we adopt an iterative method [63]. By taking tr (Wc ) − λmax (Wc ) as an
example, we have the following inequality

[n]
tr(Wc ) − (vc,max )H Wc vc,max
[n]
≥ tr(Wc ) − λmax (Wc ) ≥ 0, (4.114)

where vc,max is the normalized eigenvector corresponding to the maximum eigenvalue

λmax (Wc ). Furthermore, we define vns ,max as the corresponding eigenvector of
λmax (Wns ), and so does bc,max for λmax (Pc ) and bkt ,max for λmax (Pkt ). Let PF
118 Chapter 4. RSMA for Satellite-Terrestrial Integrated Networks

denote the iterative penalty function

 Ns
[n]

H [n]
 X
H
tr (Wns ) − vn[n]s ,max Wns vn[n]s ,max
 
PF = β tr (Wc ) − vc,max Wc vc,max +
ns =1
Kt  H

H X [n] [n] 
+ tr (Pc ) − b[n] Pc b[n]
  
c,max c,max + tr (Pkt ) − bkt ,max Pc bkt ,max .
kt =1

(4.115)

Eventually, the approximate problem at iteration-n is given by

G1 : max q − PF (4.116)
W,P,csat ,cbs ,q,r,α,η,ξ,t

s.t. (4.30), (4.33), (4.36), (4.38), (4.79) − (4.82), (4.87), (4.90), (4.93), (4.96)

(4.99), (4.101), (4.103), (4.104), (4.107), (4.108), ∀kt ∈ Kt

(4.100), (4.102), (4.105), (4.106), (4.109), (4.110), ∀ks ∈ Ks

The problem is convex involving only linear matrix inequality (LMI) and SOC
constraints, and can be effectively solved by CVX. In each iteration, the problem
defined around the solution of the previous iteration is solved. We summarize
the procedure of this robust joint beamforming scheme in Algorithm 3. Finally,
eigenvalue decomposition can be used to obtain the optimized beamforming vectors.
The optimal solution (W [n] , P [n] , η [n] , ξ [n] , t[n] ) of the n-th iteration is a feasible
solution of the (n + 1)-th iteration. Thus, this algorithm generates a non-decreasing
sequence of objective values, which are bounded above by the transmit power
constraints. Moreover, the objective function is guaranteed to converge by the
existence of lower bounds, i.e., (4.114). In other words, the rank-one constraints
can be satisfied [63]. The obtained solution satisfies the KKT optimality conditions
of G1 , which are indeed identical to those of D1 at convergence [102]. However,
the global optimality of the achieved solution can not be guaranteed. The MMF
optimization problem of RSMA-based cooperative STIN considering satellite phase
4.5. Simulation Results and Analysis 119

uncertainty remains the same as P2 . Here, we still omit the detailed optimization
framework. The process keeps the same as that for the coordinated STIN.

Algorithm 3 Robust Joint Beamforming Scheme

Initialize: n ← 0, W [n] , P [n] , ξ [n] , t[n] ;

repeat
Solve the problem G1 at (W [n] , P [n] , ξ [n] , t[n] ) to get
the optimal solution (W̆ , P̆ , ξ, ˘ t̆, objective);
˘
n ← n + 1;
Update W [n] ← W̆ , P [n] ← P̆ , ξ [n] ← ξ, ˘ t[n] ← t̆, objective[n] ← objective;
˘
[n] [n−1]
until objective − objective <ε;

Remark 4.2: Recall that the problem formulations in Algorithm 2 and Algorithm
3 involve only SOC and LMI constraints. They both can be efficiently solved by
the standard interior-point method. It suggests that the worst-case runtime can be
used to compare the computational complexities of different problems [104]. Hence,
the worst-case computational complexity of the proposed joint beamforming scheme
in Algorithm 2 and the robust joint beamforming scheme in Algorithm 3 are re-
   
spectively O [Ns2 + Nt Kt ]3.5 log (ε−1 ) and O [Ns3 + Nt2 Kt ]3.5 log (ε−1 ) [98, 105],
where ε is the convergence tolerance. Similarly, the complexity of the cooperative

STIN scenarios of Algorithm 2 and Algorithm 3 are respectively O [Ns (Ns + Kt ) +
  
Nt (Ns + Kt )]3.5 log (ε−1 ) and O [Ns2 (Ns + Kt )+Nt2 (Ns + Kt )]3.5 log (ε−1 ) , which
are higher than the coordinated STIN scenarios because of the larger number of
variables in precoder design.

4.5 Simulation Results and Analysis

In this section, simulation results are provided to evaluate the performance of the
proposed joint beamforming algorithms. Both perfect CSIT and imperfect CSIT
with satellite channel phase uncertainties are considered. The tolerance of accuracy
is set to be ε = 10−4 . Channel models have been introduced in Section 4.2.1, and
120 Chapter 4. RSMA for Satellite-Terrestrial Integrated Networks

Table 4.1: Simulation parameters [Chapter 4]

Parameter Value
Frequency band (carrier frequency) Ka (28 GHz)
Satellite height 35786 km (GEO)
Bandwidth 500 MHz
3 dB angle 0.4◦
Maximum beam gain 52 dBi
User terminal antenna gain 42.7 dBi
Rain fading parameters (µ, σ) = (−3.125, 1.591)
UPA inter-element spacing d1 = d2 = λ2
Number of NLoS paths 3

the simulation parameters are listed in Table 4.1 [68, 106]. The satellite is equipped
with Ns antennas. ρ multicasting SUs locate uniformly in each beam coverage area.
According to the architecture of single feed per beam, which is popular in modern
satellites such as Eutelsat Ka-Sat, the number of SUs is Ks = ρNs . Meanwhile,
the BS is deployed with UPA with Nt antennas. We assume Kt CUs are uniformly
distributed within the BS coverage. In the satellite channel model, since we normalize
the noise power by κTsys Bw , we can claim σksat2
s
= σkbs2
t
= 1, ∀ks ∈ Ks , ∀kt ∈ Kt in
4
the simulations. The transmit SNRs can be read from the transmit power Ps and
Pt . All MMF rate curves throughout the simulations are calculated by averaging
100 channel realizations.

At first, we assume that perfect CSI is available at the GW. Fig. 4.3 compares the
MMF rate performance of RSMA-based coordinated and cooperative scheme. The
label “coordinated rsma” means RSMA is adopted at both the satellite and BS,
while “cooperative rsma” means the satellite and BS work cooperatively as a super
transmitter while RSMA is adopted. As Pt grows, we can see that the MMF rates
of both schemes increase and tend to saturate at large Pt region. The cooperative
scheme outperforms the coordinated scheme apparently at low Pt region. The gap
between the two schemes decreases gradually as Pt grows and finally converges to
4
According to the parameters given in Table 4.1 and the satellite channel model, the long-term
received SNR is calculated to be around 0.67 times the transmit SNR.
4.5. Simulation Results and Analysis 121

5.5
coordinated rsma, Ps = 300 W, Ns = 3
5 cooperative rsma, Ps = 300 W, Ns = 3
coordinated rsma, Ps = 120 W, Ns = 3
4.5 cooperative rsma, Ps = 120 W, Ns = 3
coordinated rsma, Ps = 120 W, Ns = 7
cooperative rsma, Ps = 120 W, Ns = 7
MMF Rate (bps/Hz)

3.5

2.5

1.5

1
5 10 15 20 25 30
Pt (dB)

Figure 4.3: MMF rate versus Pt with different Ps and Ns . Nt = 16, Kt = 4,

Ks = ρNs , ρ = 2.

4.5
coordinated rsma rsma, Kt = 4, Ks = 6
cooperative rsma, Kt = 4, Ks = 6
4
coordinated rsma rsma, Kt = 8, Ks = 6
cooperative rsma, Kt = 8, Ks = 6
3.5 coordinated rsma rsma, Kt = 4, Ks = 18
cooperative rsma, Kt = 4, Ks = 18
MMF Rate (bps/Hz)

2.5

1.5

5 10 15 20 25 30
Pt (dB)

Figure 4.4: MMF rate versus Pt with different Ks and Kt . Nt = 16, Ns = 3, ,

Ps = 120W.
122 Chapter 4. RSMA for Satellite-Terrestrial Integrated Networks

the same value when Pt is sufficiently large. The reasons are as follows. When Pt
is relatively small, the STIN’s performance is restricted in the coordinated scheme
because the SINRs of CUs are much lower than the SINRs of SUs. Joint beamforming
is designed to achieve optimal MMF rates. However, in the cooperative scheme, data
exchange is assumed and the satellite can complement the services of BS to serve
CUs, thereby remaining the optimized MMF rate at a higher level than that in the
coordinated scheme. As Pt grows, the benefits of the cooperative scheme compared
with the coordinated scheme decreases. When Pt is sufficiently large, the MMF rates
of both schemes will finally converge to the same value due to the fixed satellite
transmit power budget Ps . We also investigate the influence of different Ps and
Ns setups. Apparently, the larger Ps is, the better MMF rate performance can be
achieved. When Ns is increased from 3 to 7, by keeping ρ = 2, there will be Ks = 14
SUs. We can see that larger Ns leads to lower saturation MMF rates at high Pt
region. The larger Ns is, the less transmit power is allocated to each satellite beam.
Moreover, each SU will see more inter-beam interference due to the existence of
more beams, thus resulting in performance degradation.

Fig. 4.4 depicts the MMF rates versus Pt with different number of SUs and CUs.
When Kt is increased from 4 to 8, the performance will become worse in both
coordinated and cooperative scheme especially at low Pt region, where the CUs take
a dominant position of the system’s MMF rate. On the other hand, when increasing
the number of users per beam ρ from 2 to 6, i.e., from Ks = 6 to Ks = 18, we can still
see the performance degradation in both coordinated and cooperative scheme. The
performance degrades much at high Pt region, where the MMF rate is dominated by
the satellite sub-system.

In Fig. 4.5, to investigate the influence of different transmission strategies in STIN,

we compare the proposed RSMA-assisted beamforming with baseline strategies
including SDMA, NOMA, a two-step beamforming, and fractional frequency reuse.
4.5. Simulation Results and Analysis 123

4.5
coordinated rsma cooperative rsma
4 coordinated rsma sdma cooperative sdma
coordinated sdma rsma two-step beamforming
coordinated sdma fractional frequency reuse
3.5
coordinated noma
MMF Rate (bps/Hz)

2.5

1.5

0.5

0
5 10 15 20 25 30
Pt (dB)

Figure 4.5: MMF rate versus Pt with different transmission strategies. Nt = 16,
Kt = 4, Ns = 3, Ks = 6, Ps = 120W.

4.5
coordinated rsma cooperative rsma
4 coordinated rsma sdma cooperative sdma
coordinated sdma rsma two-step beamforming
3.5 coordinated sdma fractional frequency reuse
coordinated noma
MMF Rate (bps/Hz)

2.5

1.5

0.5

0
5 10 15 20 25 30
Pt (dB)

Figure 4.6: MMF rate versus Pt for different transmission strategies. Nt = 4, Kt = 4,

Ns = 3, Ks = 6, Ps = 120W.
124 Chapter 4. RSMA for Satellite-Terrestrial Integrated Networks

According to the analysis above, here we basically assume Ns = 3, ρ = 2, Ks = 6,

Nt = 16, Kt = 4 for lower computational complexity. Different combinations of
transmission strategies are considered in the coordinated scheme, e.g., the label
“coordinated rsma sdma” means RSMA is used at the satellite while SDMA is used at
the BS. It has been shown in [68] that adopting RSMA compared with SDMA in an
overloaded system can provide more gains than in an underloaded system. Therefore,
in this STIN where the satellite sub-system is always overloaded, and Nt = 16 is large
enough to support an underloaded cellular sub-system, the performance improvement
obtained by using RSMA compared with using SDMA at the satellite is more obvious
than at the BS. As a consequence, the “coordinated rsma” successively outperforms
“coordinated rsma sdma”, “coordinated sdma rsma” and “coordinated sdma”. For
the “coordinated noma noma”, SC-SIC is implemented at both the satellite and BS.
The decoding order of NOMA at the satellite is decided by the ascending order of
the weakest user’s channel strength in each beam. We can observe that the MMF
rate achieved by NOMA is the worst compared with RSMA and SDMA. The low
performance of NOMA in multi-antenna settings is inline with the observations
in [8] and the references therein. As discussed in Fig. 4.3, cooperative schemes can
provide higher MMF rates than the corresponding coordinated schemes. Thus, the
“cooperative rsma” outperforms “coordinated rsma” in Fig. 4.5, and finally, they
tend to reach the same MMF rate restricted by the fixed Ps at very large Pt region.
Similarly, the “cooperative sdma” outperforms “coordinated sdma”. As Pt increases,
they converge to the same value which is lower than that of RSMA. For the two-step
beamforming, both CSI and data are not exchanged between the satellite and BS.
The beamforming for the satellite is at first optimized. Then, the beamforming
for the BS is optimized. Since the satellite beamfoming vectors are not jointly
designed with the BS beamforming vectors, CUs will see serious interference from
the satellite. As Pt grows, the value of minimum rate tends to reach the saturation
MMF rate of RSMA-based coordinated and cooperative schemes. For the scheme
4.5. Simulation Results and Analysis 125

of fractional frequency reuse, the satellite and BS operate on different frequency

bands. The spectrum cannot be effectively used, therefore resulting in poor MMF
rate performance. In [68], it has been demonstrated that the conventional four-color
frequency reuse of multibeam satellite systems performs the worst compared with full
frequency reuse strategies. Thus, we do not compare with the four-color frequency
reuse in this work.

In Fig. 4.6, the number of BS antennas is reduced to Nt = 2 × 2 = 4, which

is not enough to support effective beamforming at the BS so as to eliminate the
intra-cell interference and the satellite interference. Compared with Fig. 4.5 with
Nt = 4×4 = 16 antennas, the MMF rates of all strategies are suppressed. Specifically,
the performance of “coordinated sdma rsma” becomes better than the “coordinated
rsma sdma”. It implies that when Nt is not sufficient to suppress the intra-cell
interference, the gains obtained by using RSMA compared with using SDMA at the
BS can become more obvious than at the satellite. We can conclude that the larger
Nt is, the better MMF rate performance can be achieved. In other words, as Nt
increases, less Pt is required to reach the same MMF rate performance.

Furthermore, we assume imperfect CSI at the GW considering satellite phase

uncertainties. Fig. 4.7 shows the MMF rate performance of the proposed robust
joint beamforming in both RSMA-based coordinated STIN and cooperative STIN.
As the variance of phase uncertainty δ 2 increases, the MMF rates of both schemes
decrease gradually. From perfect CSIT to imperfect CSIT when δ 2 = 5◦ , δ 2 = 15◦ ,
and the phase-blind scenario, the corresponding MMF rates decrease gradually. The
cooperative STIN still outperforms coordinated STIN. For comparison, we consider
the conventional SDMA which performs well amongst the other baseline strategies.

From Fig. 4.8, we can observe that the gaps between perfect CSIT curves and
imperfect CSIT curves become larger compared with the RSMA results in Fig. 4.7.
RSMA is more robust to the channel phase uncertainty than SDMA due to its
126 Chapter 4. RSMA for Satellite-Terrestrial Integrated Networks

4
coordinated rsma, perfect cooperative rsma, perfect
coordinated rsma, 2 = 5° cooperative rsma, 2 = 5°
3.5 coordinated rsma, 2
= 15° cooperative rsma, 2 = 15°
coordinated rsma, blind cooperative rsma, blind

3
MMF Rate (bps/Hz)

2.5

1.5

5 10 15 20 25 30
Pt (dB)

Figure 4.7: MMF rate versus Pt with different satellite phase uncertainties. RSMA
is adopted at the transmitters. Nt = 16, Kt = 4, Ns = 3, Ks = 6, Ps = 120W.

4
coordinated sdma,perfect cooperative sdma,perfect
coordinated sdma, 2 = 5° cooperative sdma, 2 = 5°
3.5 2
coordinated sdma, = 15° cooperative sdma, 2 = 15°
coordinated sdma,blind cooperative sdma,blind

3
MMF Rate (bps/Hz)

2.5

1.5

5 10 15 20 25 30
Pt (dB)

Figure 4.8: MMF rate versus Pt with different satellite phase uncertainties. SDMA
is adopted at the transmitters. Nt = 16, Kt = 4, Ns = 3, Ks = 6, Ps = 120W.
4.6. Summary 127

more flexible architecture to partially decode the interference and partially treat the
interference as noise.

4.6 Summary

In this chapter, we investigate the application of RSMA to STIN considering either

perfect CSI or imperfect CSI with satellite channel phase uncertainties at the GW.
Two RSMA-based STIN schemes are presented, namely the coordinated scheme
relying on CSI exchange and the cooperative scheme relying on both CSI and data
exchange at the GW. MMF optimization problems are formulated while satisfying
transmit power budgets. To tackle the optimization, two iterative algorithms are
respectively proposed. Through simulation results, the superiority of the proposed
RSMA-based schemes for STIN is demonstrated compared with various baseline
strategies. The robustness of RSMA is verified. In conclusion, RSMA is shown very
promising for STIN to manage the interference in and between the satellite and
terrestrial sub-systems.
Chapter 5

RSMA for Integrated Sensing and

Communication Systems

This chapter introduces a general RSMA-assisted ISAC architecture, where the

ISAC platform has a dual capability to simultaneously communicate with downlink
users and probe detection signals to a moving target. To design an appropriate
ISAC waveform, we investigate the RSMA-assisted ISAC beamfoming which jointly
minimizes the CRB of target estimation and maximizes the minimum fairness rate
(MFR) amongst communication users subject to the per-antenna power constraint.
The superiority of RSMA-assisted ISAC is verified through simulation results in
both terrestrial and satellite scenarios. RSMA is demonstrated to be a powerful
multiple access and interference management strategy for ISAC, and provides a
better communication-sensing trade-off compared with the conventional baseline
strategies.

128
5.1. Introduction 129

5.1 Introduction

As the growing number of communication equipments and various types of radars are
placed on satellites, using ISAC waveform design to support simultaneous satellite
communications and sensing becomes very necessary to explore. As introduced
in the previous chapters, RSMA is a flexible and robust interference management
strategy for multi-antenna systems, which relies on linearly precoded rate-splitting
at the transmitter and SIC at the receivers, and has been proven to be promising
for multibeam satellite systems in Chapter 3 and STIN in Chapter 4.

In this chapter, we present an overview of the interplay between RSMA and ISAC.
RSMA-assisted ISAC which facilitates the integration of communications and moving
target sensing is investigated to make better use of the RF spectrum and infras-
tructure. Rather than using the MSE of transmit beampattern approximation
as the radar metric, explicit optimization of estimation performance at the radar
receiver is studied. RSMA-assisted ISAC waveform optimization is for the first
time studied to jointly minimize the CRB of the target estimation and maximize
the MFR amongst all communication users subject to transmit power constraints.
To solve the formulated non-convex problem efficiently, we propose an iterative
algorithm based on SCA to solve the optimization. Simulation results show that
RSMA is very effective for both terrestrial and satellite ISAC systems to manage
the multiuser/inter-beam interference as well as performing the radar functionality.

5.2 System Model

We consider a general RSMA-assisted ISAC, where the antenna array is shared by a

co-located monostatic MIMO radar system and a multiuser communication system as
depicted in Fig. 5.1. The ISAC platform equipped with Nt transmit antennas and Nr
130 Chapter 5. RSMA for Integrated Sensing and Communication Systems

Figure 5.1: Model of an RSMA-assisted ISAC system.

receive antennas simultaneously senses a moving target and serves K downlink single-
antenna users indexed by the set K = {1, · · · , K}. Since RSMA1 is adopted, the
messages W1 , · · · , WK intended for the communication users are split into common
parts and private parts. All common part messages {Wc,1 , · · · , Wc,K } are jointly
encoded into a common stream sc , while all private part messages {Wp,1 , · · · , Wp,K }
are respectively encoded into private streams s1 , · · · , sK . Thus, we can denote
s [l] = [sc [l] , s1 [l] , · · · , sK [l]]T as a (K + 1) × 1 vector of unit-power signal streams,
where l ∈ L = {1, · · · , L} is the discrete-time index within one coherent processing
interval (CPI), and the transmit signal at time index l writes as

X
x [l] = Ps [l] = pc sc [l] + pk sk [l] . (5.1)
k∈K

where P = [pc , p1 , · · · , pK ] ∈ CNt ×(K+1) is the beamforming matrix, which is fixed

within one CPI. If L is sufficiently large, and the data streams are assumed to be
independent of each other, satisfying L1 Ll=1 s [l] s [l]H = IK , the covariance matrix
P

1
One-layer RSMA is considered here for brevity and ease of illustration.
5.2. System Model 131

of the transmit signal can be written as

L
1X
RX = x [l] x [l]H = PPH . (5.2)
L l=1

5.2.1 Sensing Model and Metric

The Nr × 1 reflected echo signal at the radar receiver writes as

yr [l] = Hr x [l] + m [l]

= αej2πFD lT b (θ) aH (θ) x [l] + m [l] , (5.3)

where Hr ∈ CNr ×Nt is the effective radar sensing channel. α stands for the complex
reflection coefficient which is related to the radar cross-section (RCS) of the target.
2vfc
FD = c
denotes the Doppler frequency, with fc and c respectively representing the
carrier frequency and the speed of the light. v is the relative radar target velocity.
T denotes the symbol period.

Note that for a monostatic radar, the direction of arrival (DoA) and the direction
of departure (DoD) are the same, and can be denoted by θ, which is the azimuth
angle. a(θ) ∈ CNt ×1 and b(θ) ∈ CNr ×1 are the transmit and receive steering vectors,
2 2
respectively. m [l] is the AWGN distributed by m [l] ∼ CN (0Nr , σm INr ), with σm
denoting the variance of each entry.

The steering vectors a(θ) and b(θ) can be expressed as a function of the Cartesian
coordinates of the transmit and receive arrays as follows

T
 
j 2π r̄ ,··· ,r̄Nt ] [cos θ,sin θ,0]T
λ [ 1
a(θ) = e , (5.4)
T
2π
[cos θ,sin θ,0]T )
b(θ) = e(−j λ [r1 ,··· ,rNr ] . (5.5)

The matrices [r̄1 , · · · , r̄Nt ] ∈ R3×Nt and [r1 , · · · , rNr ] ∈ R3×Nr have columns rep-
132 Chapter 5. RSMA for Integrated Sensing and Communication Systems

resenting the Cartesian coordinates of the transmit and receive array elements,
respectively.

It is well-known that the CRB serves as a theoretical lower-bound of the variance of

unbiased estimators for parameter estimation [107].

In general, the CRB is inversely proportional to the square root of the product of
the SNR times L, and is valid only (by definition) for high SNR. In this Chapter,
we consider the CRB as the radar sensing performance metric for target estimation
[72, 108]. The CRB matrix can be calculated as CRB = F−1 , where F is the
Fisher information matrix (FIM) for estimating the real-valued target parameters
ξ = [θ, αR , αI , FD ]T given by

 
 Fθθ FθαR FθαI FθFD 
 
 F T R F αR αR F αR αI F αR F 
 θα D
F= . (5.6)
FT
 θαI FαTR αI FαI αI FαI FD 

 
T T T
FθFD FαR FD FαI FD FFD FD

From [107], by denoting µ [l] = yr [l] − m [l], the elements of FIM are expressed by

L
2 nX ∂µ [l]H ∂µ [l] o
[F]i,j = 2 Re , i, j ∈ {1, · · · , 4}, (5.7)
σm l=1
∂ξi ∂ξj

where ξi , ξj are the elements of ξ. By denoting A = b (θ) aH (θ), the derivatives in

(5.7) are expressed as follows

∂µ [l] ∂A
= αej2πFD lT x [l] , (5.8)
∂θ ∂θ
∂µ [l]
= ej2πFD lT Ax [l] , (5.9)
∂αR
∂µ [l]
= jej2πFD lT Ax [l] , (5.10)
∂αI
∂µ [l]
= α (j2πlT ) ej2πFD lT Ax [l] . (5.11)
∂FD
5.2. System Model 133

By substituting (5.8) - (5.11) into (5.7), the elements of the FIM are given by

2 |α|2 L n ∂A ∂A H
o
Fθ,θ = 2
Re tr R X , (5.12)
σm ∂θ ∂θ
2L n
o
FαR ,αR = FαI ,αI = 2 Re tr ARX AH , (5.13)
σm
L
2 |α|2 L n X
2
H

o
FFD ,FD = 2
Re 2πlT tr AR X A , (5.14)
σm l=1
2L n
o
FαR ,αI = 2 Re jtr ARX AH = 0, (5.15)
σm
2L n ∗ ∂A H
o
Fθ,αR = 2 Re α tr ARX , (5.16)
σm ∂θ
2L n ∂A H
o
Fθ,αI = 2 Re α∗ jtr ARX , (5.17)
σm ∂θ
L
2 |α|2 L n X
∂A H
o
Fθ,FD = 2
Re j 2πlT tr AR X , (5.18)
σm l=1
∂θ
L
2L n X
H

o
FαR ,FD = 2
Re αj 2πlT tr AR X A , (5.19)
σm l=1
L
2L n X
H

o
FαI ,FD = 2 Re α 2πlT tr ARX A . (5.20)
σm l=1

Note that [F]i,j are all dependent of RX . As discussed in [109], RX can be designed
appropriately to improve the estimation capability of a MIMO radar by minimizing
the trace, determinant or largest eigenvalue of the CRB matrix.

5.2.2 Communication Model and Metric

At each user side, the received signal is given by

yk [l] = hH
k x [l] + nk [l]
X
= hH H
k pc sc [l] + hk pk sk [l] + nk [l] , ∀k ∈ K. (5.21)
k∈K
134 Chapter 5. RSMA for Integrated Sensing and Communication Systems

where hk ∈ CNt ×1 denotes the channel between the ISAC transmitter and user-k.
2
nk [l] ∼ CN (0, σn,k ) represents the AWGN with zero mean. We assume the noise
2
variance σn,k = σn2 , ∀k ∈ K.

Following the decoding order of RSMA, each user first decodes the common stream
by treating all private streams as noise. The SINR of decoding sc at user-k is
expressed by

2
hH
k pc
γc,k = P 2 , ∀k ∈ K. (5.22)
H 2
i∈K |hk pi | + σn

Rc,k = log2 (1 + γc,k ) is the corresponding achievable rate when assuming Gaussian
signalling. To guarantee that each user is capable of decoding the common stream,
P
we define the common rate as Rc = mink∈K {Rc,k } = k∈K Ck , where Ck is the
rate of the common part of the k-th user’s message. After the common stream is
re-encoded, precoded and subtracted from the received signal through SIC, each
user then decodes its desired private stream. The SINR of decoding sk at user-k is
given by

2
hH
k pk
γk = P 2 , ∀k ∈ K. (5.23)
i∈K,i̸=k |hH 2
k pi | + σn

The achievable rate of the private stream is Rk = log2 (1 + γk ), and the total
achievable rate of user-k, assuming Gaussian signalling, writes as Rk,tot = Ck +
Rk , ∀k ∈ K.

To mitigate multiuser interference, the precoders can be designed to maximize the

MFR, which is defined by

MFR (P) = min (Ck + Rk ) . (5.24)

k∈K

For the baseline strategies, SDMA-assisted ISAC is enabled by turning off the
5.3. ISAC Beamforming Optimization 135

common stream in (5.1). NOMA-assisted ISAC relies on superposition coding at the

transmitter and SIC at each user. The precoders and decoding orders are typically
jointly optimized. By taking a two-user system as an example, and considering the
specific decoding order, where the message of user-1 is decoded before the message
of user-2, user-2 is able to decode messages of both users, while user-1 only decodes
its desired message. Therefore, RSMA boils down to NOMA by encoding W1 into
the common stream sc , encoding W2 into s2 and turning off s1 .

5.3 ISAC Beamforming Optimization

The RSMA-assisted ISAC beamforming matrix can be designed by investigating

the trade-off between communication and radar performance. In this chapter, we
employ the CRB as the radar performance metric, which represents a lower bound
on the variance of unbiased estimators, and employ the MFR as the communication
performance metric to ensure the quality of service.

The optimization problem is formulated to maximize the communication MFR

while minimizing the largest eigenvalue of the CRB matrix, which is equivalent to
maximizing the smallest eigenvalue of FIM. Assuming perfect CSIT, the optimization
problem is written as

 
max min (Ck + Rk ) + λtFIM (5.25)
P,c,tFIM k∈K

s.t. F ⪰ tFIM I4 (5.26)

P 1Nt ×1
diag(PPH ) = (5.27)
Nt
K
X
Rc,k ≥ Ci , ∀k ∈ K (5.28)
i=1

Ck ≥ 0, ∀k ∈ K, (5.29)
136 Chapter 5. RSMA for Integrated Sensing and Communication Systems

where c = [C1 , · · · , CK ]T is the vector of common rate portions. tFIM is the variable
representing the smallest eigenvalue of FIM according to (5.26). I is an identity
matrix (which is of the same dimension as F). λ is the regularization parameter
to prioritize either communications or radar sensing. P denotes the sum transmit
power budget. The constraint (5.27) ensures the transmit power of each antenna
to be the same, which is commonly used for MIMO radar to avoid saturation of
transmit power amplifiers in practical systems. The constraint (5.28) ensures that
the common stream can be successfully decoded by all communication users, and
(5.29) guarantees the non-negativity of all common rate portions.

By defining Pc = pc pH H H
c , Pk = pk pk , Hk = hk hk , the original problem (5.25) - (5.29)

can be equivalently transformed into SDP form with rank-one constraints, which is
given by

max q + λtFIM (5.30)

Pc ,{Pk }K
k=1 ,c,tFIM ,r,q

s.t. F ⪰ tFIM I4 (5.31)

 K
X  P 1Nt ×1
diag Pc + Pk = (5.32)
k=1
Nt

Pc ⪰ 0, Pk ⪰ 0, ∀k ∈ K (5.33)

rank (Pc ) = 1, rank (Pk ) = 1, ∀k ∈ K (5.34)

K
 tr (Hk Pc )  X
log2 1+ P 2
≥ Ci , (5.35)
j∈K tr (Hk Pj ) + σn i=1

Ck ≥ 0, ∀k ∈ K (5.36)
 tr (Hk Pk ) 
log2 1 + P 2
≥ rk , (5.37)
j∈K,j̸=k tr (Hk Pj ) + σn

Ck + rk ≥ q, ∀k ∈ K (5.38)

where r = [r1 , · · · , rK ]T , q are auxiliary variables. The covariance matrix of the

transmit signal is expressed by RX = PPH = Pc + K
P
k=1 Pk .
5.3. ISAC Beamforming Optimization 137

With respect to the equivalent problem (5.30) - (5.38), we can observe that the
rank-one constraints (5.34) and the rate constraints (5.35) and (5.37) are non-convex.
To deal with the non-convexity of rate constraints (5.35) and (5.37), we first rewrite
them by introducing slack variables {ηc,k }K K K K
k=1 , {βc,k }k=1 , {ηk }k=1 , {βk }k=1 as

K
X
ηc,k − βc,k ≥ Ci log 2, ∀k ∈ K, (5.39)
i=1
X
eηc,k ≤ tr (Hk Pc ) + tr (Hk Pj ) + σn2 , ∀k ∈ K, (5.40)
j∈K
X
eβc,k ≥ tr (Hk Pj ) + σn2 , ∀k ∈ K, (5.41)
j∈K

ηk − βk ≥ rk log 2, ∀k ∈ K, (5.42)
X
eηk ≤ tr (Hk Pj ) + σn2 , ∀k ∈ K, (5.43)
j∈K
X
e βk ≥ tr (Hk Pj ) + σn2 , ∀k ∈ K. (5.44)
j∈K,j̸=k

Note that (5.41) and (5.44) are still non-convex with convex left-hand sides which
can be approximated by the first-order Taylor approximation given as follows

X [n]
[n]
tr (Hk Pj ) + σn2 ≤ eβc,k βc,k − βc,k + 1 , ∀k ∈ K,


(5.45)
j∈K
X [n]
[n]
tr (Hk Pj ) + σn2 ≤ eβk βk − βk + 1 , ∀k ∈ K,


(5.46)
j∈K,j̸=k

where n represents the n-th SCA iteration. (5.40) and (5.43) belong to generalized
nonlinear convex program, which leads to high computational complexity. Aiming
at more efficient implementation, we introduce {τc,k }K K
k=1 , {τk }k=1 , and rewrite (5.40)

and (5.43) as

X
τc,k ≤ tr (Hk Pc ) + tr (Hk Pj ) + σn2 , ∀k ∈ K, (5.47)
j∈K
138 Chapter 5. RSMA for Integrated Sensing and Communication Systems

τc,k log (τc,k ) ≥ τc,k ηc,k , ∀k ∈ K, (5.48)

X
τk ≤ tr (Hk Pj ) + σn2 , ∀k ∈ K, (5.49)
j∈K

τk log (τk ) ≥ τk ηk , ∀k ∈ K. (5.50)

The left-hand sides of (5.48) and (5.50) are non-convex, so we compute the first-order
Taylor approximations, which are respectively

[n] [n]
[n]
 [n]


τc,k log τc,k + τc,k − τc,k log τc,k + 1 ≥ τc,k ηc,k , ∀k ∈ K, (5.51)
[n] [n]
[n]
 [n]

τk log τk + τk − τk log τk + 1 ≥ τk ηk ∀k ∈ K. (5.52)

The equivalent SOC forms are

h
[n]

q [n] i [n]

τc,k + ηc,k − log τc,k + 1 , 2 τc,k ≤ τc,k − ηc,k + log τc,k + 1, ∀k ∈ K,
2

(5.53)
h q i
[n]

[n] [n]

τk + ηk − log τk + 1 , 2 τk ≤ τk − ηk + log τk + 1, ∀k ∈ K. (5.54)
2

For the rank-one constraints (5.34), we can build an iterative penalty function to
[n]
insert these rank-one constraints into the objective function. By defining vc,max as
[n]

the the normalized eigenvector corresponding to the maximum eigenvalue λmax Pc ,
 [n] K  [n]
K
and vk,max k=1 as the the normalized eigenvector corresponding to λmax Pk k=1
,
the problem (5.30) - (5.38) can be reformulated by

Y: max q + λtFIM − PF (5.55)

Pc ,{Pk }K
k=1 ,c,tFIM ,r,q,η,β,τ

s.t. (5.31) − (5.33), (5.36), (5.38)

(5.39), (5.42), (5.45), (5.46), (5.47)

(5.49), (5.53), (5.54)

5.3. ISAC Beamforming Optimization 139

where η, β, τ are defined as the sets of introduced slack variables. The iterative
penalty function is expressed by

h
H i
[n] [n]
PF = λpf tr (Pc ) − vc,max Pc vc,max
K h i
X [n]
H [n]
+ tr (Pk ) − vk,max Pk vk,max . (5.56)
k=1

λpf is a proper penalty factor to guarantee the penalty function as small as possible.
Problem (5.55) is convex and can be effectively solved by the CVX toolbox. The
results obtained from the n-th iteration are treated as constants while solving (5.55).

We summarize the procedure of this ISAC beamforming design in Algorithm 4. ε

is the tolerance value. The convergence of Algorithm 4 is guaranteed since the
solution of Problem (5.55) at iteration-n is a feasible solution to the problem at
iteration-n + 1. Finally, eigenvalue decomposition can be used to calculate the
optimized beamforming vectors, and the optimized CRB is obtained accordingly.
Note that Problem (5.55) involves only SOC and LMI constraints, it can be solved
by using interior-point methods with the worst-case computational complexity
O log(ε−1 )[Nt2 (K + 1)]3.5 [17, 105, 110].

Algorithm 4 ISAC Beamforming Optimization

[n]  [n] K
Initialize: n ← 0, Pc , Pk k=1 , β [n] , τ [n] ;
repeat
[n]  [n] K
Solve the problem Y at Pc , Pk k=1 , β [n] , τ [n] to get
 K ˘
the optimal P̆c , P̆k k=1 , β̆, τ̆ , objective;
n ← n + 1;
[n]  [n] K K
Update Pc ← P̆c , Pk k=1 ← P̆k k=1 , β [n] ← β̆, τ [n] ← τ̆ , objective[n] ←

˘
objective;
until objective[n] − objective[n−1] < ε ;
140 Chapter 5. RSMA for Integrated Sensing and Communication Systems

5.4 Simulation Results and Analysis

In this section, the performance of the proposed algorithm is evaluated using

simulation results of both terrestrial and satellite ISAC systems. The performance
of RSMA-assisted ISAC is evaluated in terms of the trade-off between MFR and
Root CRB (RCRB).

First, we consider a terrestrial radar-communication system where the ISAC platform

is equipped with Nt = 8 transmit antennas and Nr = 9 receive antennas. The
system employs a uniform linear array (ULA) with half-wavelength adjacent antenna
spacing. The sum transmit power budget is P = 20 dBm, and the noise power at
2
each user is σm = 0 dBm. The communication channel is set as Rayleigh fading
with each entry drawn from CN ∼ (0, 1). We assume K = 4 communication users,
and the target is located at θ = 45◦ . The relative target velocity is v = 8 m/s. The
number of transmit symbols within one CPI is L = 1024. In Fig. 5.2, the curves of
the trade-off between MFR and RCRB of different target parameters are plotted.
All results are obtained by solving the formulated optimization problem and all
results are averaged over 100 channel realizations. The radar SNR is defined as
SNRradar = |α|2 P/σm
2
= −20 dB. For the baseline strategies, SDMA-assisted ISAC
can be simulated as a special case of RSMA by turning off the common stream. The
decoding order of NOMA-assisted ISAC is the ascending order of channel strengths.
No user grouping is considered. We can observe that when the priority is the
communication functionality, both RSMA-assisted and SDMA-assisted ISAC achieve
similar MFR. As the priority is shifted to sensing, the RSMA-assisted ISAC achieves
a considerably better trade-off compared with SDMA. Similar trade-off performance
can be observed in [82] where the ISAC beamforming was designed by optimizing the
communication and radar metric, namely, WSR and beampattern MSE. From Fig.
5.2, the NOMA-assisted ISAC achieves the poorest trade-off due to the DoF loss in
multi-antenna NOMA [8]. At the leftmost points which correspond to prioritizing
5.4. Simulation Results and Analysis 141

8 8

MFR (bps/Hz)

MFR (bps/Hz)
6 6

4 4

2 2

0 0
6 8 10 0.04 0.06 0.08 0.1 0.12
RCRB 10-3 RCRB

8 8
MFR (bps/Hz)

MFR (bps/Hz)
6 6

4 4

2 2

0 0
0.05 0.1 0.15 0.02 0.03 0.04
RCRB RCRB

Figure 5.2: MFR versus RCRB in a terrestrial ISAC system, (a) θ (◦ ), (b) αR , (c)
αI , (d) FD . Nt = 8, Nr = 9, K = 4, L = 1024, SNRradar = −20 dB.

the radar functionality, the optimized precoders are linearly dependent2 on each
other. Thus, the SDMA-assisted ISAC can no longer exploit spatial DoF provided
by multiple antennas and leads to lower MFR compared with the RSMA-assisted
and NOMA-assisted ISAC which employ SIC at user sides to manage the multiuser
interference.

The sensing capability at the radar receiver is evaluated in Fig. 5.3 in terms of the
target estimation root mean square error (RMSE). Radar subspace-based estimation
algorithms, e.g., [111] can be used to estimate the Doppler frequency, the direction of
the target and the reflection coefficient from the radar received signal. Throughout
the simulations, communication symbols s [l] in (5.1) are generated as random
quadrature-phase-shift-keying (QPSK) modulated sequences, and the precoders are
obtained by solving the formulated ISAC beamforming optimization problem. Fig.
2
From the simulation results, we can observe that the optimized precoders are linearly dependent
on each other. Intuitively, when mostly prioritizing the radar functionality, the optimized precoders
are designed to radiate the highest power towards the target angle.
142 Chapter 5. RSMA for Integrated Sensing and Communication Systems

100 100

RMSE

RMSE
10-2 10-2

-20 -15 -10 -5 -20 -15 -10 -5

Radar SNR (dB) Radar SNR (dB)

100 100
RMSE

RMSE
10-2 10-2

-20 -15 -10 -5 -20 -15 -10 -5

Radar SNR (dB) Radar SNR (dB)

Figure 5.3: Target estimation performance in a terrestrial ISAC system, (a) θ (◦ ),

(b) αR , (c) αI , (d) FD . Nt = 8, Nr = 9, K = 4, L = 1024.

5.3 depicts the RMSE and RCRB with the increase of radar SNR while setting the
MFR of RSMA-assisted and SDMA-assisted ISAC to be 6 bps/Hz. NOMA-assisted
ISAC is not evaluated due to its poor MFR performance and 6 bps/Hz cannot be
satisfied. We can observe that the RMSEs of different target parameters are lower-
bounded by the corresponding RCRBs, and are expected to approach the RCRBs at
high radar SNR regimes. As expected, the RSMA-assisted ISAC always outperforms
SDMA-assisted ISAC in terms of the target parameter estimation performance.

Next, a satellite radar-communication system is considered, where the ISAC satellite

could be a multibeam LEO satellite simultaneously serving single-antenna satellite
users and sensing a moving target within the satellite coverage area. Considering
a single feed per beam architecture, which is popular in modern satellites such as
Eutelsat Ka-Sat, where one antenna feed is required to generate one beam. We
can simply assume ρ = 2 uniformly distributed satellite users in each beam, and
5.4. Simulation Results and Analysis 143

3 3

MFR (bps/Hz)

MFR (bps/Hz)
2 2

1 1

0 0
2 2.5 3 0.015 0.02 0.025 0.030.035
RCRB 10-3 RCRB

3 3
MFR (bps/Hz)

MFR (bps/Hz)
2 2

1 1

0 0
0.02 0.03 0.04 6 8 10 12
RCRB RCRB 10-3

Figure 5.4: MFR versus RCRB in a satellite ISAC system, (a) θ (◦ ), (b) αR , (c) αI ,
(d) FD . Nt = 8, Nr = 9, K = 16, L = 1024, SNRradar = −20 dB.

the multibeam satellite channel model has been discussed in the previous chapters.
K = ρNt = 16 satellite users follow multibeam multicast transmission. Fig. 5.4 shows
the trade-off curves between MFR and RCRB in a multibeam satellite ISAC system.
From Fig. 5.4, the trade-off performance gain provided by RSMA-assisted design
is more obvious than the terrestrial scenario given in Fig. 5.2. The gaps between
RSMA-assisted and SDMA-assisted ISAC can be observed from the rightmost points
which correspond to prioritizing the communication functionality. This is due to
the superiority of using RSMA in an overloaded communication system [68]. Since
NOMA leads to extremely high receiver complexity when the number of users is
large and also a waste of spatial resources in multi-antenna settings, we do not
compare with NOMA-assisted ISAC in this scenario. Above all, we can conclude
that RSMA is a very effective and powerful strategy for both terrestrial and satellite
ISAC systems to manage the multiuser/inter-beam interference as well as performing
the radar functionality.
144 Chapter 5. RSMA for Integrated Sensing and Communication Systems

5.5 Summary

In this chapter, we provide an overview of the interplay between two promising

technologies, namely, RSMA and ISAC. We start from a general RSMA-assisted
ISAC model and introduced the performance metrics for both radar sensing and
communications. Then, we introduce a design example which jointly minimizes
the CRB of target estimation and maximizes MFR amongst communication users
subject to the per-antenna power constraint. Through simulation results, RSMA is
demonstrated to be a very powerful and promising technique for ISAC systems in
both terrestrial and satellite scenarios.
Chapter 6

Conclusion

6.1 Summary of Thesis Achievements

In this thesis, we considered the problems of applying RSMA to non-terrestrial

communication and sensing networks, and addressed a number of optimization
problems and algorithms in various scenarios, namely the multigroup multicast and
multibeam satellite systems, STIN and ISAC systems, which are envisioned to play
key roles in next-generation wireless networks. Simulation results and analysis are
presented to evaluate the performance of all the proposed algorithms.

In Chapter 3, we explored the benefits of adopting RSMA for multigroup/multibeam

multicast in the presence of imperfect CSIT. We considered both underloaded and
overloaded regimes and addressed the problem to achieve max-min fairness. Through
MMF-DoF analysis, RSMA was shown to provide gains in both underloaded and
overloaded regimes compared with the conventional scheme. Then, we formulated
a generic MMF optimization problem and developed a WMMSE algorithm based
on SAA to solve the optimization. Through simulation results, the DoF gains of
RSMA over the conventional scheme were shown to translate into rate benefits. The

145
146 Chapter 6. Conclusion

effectiveness of using RSMA for multigroup multicast and multibeam satellite systems
was demonstrated taking into account CSIT uncertainty and practical challenges in
multibeam satellite systems, such as per-feed transmit power constraints, hotspots,
uneven user distribution per beam and overloaded regimes. The RSMA transmitter
and receiver architecture, PHY layer design and LLS platform were also investigated,
including finite length polar coding, finite alphabet modulation, AMC algorithm,
etc. LLS results showed that RSMA is a very promising MA scheme for practical
implementation in numerous application areas.

In Chapter 4, motivated by the benefits of RSMA presented in Chapter 3, we further

investigated the application of RSMA to STIN considering either perfect CSI or
imperfect CSI with satellite channel phase uncertainties at the GW to manage the
interference within and between both sub-networks. A multiuser downlink framework
was presented for the integrated network where the satellite exploits multibeam
multicast communication to serve SUs, while the terrestrial BS employs UPA and
serves CUs in a densely populated area. RSMA can be used at both the satellite
and the BS to mitigate the interference including inter-beam interference, intra-cell
interference and interference between the two sub-systems. Two RSMA-based STIN
schemes were presented, namely the coordinated scheme and the cooperative scheme.
For the coordinated scheme, the satellite and BS exchanged CSI of both direct and
interfering links at the GW, and coordinated beamforming to manage the interference.
For the cooperative scheme, the satellite and BS exchanged both CSI and data
at the GW. All propagation links (including interfering ones) were exploited to
carry useful data upon appropriate beamforming. MMF optimization problems were
formulated. To tackle the optimization, an iterative algorithm was proposed based
on the SCA approach to reformulating the original problem into an equivalent convex
one, which belongs to a SOCP. Then, we considered imperfect CSIT with satellite
channel phase uncertainty. An expectation-based robust beamforming optimization
algorithm was developed using SCA together with a penalty function. Simulation
6.2. Future Work 147

results demonstrated the superiority and robustness of the proposed RSMA-based

cooperative scheme and coordinated scheme compared with the baseline strategies.
Therefore, RSMA was shown very promising for STIN to manage the interference in
and between the satellite and terrestrial sub-systems.

In Chapter 5, RSMA was extended to the ISAC setup to make better use of the
RF spectrum and infrastructure. We investigated a general RSMA-assisted ISAC
system, where the antenna array is shared by a co-located monostatic MIMO
radar system and a multiuser communication system. The problem addressed
the trade-off between serving multiple downlink communication users and sensing
a moving target. Explicit optimization of estimation performance at the radar
receiver was concerned with. We formulated an RSMA-assisted ISAC beamforming
optimization problem to jointly minimize the CRB of the target estimation and
maximize the minimum fairness rate amongst all communication users subject to
transmit power constraints. An iterative algorithm based on SCA was developed
to solve the optimization. Simulation results demonstrated the benefits of RSMA
for both terrestrial and satellite ISAC systems to manage the multiuser/inter-beam
interference and simultaneously perform the radar functionality.

6.2 Future Work

In conclusion of this thesis, some potential future research directions are listed as
follows:

1) RSMA for SAGIN:

The space-air-ground integrated network (SAGIN), which integrates spaceborne, air-

borne and terrestrial/marine networks has been envisioned to provide heterogeneous
services and seamless network coverage. The spaceborne part consists of diverse
148 Chapter 6. Conclusion

types of satellites and constellations, while the airborne network consists of balloons,
aeroplanes, unmanned aerial vehicles (UAVs), etc. However, due to the spectrum
sharing among these segments, interference becomes one of the major challenges and
advanced interference management schemes are required. RSMA is envisioned to en-
hance the system performance as it leverages two extreme interference management
strategies, namely fully treating interference as noise and fully decoding interference.
In addition to the work addressed in Chapter 4 of this thesis, which focused on the
integration of a GEO satellite and a single terrestrial BS, the integration between
more platforms could be further explored. Compared with satellites and terrestrial
BSs, UAVs enjoy much higher mobility, ease of deployment, coverage extension and
low cost. The challenges are their high mobility and limited battery capacity to fly,
hover and communicate. Facing these practical issues, RSMA has great potential to
tackle these challenges because of its robustness towards CSIT imperfections, and
capability to reduce communication energy consumption. UAVs may act as aerial
BSs, relays or aerial receivers, which present great compatibility with SAGIN to
enhance the network services. Moreover, the interplay of RSMA for SAGIN with
other enablers such as machine learning (ML) is also worth studying to achieve
ubiquitous intelligent connectivity.

2) RSMA-assisted ISAC with mmWave:

The explosive growth of data traffic and the scarcity of spectrum resources have
motivated the investigation of millimeter wave (mmWave) communications. A
number of ISAC scenarios involve mmWave frequencies. The frequency band from
30 GHz to 300 GHz requires massive antennas to overcome path losses. It shows
potentials to achieve high data rates for communication and high resolution for radar
operation due to the huge available bandwidth in the mmWave frequency bands and
multiplexing gains achievable with massive antenna arrays. To reduce the transceiver
hardware complexity and power consumption, hybrid analog-digital (HAD) structure
6.2. Future Work 149

is typically used, which is able to reduce the number of required RF chains and
achieve higher energy efficiency compared to fully digital precoding. HAD precoding
design for ISAC systems at the mmWave band has been investigated in [112, 113] to
provide efficient trade-offs between downlink communications and radar performance.
Inspired by the appealing advantages of RSMA in multi-antenna systems, the benefits
of introducing RSMA and HAD to tackle the multiuser interference in the context of
mmWave communications have been demonstrated in [31,96,114]. As a consequence,
the interplay between RSMA-assisted ISAC and HAD for mmWave is becoming
another interesting research topic.

3) RSMA-assisted ISAC with V2X:

For the coming generation of vehicle-to-everything (V2X) networks, ISAC serves

as a particularly suitable technology aiming to jointly provide high throughput
vehicular communication service and remote sensing service for vehicle localization
and anti-collision detection [115]. The characteristics of vehicular networks include
high mobility, rigorous requirements on transmission latency and reliability, etc.
Recent studies have shown that RSMA is robust against CSIT imperfections resulting
from user mobility and feedback delay in multiuser (Massive) MIMO [116], and it
outperforms existing MA schemes in finite block length regimes [117,118]. Therefore,
RSMA-assisted ISAC has great potential to become a promising research topic for
future V2X networks.

4) RSMA-assisted ISAC with OFDM:

OFDM has been widely investigated as one of the key techniques in wireless net-
works. It was also found to be useful in radar sensing [74]. Due to the promising
application to radar sensing, and the key role in 4G and 5G wireless communication
standards, OFDM waveforms for ISAC systems have been explored in [119, 120].
The benefits of implementing RSMA in a multicarrier communication system have
150 Chapter 6. Conclusion

been demonstrated in [121, 122]. Thus, implementing RSMA in an OFDM-based

ISAC system is worth being investigated as a future direction.
Appendix A

Transceiver modules

The transmitter and receiver architecture for RSMA multigroup multicast is depicted
in Fig. 3.9. Detailed explanations of each module are described as follows:

1) Encoder :

From Fig. 3.9, wc,1 , · · · , wc,M represent all common parts of the group messages,
which are bit vectors of length Kc,1 , · · · , Kc,M . All private parts of the group messages
are denoted by wp,1 , · · · , wp,M , which are bit vectors of length Kp,1 , · · · , Kp,M .
Through the encoder, all common parts wc,1 , · · · , wc,M are jointly encoded into a
common codeword νc of code block length Nc , while the private parts wp,1 , · · · , wp,M
are encoded individually into private codewords νp,1 , · · · , νp,M . The code block
lengths are respectively Np,1 , · · · , Np,M . We consider polar coding for the channel
coding process. The block length of a conventional polar code is expressed as
N = 2n , where n is a positive integer. The polar encoding operation can be written
" #⊗n
1 0
as ν = uGN , where GN = BN . BN is the bit-reversal matrix and ⊗n
1 1
represents the n-fold Kronecker product. u denotes the length-N uncoded bit vector
input to the encoder which consists of K information bits and N − K frozen bits. Let
A ∈ {1, · · · , N } be the set of positions of the information bits, and Ac be the set of

151
152 Appendix A. Transceiver modules

positions of the frozen bits. Therefore, we have A∩Ac = ϕ and A∪Ac = {1, · · · , N }.
Specifically, we can construct the private uncoded bit vectors up,1 , · · · , up,M by setting
up,m,Am = wp,m , ∀m ∈ M. The sets Ap,1 , · · · , Ap,M contain information bit indices
of the private messages. To jointly encode the common information bit vectors,
wc,1 , · · · , wc,M are at first appended into wc = [wc,1 , · · · , wc,M ]. Then, the common
uncoded bit vector uc is constructed by setting uc,Ac = wc , where the set Ac collects
information bit indices of the common message. Values of all frozen bits are fixed
and known by both the encoder and the decoder. After obtaining the codewords νc
and ν1 , · · · , νM , interleavers are adopted before modulation.

2) Modulator :

The interleavered bit vectors ν ′ c , ν ′ 1 , · · · , ν ′ M are respectively modulated into a

common stream sc and multiple private streams s1 , · · · , sM . For a given modulation
scheme with alphabet M and modulation order |M| = 2m , the interleavered bits
′ ′
), i ∈ 0, 1, · · · , N

(νmi+1 , · · · , νmi+m m
− 1 are mapped to a constellation signal
s ∈ M according to the Gray labeling. If a stream s is of length S, its corresponding
code block length is N = mS.

3) AMC Algorithm:

Appropriate modulation schemes and coding parameters are determined by the

AMC algorithm to maximize the system throughput depending on the channel
characteristics. The algorithm uses the Average rates Rc and r1 , · · · , rM obtained
from the MMF optimization problems with assumptions of Gaussian signalling
and infinite block length. The Average rates of the common and private streams
are actually calculated based on the optimized precoders by taking an average of
1000 channel realizations due to the effects of imperfect CSIT. According to each
given Average rate, we first determine a corresponding modulation scheme from a
modulation alphabet set Q. Here, we consider quadrature amplitude modulation
 153

(QAM) schemes including 4-QAM, 16-QAM, 64-QAM and 256-QAM. The set of

feasible modulation schemes for a given rate Rl ∈ Rc , r1 , · · · , rM is given by

n R  o
l
Q Rl , β = M : log2 |M| ≥ min , m′ , M ∈ Q . (A.1)
β

where β is the maximum code rate indicating the proportion of information. m′ is

the logarithm of the highest modulation order, i.e., m′ = 8 for 256-QAM in this

work. For all Rl ∈ Rc , r1 , · · · , rM , the modulation alphabets of the common and
private streams are determined by

Ml = argminM∈Q(Rl ,β ) |M| , ∀l ∈ {c, 1, · · · , M } . (A.2)

Thus, when all the streams are of length S, the code block lengths and code rates
are respectively calculated as

Nl = Slog2 (|Ml |) , ∀l ∈ {c, 1, · · · , M } , (A.3)

l  m
Nl min log R|M
l
l|
, β
2
rl = , ∀l ∈ {c, 1, · · · , M } . (A.4)
Nl

4) Equalizer :

For each user-k ∈ K, MMSE equalizers are used to detect the common and private
M M SE
streams. The common stream equalizer gc,k is calculated by minimising the
MSE εc,k = E |gc,k yk − sc |2 = |gc,k |2 Tc,k − 2R gc,k hH
 
k pc + 1, where Tc,k =
2 2 2
hH + hH + M H
+ σn2 . To minimize the MSEs, we let
P
k pc k pµ(k) j=1,j̸=µ(k) hk pj
∂εc,k
∂gc,k
= 0 and obtain

M M SE −1 pHc hk
gc,k = pH
c hk Tc,k = 2 M 2 . (A.5)
|hH H
P 2
k p c | + j=1 |h k p j | + σ n

After the common stream is reconstructed and subtracted, the private stream equal-
154 Appendix A. Transceiver modules


2
izer gkM M SE is calculated by minimising the MSE εk = E gk yk − hH k pc sc − sk =
2
|gk |2 Tk − 2R gk hH ∂εk
 H
k pµ(k) + 1, where Tk = Tc,k − hk pc . By letting ∂gk = 0, the

MMSE equalizers for private streams writes as

−1
pH
µ(k) hk
gkM M SE = pH
µ(k) hk Tk = PM 2 . (A.6)
j=1 |hH 2
k pj | + σn

5) Demodulator and Decoder :

We use the log-likelihood ratio (LLR) method [32, 123], which is an efficient demod-
ulator in bit-interleaved coded modulation (BICM) systems and is calculated from
the equalized signal for Soft Decision (SD) decoding of polar codes. A conventional
polar decoder is then employed [124]. From Fig. 3.9, it should be noted that signal
reconstruction is performed at the output of the polar decoder. The reconstruction
module is the same as the process at the transmitter to reconstruct a precoded
signal for SIC.
References

[1] B. Clerckx, Y. Mao, E. A. Jorswieck, J. Yuan, D. J. Love, E. Erkip, and

D. Niyato, “A primer on rate-splitting multiple access: Tutorial, myths, and
frequently asked questions,” arXiv preprint arXiv:2209.00491, 2022.

[2] Y. Mao, O. Dizdar, B. Clerckx, R. Schober, P. Popovski, and H. V. Poor,

“Rate-splitting multiple access: Fundamentals, survey, and future research
trends,” IEEE Communications Surveys & Tutorials, 2022.

[3] M. Alodeh, S. Chatzinotas, and B. Ottersten, “Constructive multiuser inter-

ference in symbol level precoding for the MISO downlink channel,” IEEE
Transactions on Signal processing, vol. 63, no. 9, pp. 2239–2252, 2015.

[4] C. Masouros and G. Zheng, “Exploiting known interference as green signal

power for downlink beamforming optimization,” IEEE Transactions on Signal
processing, vol. 63, no. 14, pp. 3628–3640, 2015.

[5] Y. Mao, B. Clerckx, and V. O. Li, “Rate-splitting multiple access for downlink
communication systems: Bridging, generalizing, and outperforming SDMA
and NOMA,” EURASIP journal on wireless communications and networking,
vol. 2018, no. 1, p. 133, 2018.

[6] B. Clerckx, H. Joudeh, C. Hao, M. Dai, and B. Rassouli, “Rate-splitting for

MIMO wireless networks: A promising PHY-layer strategy for LTE evolution,”
IEEE Communications Magazine, vol. 54, no. 5, pp. 98–105, 2016.

155
156 REFERENCES

[7] T. Cover, “Broadcast channels,” IEEE Transactions on Information Theory,

vol. 18, no. 1, pp. 2–14, 1972.

[8] B. Clerckx, Y. Mao, R. Schober, E. A. Jorswieck, D. J. Love, J. Yuan, L. Hanzo,

G. Y. Li, E. G. Larsson, and G. Caire, “Is NOMA efficient in multi-antenna
networks? A critical look at next generation multiple access techniques,” IEEE
Open Journal of the Communications Society, vol. 2, pp. 1310–1343, 2021.

[9] B. Clerckx, Y. Mao, R. Schober, and H. V. Poor, “Rate-splitting unifying

SDMA, OMA, NOMA, and multicasting in MISO broadcast channel: A simple
two-user rate analysis,” IEEE Wireless Communications Letters, 2019.

[10] M. Giordani and M. Zorzi, “Non-terrestrial networks in the 6G era: Challenges

and opportunities,” IEEE Network, vol. 35, no. 2, pp. 244–251, 2020.

[11] M. Á. Vázquez, A. Perez-Neira, D. Christopoulos, S. Chatzinotas, B. Ottersten,

P.-D. Arapoglou, A. Ginesi, and G. Taricco, “Precoding in multibeam satellite
communications: Present and future challenges,” IEEE Wireless Commun.,
vol. 23, no. 6, pp. 88–95, 2016.

[12] A. I. Perez-Neira, M. A. Vazquez, M. B. Shankar, S. Maleki, and S. Chatzinotas,

“Signal processing for high-throughput satellites: Challenges in new interference-
limited scenarios,” IEEE Signal Processing Magazine, vol. 36, no. 4, pp. 112–
131, 2019.

[13] X. Zhu and C. Jiang, “Integrated satellite-terrestrial networks toward 6G:

Architectures, applications, and challenges,” IEEE Internet of Things Journal,
vol. 9, no. 1, pp. 437–461, 2021.

[14] F. Liu, Y. Cui, C. Masouros, J. Xu, T. X. Han, Y. C. Eldar, and S. Buzzi,

“Integrated sensing and communications: Towards dual-functional wireless net-
works for 6G and beyond,” IEEE journal on selected areas in communications,
2022.
REFERENCES 157

[15] B. Lee and W. Shin, “Max-min fairness precoder design for rate-splitting
multiple access: Impact of imperfect channel knowledge,” IEEE Transactions
on Vehicular Technology, vol. 72, no. 1, pp. 1355–1359, 2023.

[16] H. Joudeh and B. Clerckx, “Sum-rate maximization for linearly precoded down-
link multiuser MISO systems with partial CSIT: A rate-splitting approach,”
IEEE Transactions on Communications, vol. 64, no. 11, pp. 4847–4861, 2016.

[17] Y. Mao, B. Clerckx, and V. O. Li, “Rate-splitting for multi-antenna non-

orthogonal unicast and multicast transmission: Spectral and energy efficiency
analysis,” IEEE Transactions on Communications, vol. 67, no. 12, pp. 8754–
8770, 2019.

[18] A. A. Ahmad, H. Dahrouj, A. Chaaban, A. Sezgin, and M.-S. Alouini, “Inter-

ference mitigation via rate-splitting and common message decoding in cloud
radio access networks,” IEEE Access, vol. 7, pp. 80350–80365, 2019.

[19] J. Zhang, B. Clerckx, J. Ge, and Y. Mao, “Cooperative rate splitting for
MISO broadcast channel with user relaying, and performance benefits over
cooperative NOMA,” IEEE Signal Processing Letters, vol. 26, no. 11, pp. 1678–
1682, 2019.

[20] C. Hao, Y. Wu, and B. Clerckx, “Rate analysis of two-receiver MISO broad-
cast channel with finite rate feedback: A rate-splitting approach,” IEEE
Transactions on Communications, vol. 63, no. 9, pp. 3232–3246, 2015.

[21] H. Joudeh and B. Clerckx, “Robust transmission in downlink multiuser MISO

systems: A rate-splitting approach,” IEEE Transactions on Signal Processing,
vol. 64, no. 23, pp. 6227–6242, 2016.

[22] E. Piovano and B. Clerckx, “Optimal DoF region of the K-user MISO BC with
partial CSIT,” IEEE Communications Letters, vol. 21, no. 11, pp. 2368–2371,
2017.
158 REFERENCES

[23] M. Dai, B. Clerckx, D. Gesbert, and G. Caire, “A rate splitting strategy

for massive MIMO with imperfect CSIT,” IEEE Transactions on Wireless
Communications, vol. 15, no. 7, pp. 4611–4624, 2016.

[24] Y. Mao and B. Clerckx, “Beyond dirty paper coding for multi-antenna broad-
cast channel with partial CSIT: A rate-splitting approach,” IEEE Transactions
on Communications, vol. 68, no. 11, pp. 6775–6791, 2020.

[25] M. Caus, A. Pastore, M. Navarro, T. Ramı́rez, C. Mosquera, N. Noels,

N. Alagha, and A. I. Perez-Neira, “Exploratory analysis of superposition
coding and rate splitting for multibeam satellite systems,” in 2018 15th Inter-
national Symposium on Wireless Communication Systems (ISWCS), pp. 1–5,
IEEE, 2018.

[26] H. Joudeh and B. Clerckx, “Rate-splitting for max-min fair multigroup mul-
ticast beamforming in overloaded systems,” IEEE Transactions on Wireless
Communications, vol. 16, no. 11, pp. 7276–7289, 2017.

[27] H. Chen, D. Mi, B. Clerckx, Z. Chu, J. Shi, and P. Xiao, “Joint power and
subcarrier allocation optimization for multigroup multicast systems with rate
splitting,” IEEE Transactions on Vehicular Technology, 2019.

[28] A. Z. Yalcin, M. Yuksel, and B. Clerckx, “Rate splitting for multi-group

multicasting with a common message,” IEEE Transactions on Vehicular
Technology, vol. 69, no. 10, pp. 12281–12285, 2020.

[29] O. Tervo, L.-N. Trant, S. Chatzinotas, B. Ottersten, and M. Juntti, “Multi-

group multicast beamforming and antenna selection with rate-splitting in
multicell systems,” in 2018 IEEE 19th International Workshop on Signal
Processing Advances in Wireless Communications (SPAWC), pp. 1–5, IEEE,
2018.
REFERENCES 159

[30] A. Papazafeiropoulos, B. Clerckx, and T. Ratnarajah, “Rate-splitting to

mitigate residual transceiver hardware impairments in massive MIMO systems,”
IEEE Trans. Veh. Technol., vol. 66, no. 9, pp. 8196–8211, 2017.

[31] M. Dai and B. Clerckx, “Multiuser millimeter wave beamforming strategies

with quantized and statistical CSIT,” IEEE Trans. Wireless Commun., vol. 16,
no. 11, pp. 7025–7038, 2017.

[32] O. Dizdar, Y. Mao, W. Han, and B. Clerckx, “Rate-splitting multiple access for
downlink multi-antenna communications: Physical layer design and link-level
simulations,” in 2020 IEEE 31st Annual International Symposium on Personal,
Indoor and Mobile Radio Communications, pp. 1–6, IEEE, 2020.

[33] L. Yin, O. Dizdar, and B. Clerckx, “Rate-splitting multiple access for multi-
group multicast cellular and satellite communications: PHY layer design and
link-level simulations,” in 2021 IEEE International Conference on Communi-
cations Workshops (ICC Workshops), pp. 1–6, IEEE, 2021.

[34] T. De Cola, A. Ginesi, G. Giambene, G. C. Polyzos, V. A. Siris, N. Fotiou,

Y. Thomas, et al., “Network and protocol architectures for future satellite
systems,” Foundations and Trends® in Networking, vol. 12, no. 1-2, pp. 1–161,
2017.

[35] D. Christopoulos, S. Chatzinotas, and B. Ottersten, “Multicast multigroup

precoding and user scheduling for frame-based satellite communications,” IEEE
Transactions on Wireless Communications, vol. 14, no. 9, pp. 4695–4707, 2015.

[36] J. Wang, L. Zhou, K. Yang, X. Wang, and Y. Liu, “Multicast precoding for
multigateway multibeam satellite systems with feeder link interference,” IEEE
Transactions on Wireless Communications, vol. 18, no. 3, pp. 1637–1650, 2019.
160 REFERENCES

[37] N. D. Sidiropoulos, T. N. Davidson, and Z.-Q. Luo, “Transmit beamforming for

physical-layer multicasting,” IEEE Trans. Signal Processing, vol. 54, no. 6-1,
pp. 2239–2251, 2006.

[38] E. Karipidis, N. D. Sidiropoulos, and Z.-Q. Luo, “Quality of service and

max-min fair transmit beamforming to multiple cochannel multicast groups,”
IEEE Transactions on Signal Processing, vol. 56, no. 3, pp. 1268–1279, 2008.

[39] G. R. Lanckriet and B. K. Sriperumbudur, “On the convergence of the concave-

convex procedure,” in Advances in neural information processing systems,
pp. 1759–1767, 2009.

[40] E. Chen and M. Tao, “ADMM-based fast algorithm for multi-group mul-
ticast beamforming in large-scale wireless systems,” IEEE Transactions on
Communications, vol. 65, no. 6, pp. 2685–2698, 2017.

[41] D. Christopoulos, S. Chatzinotas, and B. Ottersten, “Weighted fair multi-

cast multigroup beamforming under per-antenna power constraints,” IEEE
Transactions on Signal Processing, vol. 62, no. 19, pp. 5132–5142, 2014.

[42] B. Hu, C. Hua, C. Chen, and X. Guan, “Multicast beamforming for wire-
less backhaul with user-centric clustering in cloud-RANs,” in 2016 IEEE
International Conference on Communications (ICC), pp. 1–6, IEEE, 2016.

[43] Z. Xiang, M. Tao, and X. Wang, “Coordinated multicast beamforming in

multicell networks,” IEEE Transactions on Wireless Communications, vol. 12,
no. 1, pp. 12–21, 2012.

[44] M. Tao, E. Chen, H. Zhou, and W. Yu, “Content-centric sparse multicast

beamforming for cache-enabled cloud RAN,” IEEE Transactions on Wireless
Communications, vol. 15, no. 9, pp. 6118–6131, 2016.
REFERENCES 161

[45] M. Sadeghi, E. Björnson, E. G. Larsson, C. Yuen, and T. L. Marzetta, “Max–

min fair transmit precoding for multi-group multicasting in massive MIMO,”
IEEE Transactions on Wireless Communications, vol. 17, no. 2, pp. 1358–1373,
2017.

[46] G. Zheng, S. Chatzinotas, and B. Ottersten, “Generic optimization of linear

precoding in multibeam satellite systems,” IEEE Transactions on Wireless
Communications, vol. 11, no. 6, pp. 2308–2320, 2012.

[47] V. Joroughi, M. Á. Vázquez, and A. I. Pérez-Neira, “Generalized multicast

multibeam precoding for satellite communications,” IEEE Transactions on
Wireless Communications, vol. 16, no. 2, pp. 952–966, 2016.

[48] S. S. Christensen, R. Agarwal, E. De Carvalho, and J. M. Cioffi, “Weighted

sum-rate maximization using weighted MMSE for MIMO-BC beamforming
design,” IEEE Transactions on Wireless Communications, vol. 7, no. 12,
pp. 4792–4799, 2008.

[49] Y. Kawamoto, Z. M. Fadlullah, H. Nishiyama, N. Kato, and M. Toyoshima,

“Prospects and challenges of context-aware multimedia content delivery in
cooperative satellite and terrestrial networks,” IEEE Commun. Mag., vol. 52,
no. 6, pp. 55–61, 2014.

[50] H. Zhang, C. Jiang, J. Wang, L. Wang, Y. Ren, and L. Hanzo, “Multicast

beamforming optimization in cloud-based heterogeneous terrestrial and satellite
networks,” IEEE Trans. Veh. Technol., vol. 69, no. 2, pp. 1766–1776, 2019.

[51] N. U. Hassan, C. Huang, C. Yuen, A. Ahmad, and Y. Zhang, “Dense small

satellite networks for modern terrestrial communication systems: Benefits,
infrastructure, and technologies,” IEEE Wireless Commun., vol. 27, no. 5,
pp. 96–103, 2020.
162 REFERENCES

[52] M. Lin, Z. Lin, W.-P. Zhu, and J.-B. Wang, “Joint beamforming for secure
communication in cognitive satellite terrestrial networks,” IEEE J. Sel. Areas
Commun., vol. 36, no. 5, pp. 1017–1029, 2018.

[53] S. K. Sharma, S. Chatzinotas, and B. Ottersten, “Transmit beamforming for

spectral coexistence of satellite and terrestrial networks,” in 8th Int. Conf.
Cogn. Radio Oriented Wireless Netw., pp. 275–281, 2013.

[54] X. Zhu, C. Jiang, L. Yin, L. Kuang, N. Ge, and J. Lu, “Cooperative multigroup
multicast transmission in integrated terrestrial-satellite networks,” IEEE J.
Sel. Areas Commun., vol. 36, no. 5, pp. 981–992, 2018.

[55] Y. Zhang, L. Yin, C. Jiang, and Y. Qian, “Joint beamforming design and
resource allocation for terrestrial-satellite cooperation system,” IEEE Trans.
Commun., vol. 68, no. 2, pp. 778–791, 2019.

[56] Z. Lin, M. Lin, J.-B. Wang, X. Wu, and W.-P. Zhu, “Joint optimization
for secure WIPT in satellite-terrestrial integrated networks,” in IEEE Global
Commun. Conf. (GLOBECOM), pp. 1–6, 2018.

[57] J. Li, K. Xue, D. S. Wei, J. Liu, and Y. Zhang, “Energy efficiency and traffic
offloading optimization in integrated satellite/terrestrial radio access networks,”
IEEE Trans. Wireless Commun., vol. 19, no. 4, pp. 2367–2381, 2020.

[58] W. Lu, K. An, and T. Liang, “Robust beamforming design for sum secrecy rate
maximization in multibeam satellite systems,” IEEE Trans. Aerosp. Electron.
Syst., vol. 55, no. 3, pp. 1568–1572, 2019.

[59] X. Guo, D. Yang, Z. Luo, H. Wang, and J. Kuang, “Robust THP design for
energy efficiency of multibeam satellite systems with imperfect CSI,” IEEE
Commun. Lett., vol. 24, no. 2, pp. 428–432, 2019.
REFERENCES 163

[60] Z. Lin, M. Lin, J. Ouyang, W.-P. Zhu, A. D. Panagopoulos, and M.-S. Alouini,
“Robust secure beamforming for multibeam satellite communication systems,”
IEEE Trans. Veh. Technol., vol. 68, no. 6, pp. 6202–6206, 2019.

[61] A. Gharanjik, M. B. Shankar, P.-D. Arapoglou, M. Bengtsson, and B. Ottersten,

“Robust precoding design for multibeam downlink satellite channel with phase
uncertainty,” in Int. Conf. Acoust., Speech, Signal Process. (ICASSP), pp. 3083–
3087, 2015.

[62] W. Wang, L. Gao, R. Ding, J. Lei, L. You, C. A. Chan, and X. Gao, “Re-
source efficiency optimization for robust beamforming in multi-beam satellite
communications,” IEEE Trans. Veh. Technol., 2021.

[63] J. Chu, X. Chen, C. Zhong, and Z. Zhang, “Robust design for noma-based
multibeam LEO satellite internet of things,” IEEE Internet Things J., vol. 8,
no. 3, pp. 1959–1970, 2020.

[64] L. Yin, B. Clerckx, and Y. Mao, “Rate-splitting multiple access for multi-
antenna broadcast channels with statistical CSIT,” in Wireless Commun.
Netw. Conf. Workshops (WCNCW), pp. 1–6, 2021.

[65] “Second generation framing structure, channel coding and modulation systems
for broadcasting, interative services, news gathering and other broadband
satellite applications; Part 2: DVB-S2 Extensions (DVB-S2X),” standard,
European Broadcasting Union (EBU), document ETSI EN 302-307-2 V1.1.1,
Oct. 2014.

[66] M. Vazquez, M. Caus, and A. Perez-Neira, “Rate-splitting for MIMO multi-

beam satellite systems,” in WSA 2018; 22nd Int. ITG Workshop Smart
Antennas, pp. 1–6, 2018.
164 REFERENCES

[67] L. Yin and B. Clerckx, “Rate-splitting multiple access for multibeam satellite
communications,” in IEEE Int. Conf. Commun. Workshops (ICC Workshops),
pp. 1–6, 2020.

[68] L. Yin and B. Clerckx, “Rate-splitting multiple access for multigroup multicast
and multibeam satellite systems,” IEEE Trans. Commun., vol. 69, no. 2,
pp. 976–990, 2020.

[69] Z. W. Si, L. Yin, and B. Clerckx, “Rate-splitting multiple access for multigate-
way multibeam satellite systems with feeder link interference,” IEEE Trans.
Commun., vol. 70, no. 3, pp. 2147–2162, 2022.

[70] Z. Lin, M. Lin, T. de Cola, J.-B. Wang, W.-P. Zhu, and J. Cheng, “Support-
ing IoT with rate-splitting multiple access in satellite and aerial integrated
networks,” IEEE Internet Things J., 2021.

[71] Z. Lin, M. Lin, B. Champagne, W.-P. Zhu, and N. Al-Dhahir, “Secure and
energy efficient transmission for RSMA-based cognitive satellite-terrestrial
networks,” IEEE Wireless Commun. Lett., vol. 10, no. 2, pp. 251–255, 2020.

[72] F. Liu, Y.-F. Liu, A. Li, C. Masouros, and Y. C. Eldar, “Cramér-rao bound
optimization for joint radar-communication beamforming,” IEEE Transactions
on Signal Processing, vol. 70, pp. 240–253, 2021.

[73] A. Hassanien, M. G. Amin, Y. D. Zhang, and F. Ahmad, “Dual-function

radar-communications: Information embedding using sidelobe control and
waveform diversity,” IEEE Transactions on Signal Processing, vol. 64, no. 8,
pp. 2168–2181, 2015.

[74] C. Sturm and W. Wiesbeck, “Waveform design and signal processing aspects
for fusion of wireless communications and radar sensing,” Proceedings of the
IEEE, vol. 99, no. 7, pp. 1236–1259, 2011.
REFERENCES 165

[75] F. Liu, C. Masouros, A. P. Petropulu, H. Griffiths, and L. Hanzo, “Joint radar

and communication design: Applications, state-of-the-art, and the road ahead,”
IEEE Transactions on Communications, vol. 68, no. 6, pp. 3834–3862, 2020.

[76] H. Wymeersch, G. Seco-Granados, G. Destino, D. Dardari, and F. Tufvesson,

“5G mmWave positioning for vehicular networks,” IEEE Wireless Communica-
tions, vol. 24, no. 6, pp. 80–86, 2017.

[77] C. Yang and H.-R. Shao, “WiFi-based indoor positioning,” IEEE Communi-
cations Magazine, vol. 53, no. 3, pp. 150–157, 2015.

[78] F. Liu, C. Masouros, A. Li, H. Sun, and L. Hanzo, “MU-MIMO communi-

cations with MIMO radar: From co-existence to joint transmission,” IEEE
Transactions on Wireless Communications, vol. 17, no. 4, pp. 2755–2770, 2018.

[79] F. Liu, L. Zhou, C. Masouros, A. Li, W. Luo, and A. Petropulu, “Toward

dual-functional radar-communication systems: Optimal waveform design,”
IEEE Transactions on Signal Processing, vol. 66, no. 16, pp. 4264–4279, 2018.

[80] C. Xu, B. Clerckx, S. Chen, Y. Mao, and J. Zhang, “Rate-splitting multiple

access for multi-antenna joint communication and radar transmissions,” in
IEEE Int. Conf. Commun. Workshops (ICC Workshops), pp. 1–6, 2020.

[81] C. Xu, B. Clerckx, S. Chen, Y. Mao, and J. Zhang, “Rate-splitting multiple

access for multi-antenna joint radar and communications,” IEEE Journal of
Selected Topics in Signal Processing, vol. 15, no. 6, pp. 1332–1347, 2021.

[82] R. C. Loli, O. Dizdar, and B. Clerckx, “Rate-splitting multiple access for

multi-antenna joint radar and communications with partial CSIT: Precoder
optimization and link-level simulations,” arXiv preprint arXiv:2201.10621,
2022.
166 REFERENCES

[83] O. Dizdar, A. Kaushik, B. Clerckx, and C. Masouros, “Energy efficient dual-

functional radar-communication: Rate-splitting multiple access, low-resolution
DACs, and RF chain selection,” IEEE Open Journal of the Communications
Society, vol. 3, pp. 986–1006, 2022.

[84] M. Caus, A. Pastore, M. Navarro, T. Ramirez, C. Mosquera, N. Noels,

N. Alagha, and A. I. Perez-Neira, “Exploratory analysis of superposition
coding and rate splitting for multibeam satellite systems,” in 2018 15th Inter-
national Symposium on Wireless Communication Systems (ISWCS), pp. 1–5,
2018.

[85] M. Vazquez, M. Caus, and A. Perez-Neira, “Rate splitting for MIMO multi-
beam satellite systems,” in WSA 2018; 22nd International ITG Workshop on
Smart Antennas, pp. 1–6, March 2018.

[86] B. Lee, W. Shin, and H. V. Poor, “Weighted sum-rate maximization for

rate-splitting multiple access with imperfect channel knowledge,” in 2021
International Conference on Information and Communication Technology
Convergence (ICTC), pp. 218–220, 2021.

[87] N. Jindal, “MIMO broadcast channels with finite-rate feedback,” IEEE Trans-
actions on information theory, vol. 52, no. 11, pp. 5045–5060, 2006.

[88] G. Caire, N. Jindal, and S. Shamai, “On the required accuracy of transmitter
channel state information in multiple antenna broadcast channels,” in 2007
Conference Record of the Forty-First Asilomar Conference on Signals, Systems
and Computers, pp. 287–291, IEEE, 2007.

[89] H. Chen and C. Qi, “User grouping for sum-rate maximization in multiuser
multibeam satellite communications,” in ICC 2019-2019 IEEE International
Conference on Communications (ICC), pp. 1–6, IEEE, 2019.
REFERENCES 167

[90] H. Chen, D. Mi, Z. Chu, P. Xiao, Y. Xu, and D. He, “Link-level performance
of rate-splitting based downlink multiuser MISO systems,” in 2020 IEEE
31st Annual International Symposium on Personal, Indoor and Mobile Radio
Communications, pp. 1–5, IEEE, 2020.

[91] S. Vassaki, M. I. Poulakis, A. D. Panagopoulos, and P. Constantinou, “Power

allocation in cognitive satellite terrestrial networks with QoS constraints,”
IEEE Commun. Lett., vol. 17, no. 7, pp. 1344–1347, 2013.

[92] M. Jia, X. Gu, Q. Guo, W. Xiang, and N. Zhang, “Broadband hybrid satellite-
terrestrial communication systems based on cognitive radio toward 5G,” IEEE
Wireless Commun., vol. 23, no. 6, pp. 96–106, 2016.

[93] J. P. Choi and C. Joo, “Challenges for efficient and seamless space-terrestrial
heterogeneous networks,” IEEE Commun. Mag., vol. 53, no. 5, pp. 156–162,
2015.

[94] Y. Zhang, Y. Wu, A. Liu, X. Xia, T. Pan, and X. Liu, “Deep learning-based
channel prediction for LEO satellite massive MIMO communication system,”
IEEE Wireless Communications Letters, vol. 10, no. 8, pp. 1835–1839, 2021.

[95] Y. Zhang, A. Liu, P. Li, and S. Jiang, “Deep learning (DL)-based channel
prediction and hybrid beamforming for LEO satellite massive MIMO system,”
IEEE Internet of Things Journal, vol. 9, no. 23, pp. 23705–23715, 2022.

[96] Z. Li, S. Yang, and T. Clessienne, “A general rate splitting scheme for hybrid
precoding in mmWave systems,” in IEEE Int. Conf. Commun. (ICC), pp. 1–6,
2019.

[97] M. Aloisio and P. Angeletti, “Multi-amplifiers architectures for power recon-

figurability,” in IEEE Int. Vac. Electron. Conf., pp. 1–2, 2007.
168 REFERENCES

[98] Y. Mao, B. Clerckx, and V. O. Li, “Rate-splitting for multi-antenna non-

orthogonal unicast and multicast transmission: Spectral and energy efficiency
analysis,” IEEE Trans. Commun., vol. 67, no. 12, pp. 8754–8770, 2019.

[99] O. Tervo, L.-N. Tran, and M. Juntti, “Optimal energy-efficient transmit

beamforming for multi-user MISO downlink,” IEEE Trans. Signal Process.,
vol. 63, no. 20, pp. 5574–5588, 2015.

[100] K.-G. Nguyen, Q.-D. Vu, M. Juntti, and L.-N. Tran, “Distributed solutions for
energy efficiency fairness in multicell MISO downlink,” IEEE Trans. Wireless
Commun., vol. 16, no. 9, pp. 6232–6247, 2017.

[101] M. S. Lobo, L. Vandenberghe, S. Boyd, and H. Lebret, “Applications of

second-order cone programming,” Linear algebra and its applications, vol. 284,
no. 1-3, pp. 193–228, 1998.

[102] B. R. Marks and G. P. Wright, “A general inner approximation algorithm

for nonconvex mathematical programs,” Operations research, vol. 26, no. 4,
pp. 681–683, 1978.

[103] M. Shao and W.-K. Ma, “A simple way to approximate average robust multiuser
MISO transmit optimization under covariance-based CSIT,” in Int. Conf.
Acoust., Speech, Signal Process. (ICASSP), pp. 3504–3508, 2017.

[104] K.-Y. Wang, A. M.-C. So, T.-H. Chang, W.-K. Ma, and C.-Y. Chi, “Out-
age constrained robust transmit optimization for multiuser MISO downlinks:
Tractable approximations by conic optimization,” IEEE Trans. Signal Process.,
vol. 62, no. 21, pp. 5690–5705, 2014.

[105] Y. Ye, Interior point algorithms: theory and analysis, vol. 44. John Wiley &
Sons, 2011.
REFERENCES 169

[106] Z. Lin, M. Lin, J.-B. Wang, T. De Cola, and J. Wang, “Joint beamforming
and power allocation for satellite-terrestrial integrated networks with non-
orthogonal multiple access,” IEEE J. Sel. Topics Signal Process., vol. 13, no. 3,
pp. 657–670, 2019.

[107] S. M. Kay, Fundamentals of statistical signal processing: Estimation theory.

Prentice-Hall, Inc., 1993.

[108] F. Liu, Y.-F. Liu, C. Masouros, A. Li, and Y. C. Eldar, “A joint radar-
communication precoding design based on Cramér-rao bound optimization,”
in 2022 IEEE Radar Conference (RadarConf22), pp. 1–6, 2022.

[109] J. Li, L. Xu, P. Stoica, K. W. Forsythe, and D. W. Bliss, “Range compression

and waveform optimization for MIMO radar: A Cramér–Rao bound based
study,” IEEE Transactions on Signal Processing, vol. 56, no. 1, pp. 218–232,
2007.

[110] Y. Mao, B. Clerckx, J. Zhang, V. O. Li, and M. A. Arafah, “Max-min fairness

of K-user cooperative rate-splitting in MISO broadcast channel with user
relaying,” IEEE Trans. Wireless Commun., vol. 19, no. 10, pp. 6362–6376,
2020.

[111] H. Ren and A. Manikas, “MIMO radar with array manifold extenders,” IEEE
Transactions on Aerospace and Electronic Systems, vol. 56, no. 3, pp. 1942–
1954, 2020.

[112] F. Liu and C. Masouros, “Hybrid beamforming with sub-arrayed MIMO radar:
Enabling joint sensing and communication at mmWave band,” in ICASSP
2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal
Processing (ICASSP), pp. 7770–7774, 2019.
170 REFERENCES

[113] A. Kaushik, C. Masouros, and F. Liu, “Hardware efficient joint radar-

communications with hybrid precoding and RF chain optimization,” in ICC
2021 - IEEE International Conference on Communications, pp. 1–6, 2021.

[114] O. Kolawole, A. Panazafeironoulos, and T. Ratnarajah, “A rate-splitting

strategy for multi-user millimeter-wave systems with imperfect CSI,” in 2018
IEEE 19th International Workshop on Signal Processing Advances in Wireless
Communications (SPAWC), pp. 1–5, 2018.

[115] F. Liu and C. Masouros, “A tutorial on joint radar and communication

transmission for vehicular networks—Part I: Background and fundamentals,”
IEEE Communications Letters, vol. 25, no. 2, pp. 322–326, 2020.

[116] O. Dizdar, Y. Mao, and B. Clerckx, “Rate-splitting multiple access to mitigate

the curse of mobility in (massive) MIMO networks,” IEEE Transactions on
Communications, vol. 69, no. 10, pp. 6765–6780, 2021.

[117] Y. Xu, Y. Mao, O. Dizdar, and B. Clerckx, “Max-min fairness of rate-splitting

multiple access with finite blocklength communications,” IEEE Transactions
on Vehicular Technology, pp. 1–6, 2022.

[118] Y. Xu, Y. Mao, O. Dizdar, and B. Clerckx, “Rate-splitting multiple access with
finite blocklength for short-packet and low-latency downlink communications,”
IEEE Transactions on Vehicular Technology, vol. 71, no. 11, pp. 12333–12337,
2022.

[119] M. F. Keskin, H. Wymeersch, and V. Koivunen, “MIMO-OFDM joint radar-

communications: Is ICI friend or foe?,” IEEE Journal of Selected Topics in
Signal Processing, vol. 15, no. 6, pp. 1393–1408, 2021.

[120] J. A. Zhang, F. Liu, C. Masouros, R. W. Heath, Z. Feng, L. Zheng, and

A. Petropulu, “An overview of signal processing techniques for joint com-
REFERENCES 171

munication and radar sensing,” IEEE Journal of Selected Topics in Signal

Processing, vol. 15, no. 6, pp. 1295–1315, 2021.

[121] H. Chen, D. Mi, T. Wang, Z. Chu, Y. Xu, D. He, and P. Xiao, “Rate-splitting
for multicarrier multigroup multicast: Precoder design and error performance,”
IEEE Transactions on Broadcasting, vol. 67, no. 3, pp. 619–630, 2021.

[122] O. Dizdar and B. Clerckx, “Rate-splitting multiple access for communications

and jamming in multi-antenna multi-carrier cognitive radio systems,” IEEE
Transactions on Information Forensics and Security, vol. 17, pp. 628–643,
2022.

[123] D. Seethaler, G. Matz, and F. Hlawatsch, “An efficient mmse-based demodu-

lator for MIMO bit-interleaved coded modulation,” in IEEE Global Telecom-
munications Conference, 2004. GLOBECOM ’04., vol. 4, pp. 2455–2459 Vol.4,
2004.

[124] I. Tal and A. Vardy, “List decoding of polar codes,” IEEE Transactions on
Information Theory, vol. 61, no. 5, pp. 2213–2226, 2015.