2.1. Security Threats in Untrusted Relays

In wireless communication systems, untrusted relay nodes present several security challenges that can compromise the confidentiality of the data being transmitted. The presence of these relay nodes, which might act as potential eavesdroppers, complicating secure transmission. These challenges arise from the following key factors:

Risk of Eavesdropping: Untrusted relays, although intended to assist in signal forwarding, can intercept and decode confidential information. As described in various studies, including, untrusted relays can act as eavesdroppers while forwarding information between two parties. The relay may opportunistically exploit its position to decode and intercept sensitive information without the authorization of the legitimate users [12,13].

Relaying Vulnerabilities: In the relay node amplifies the received signal, amplify-and-forward (AF), which includes both the legitimate data and any noise, or decode the received signal DF and forwards it to the destination. However, if the relay is untrusted, it can potentially decode the signal before forwarding it. This issue is compounded in multi-input multi-output (MIMO) systems, where the untrusted relay could have access to multiple data streams, increasing the chances of successful interception [14,15].

PLS Limitations: while PLS is employed to protect transmissions by exploiting the characteristics of the wireless channel, its effectiveness is reduced when the relay itself is untrusted. The relay may still capture the signal, especially if it is equipped with multiple antennas or operates in full-duplex mode, allowing for simultaneous transmission and reception. This setup enhances its eavesdropping capabilities [16,17].

Mitigating these challenges requires advanced techniques, such as adaptive precoding and cooperative jamming, where the legitimate parties deliberately introduce interference to confuse the untrusted relay, thereby preventing it from extracting useful information.

2.2. Challenges and Opportunities of Lossy Channels on Secure Communication

Lossy channels present significant challenges for information transmission, particularly when implementing PLS. The inherent degradation in signal quality due to noise, fading, and interference increases the bit error rate (BER) and complicates the secure transmission of confidential data. One primary issue is that lower signal-to-noise ratios (SNR) result in higher BER, which makes it easier for legitimate receivers or relays to misinterpret the transmitted message. This potentially leads to compromising the integrity of sensitive information. Traditional PLS approaches, such as artificial noise and cooperative jamming injection, often face challenges in these conditions due to their reliance on the stability and quality of the channel. As the channel fluctuates, these methods become less reliable [18].

In systems that use decode-and-forward (DF) relays, lossy channels can introduce errors in the relayed signals, which untrusted relays could exploit to intercept and decode part of the confidential message [19]. AF relays, on the other hand, amplify both the signal and the noise, complicating the implementation of security measures such as cooperative jamming. Moreover, the randomness and unpredictability of fading conditions could sometimes allow the eavesdropper’s channel to outperform that of the legitimate receiver, eventually reducing the secrecy capacity and compromising the system’s security.

Despite these challenges, lossy channels can also be leveraged to enhance PLS. The natural noise and interference in these channels make it harder for eavesdroppers to reconstruct the original message accurately. Typically, the eavesdropper has imperfect channel knowledge, and as a result, receives a more distorted version of the signal compared to the legitimate receiver. This gives the legitimate user, supported by error correction mechanisms, a greater chance of successfully recovering the message while the eavesdropper struggles to do so [20].

Moreover, lossy channels create opportunities to implement controlled interference techniques such as cooperative jamming and artificial noise injection. By introducing targeted noise that selectively degrades the signal quality at the eavesdropper’s receiver without significantly affecting the legitimate receiver, these methods can obscure the eavesdropper’s ability to decode the transmission. In DF relay systems, lossy channels can be used to introduce controlled errors, making it more difficult for eavesdroppers to extract useful information. While this may introduce additional challenges for legitimate receivers, effective error correction techniques can still allow them to recover the message accurately.

3.1. Physical Layer Security in Untrusted Lossy One-Way Relay Networks

In this work, the main focus is on a short-packet transmission system involving three nodes, as illustrated in Figure 2. In this system, a legitimate sender, the source (Alice), communicates confidential data to another legitimate recipient, the destination (Bob), with assistance from an intermediary node (Eve). However, Eve is considered an untrusted relay with low trust level. We consider a single antenna setup due to limitations in power and size.

We consider a Time Division Multiple access (TDMA) scheme, where transmission is divided into two distinct time phases. In the 1st time phase, Alice broadcasts the independent and identically distributed (i.i.d.) message ${\mathbf{b}}_{\mathrm{A}}$ . During the 2nd time phase, the relay node, Eve, attempts to decode ${\mathbf{b}}_{\mathrm{A}}$ and forward it to Bob, utilizing orthogonal transmission. Due to the lossy decode-and-forward (DF) relay setup, Eve’s decoding result, denoted as $\mathbf{b}\mathrm{E}$ , may contain errors. Nevertheless, Eve interleaves ${\mathbf{b}}_{\mathrm{E}}$ , encodes the received message, re-transmits it to the destination Bob. Alice remains inactive in the 2nd time phase.

After receiving signals from both, Alice and Eve, Bob performs joint decoding to recover the original sequence ${\mathbf{b}}_{\mathrm{A}}$ . An iterative decoding process with two decoders is used to retrieve Alice’s original messages. The two decoders exchange the decoded bits’ log-likelihood ratios through an interleaver and de-interleaver.

3.2. Channel Type

The received signals in Bob and Eva are presented by

$P_i^t$ is the transmit power, $n_i$ $(i\in \mathrm{A},\mathrm{E},\mathrm{B})$ and $x_i$ represent the zero-mean additive white Gaussian noise (AWGN) and modulated symbols for the encoded information sequences. ${n^{\prime }}_{\mathrm{B}}$ and ${y^{\prime }}_{\mathrm{B}}$ refer to the noise and received signals during the 2nd time phase.

We assume $\mathbb{E}\left[\right|h_j{|}^2]=1$ and that $h_j$ remains constant during one block ubder the assumption of block-fading. For clarity, symbol indices have been omitted in Equations (20)–(22).

Each of the communication links is assumed to undergo Rayleigh fading. The PDF for the instantaneous SNR ${\gamma }_j$ following Rayleigh distribution is expressed as

3.3. RSP (Reliable-and-secure probability) Reliable-and-Secure Probability (RSP) Definition

The RSP refers to the likelihood that Bob decodes the information sent by Alice successfully, while the untrusted relay Eva experiences an outage. This probability can be formulated as

Here, $P^{\mathrm{E}}\mathrm{out}$ denotes the probability that outage happens at the untrusted relay, while $P^{\mathrm{B}}\mathrm{out}$ represents the outage probability occurs at both the untrusted relay and the destination simultaneously.

To provide a clearer understanding of the proposed RSP metric, a comparative analysis has been included in the Table 1 with other commonly used metrics in physical layer security. This comparison highlights the unique advantages of RSP in addressing finite blocklength constraints, which are critical for short-packet communications in 5G/6G networks.

3.4. Bob and Eva’s Admissible Rate Region

As destination, Bob’s objective is to decode ${\mathbf{b}}_{\mathrm{A}}$ , the signal ${\mathbf{b}}_{\mathrm{E}}$ sent from Eva is regarded as supplementary information for ${\mathbf{b}}_{\mathrm{A}}$ . Let ${\mathbf{b}}_{\mathrm{A}}$ and ${\mathbf{b}}_{\mathrm{E}}$ have transmission rates of $R_1^s$ and $R_2^s$ , respectively. Based on the source coding with helper theorem ([21], Section 10.4), Bob is able to successfully retrieve ${\mathbf{b}}_{\mathrm{A}}$ when $R_1^s$ and $R_2^s$ meet the following conditions

In this context, $H(·|·)$ represents the conditional entropy, while $I(·;·)$ refers to the mutual information between the respective variables. The term $\widehat{\mathbf{b}}\mathrm{E}$ denotes Bob’s estimate of $\mathbf{b}\mathrm{E}$ , with an associated error probability of ${ϵ}_2$ . The rate region defined by Equation (6) is illustrated in Figure 3.

In this paper, we focus on a source that is independently and i.i.d., and thus Equation (6) can be reformulated as

$A\ast B=(1-A)B+(1-B)A$ is the binary convolution. With binary source, $H\left({\widehat{\mathbf{b}}}_{\mathrm{E}}\right)-H\left({\widehat{\mathbf{b}}}_{\mathrm{E}}\right|{\mathbf{b}}_{\mathrm{E}})=1-H\left({ϵ}_2\right)=1-[-{ϵ}_2{log}_2{ϵ}_2-(1-{ϵ}_2){log}_2(1-{ϵ}_2)]$ . For Gaussian source, $H\left({\widehat{\mathbf{b}}}_{\mathrm{E}}\right)-H\left({\widehat{\mathbf{b}}}_{\mathrm{E}}\right|{\mathbf{b}}_{\mathrm{E}})={\displaystyle \frac{1}{2}}log\left(2\pi {\sigma }_E^2\right)-{\displaystyle \frac{1}{2}}log\left(2\pi {\sigma }_{{ϵ}_2}^2\right)$ , where ${\sigma }_E^2$ and ${\sigma }_{{ϵ}_2}^2$ are the variances of ${\widehat{\mathbf{b}}}_{\mathrm{E}}$ and ${ϵ}_2$ , respectively.

In accordance with source-channel separation theorem, if

The error probability can be reduced to an arbitrarily small value. $\acute{R}1$ and $\acute{R}2$ denote the overall source-channel joint coding rates for the source-destination and relay-destination channels, respectively. The terms $C_1^s({\gamma }_1,n_1,{ϵ}_1)$ and $C_2^s({\gamma }_2,n_2,{ϵ}_2)$ refer to secrecy capacities of the respective links when considering a finite blocklength.

3.5. Bob’s Outage Probability

Since we assume block fading, for a given ${\gamma }_0,n_0$ , and ${ϵ}_0$ , an outage occurs during a transmission if ( $R_1^s$ , $R_2^s$ ) lies in the inadmissible rate region shown in Figure 3. Under the assumption, the fading variation varies block by block, the values of $\mathcal{D}$ , ${\gamma }_j$ , $n_i$ , and ${ϵ}_i$ $(j\in 0,1,2)$ vary from block to block. As a result, the Berger-Tung bound, which requires a fixed distortion level, is not applied to calculate the outage probability for the Eva-Bob transmission.

As defined in Equation (24), the probability $P_{\mathrm{out}}^{\mathrm{B}}$ can be expressed as

where

and

These probabilities represent the likelihood that ( $R_1^s$ , $R_2^s$ ) fall outside the admissible rate area can be separated into two distinct regions, 1 and 2, as illustrated in Figure 3.

Note that Equations (10) and (11) do not include the case when ${ϵ}_0=0$ , which indicates the perfect decoding at Eva (no outage). When ${ϵ}_0=0$ , the inadmissible rate region becomes a triangle area as presented in Figure 2 in [22].

The error probabilities ${ϵ}_0$ , ${ϵ}_1$ , ${ϵ}_2$ and the rates $R_0^s$ , $R_1^s$ , $R_2^s$ are related to the corresponding link’s instantaneous SNRs, as shown in Equations (8), (28) and (29). Consequently, $P_1$ and $P_2$ can be written as

and

$H^{-1}(·)$ is the inverse function of $H(·)$ for binary source.

With Gaussian source,

and

The derivation of explicit expressions for Equations (12)–(15) can become highly complicated due to the intricacy of $C_1^s({\gamma }_1,n_1,{ϵ}_1)$ and $C_2^s({\gamma }_2,n_2,{ϵ}_2)$ . As a result, a Monte Carlo simulation is employed to compute $P^{\mathrm{E}}\mathrm{out}$ and $P^{\mathrm{B}}\mathrm{out}$ numerically.

3.6. Optimal Power Allocation

Under the total power constraint $E_T$ , the optimal power allocation corresponds to the point where the maximum RSP is achieved. By normalizing the noise variance for each channel to unity, the average SNR, which corresponds to the transmit power, allocated to Alice and Eva is represented by $k{E}_T$ and $(1-k)E_T$ , respectively, where k ( $0\le k\le 1$ ) represent the ratio for allocated power. Given that $\Gamma 0=k{E}_T$ and $\Gamma 0=(1-k)E_T$ , the expression for $P^{\mathrm{E}}\mathrm{out}$ becomes $P^{\mathrm{E}}\mathrm{out}=F(k{E}_T,;2^{R_0\left({\mathcal{D}}_0\right){\acute{R}}_0}-1)$ . and $P_{\mathrm{out}}^{\mathrm{B}}$ can be written as

The main difficulty is determining the optimal k that maximizes the system’s RSP. This involves improving Bob’s capability to correctly decode Alice’s message while simultaneously reducing Eva’s likelihood of intercepting and decoding the original information. The goal is to strike a balance between these two factors, therefore, optimizing the system’s security. This challenge can be formally represented through maximizing the RSP, and the optimization can be formulated as:

In theory, by calculating the second-order partial derivative of $P_{\mathrm{RSP}}$ with respect to k, the extreme point can be identified, and the convexity can be verified by determining whether the integral results are positive or negative within the range $k\in (0,1)$ . Given the complexity in deriving the explicit expression for $P_{\mathrm{out}}^{\mathrm{B}}$ , a numerical approach is employed to find the optimal power ratio.

3.7. Key Insights for RSP Analysis

The derived expressions for RSP reveal a fundamental trade-off between ensuring reliable communication at the destination and preventing information leakage at untrusted relays. This highlights the importance of balancing these factors through power allocation. The finite blocklength constraint introduces decoding errors at the relays, which can inadvertently improve security by increasing the likelihood of an outage at the untrusted relay nodes. However, it also necessitates the precise power optimization to maintain reliability. The RSP’s dependence on multiple variables, such as blocklength, SNR, and decoding error probabilities, underscores the need for computationally efficient optimization techniques, especially in dynamic networks. These insights offer valuable guidelines for designing secure wireless systems where short-packet communication is essential, such as in uRLLC and mMTC scenarios.

4.1. Short Package Transmission

Channel capacity is often regarded as a fundamental performance metric in traditional wireless communication systems. Typically, performance analyses assume an infinite block-length, which leads to the derived performance limits serving as upper bounds. However, in practical scenarios such as uRLLC and mMTC, the infinite packet-length assumption is no longer appropriate to evaluate the performance of secrecy. Additionally, short-block communications offer a solution for reducing delays, making them ideal for applications which is time-sensitive, by minimizing transmission latency [25].

The maximum rate for finite packet-length (short-block) communications, corresponding to $C_s$ (channel capacity with short-block), is provided by [26]

$C\left(\gamma \right)={log}_2(1+\gamma )$ is the conventional Shannon’s Gaussian channel capacity. n, $ϵ$ and V represent the block-length, the decoding error probability, and the fluctuations is express as

e and $Q^{-1}\left(x\right)$ are Euler’s number and inverse Q-function $Q\left(x\right)$ [27], respectively.

4.2. Channel Model

Let $P_i^t$ $(i\in \mathrm{A},{\mathrm{E}}_1,\mathrm{E}2)$ represent the transmission power of the respective node, and let $G_k$ denote the geometric gains, where $k\in \{1,2,3,4\}$ corresponds to A ${\mathrm{E}}_1$ , A ${\mathrm{E}}_2$ , ${\mathrm{E}}_1$ B, and ${\mathrm{E}}_2$ B links, respectively. The signals received by ${\mathrm{Eva}}_1$ , ${\mathrm{Eva}}_2$ , and Bob are

where

• n represent the symbols’ timing index, under an orthogonal transmission assumption. We omit the symbol indices in the subsequent sections for brevity.

• $x_j$ denotes the coded transmit symbol.

• $n_k$ $(k\in \mathrm{E}1,\mathrm{E}2,\mathrm{B})$ is AWGN with zero-mean and variance of ${\displaystyle \frac{N_0}{2}}$ in each dimension.

• $h_j$ refers to the gain of the corresponding channels. $h_j$ remains constant over the duration of one block under a block-fading assumption, with $\mathbb{E}\left[\right|h_j{|}^2]=1$ .

• ${y^{\prime }}_{\mathrm{B}}$ represent received signal.

• ${n^{\prime }}_{\mathrm{B}}$ indicate AWGN.

Let ${\gamma }_j={\left|h_j\right|}^2{\Gamma }_j$ and $\Gamma j=P_k^r{\displaystyle \frac{E_s}{N_0}}$ $(j\in 1,2,3,4,k\in {\mathrm{E}}_1,{\mathrm{E}}_2,\mathrm{B})$ be the instantaneous and average SNRs at $\mathrm{Eva}1$ , $\mathrm{Eva}2$ , and Bob, respectively, where $E_s$ represents the transmission power for each symbol. It is assumed that all links are subject to i.i.d. Rayleigh fading.

4.3. Definition for Reliable-and-Secure Probability

The objective is to transmit messages to Bob, while ensuring that untrusted relays cannot access the confidential information. A common way is to transmit the information sequences at a rate lower than the secrecy rate [28] from Alice to Bob. However, if there is no direct link between Alice and Bob, it is not possible to achieve the secrecy rate [13].

With lossy DF employed at the relays, decoding errors are permissible at Eva. The decoded message at the Eva are then re-transmit to Bob after re-encoding. Same as the previous section, we define the RSP with the probability that Bob successfully decodes the message sent by Alice, while an outage occurs at ${\mathrm{Eva}}_1$ and ${\mathrm{Eva}}_2$ , as

Here, $P_{\mathrm{out}}^{\mathrm{E}}$ represents the outage probability at both untrusted relays. Meanwhile, $P^{\mathrm{B}}\mathrm{out}$ denotes the outage probability at the Bob and at the, ${\mathrm{Eva}}_1$ and ${\mathrm{Eva}}_2$ , simultaneously.

4.4. Analyses for Relay-Destination Links’ Error Probability

Due to the lossy DF setup, errors are permitted in the untrusted relay-destination links (Alice- ${\mathrm{Eva}}_1$ and Alice- ${\mathrm{Eva}}_2$ ), the rates for the Alice- ${\mathrm{Eva}}_1$ and Alice- ${\mathrm{Eva}}_2$ links can be achieved when

based on the theorem of source-channel separation coding with distortion ([21], Theorem 3.7), with distortions ${\mathcal{D}}_1$ and ${\mathcal{D}}_2$ . $R_1^s\left(\mathcal{D}1\right)$ and $R{2}^s\left({\mathcal{D}}_2\right)$ are the rate-distortion functions for the Alice- ${\mathrm{Eva}}_1$ and Alice- ${\mathrm{Eva}}_2$ links, respectively, with distortion $\mathcal{D}1$ and $\mathcal{D}2$ . The terms $\acute{R}1$ and $\acute{R}2$ denote the source-channel coding total rates.

With binary source following Bernoulli(p) distribution,

Here, $H\left(X\right)=-X{log}_2\left(X\right)-(1-X){log}_2(1-X)$ represents the entropy function.

For Gaussian source,

with distribution $N(0,{\sigma }^2)$ .

It should be noted that the minimum distortions $min{\mathcal{D}}_1$ and $min{\mathcal{D}}_2$ correspond to the transmission error probabilities in Alice- ${\mathrm{Eva}}_1$ and Alice- ${\mathrm{Eva}}_2$ channels, respectively, with the Hamming distortion measure.

Based on Equations (18) and (25), the contact between $R^s\left(\mathcal{D}\right)$ and $\gamma$ can be established with

for Bernoulli( ${\displaystyle \frac{1}{2}}$ ) sources. Whereas for Gaussian sources

4.5. Outage Probabilities at Eva 1 and Eva 2

In short-packet communications, $P_{\mathrm{out}}^{\mathrm{E}}$ in Equation (24) is defined as the scenario where the transmission rates ${\acute{R}}_1$ and ${\acute{R}}_2$ , with block-lengths $n_1$ and $n_2$ , respectively, exceed the tolerable distortion levels ${\mathcal{D}}_1$ and ${\mathcal{D}}_2$ , as

$C_s^{-1}(·,·,·)$ represents $C_s(·,·,·)$ ’ inverse function. Equation (30) establishes based on theorem of Shannon’s source-channel separation and independent fading assumption.

During each fading phase, quasi-static fading channel behaves equivalently to an AWGN channel. The gain of channel remains constant within each phase but varies from phase to phase, based on channel fading distributions. As a result, the outage probability $P_{\mathrm{out}}^{\mathrm{E}}$ is computed with integrals of the probability density function (PDF) of the instantaneous SNRs within the rate region of inadmissibility.

4.6. Analyses for Inadmissible Rate Region for Chief Executive Officer (CEO) Problem with Slepian-Wolf Coding

Let U represent the messages transmitted by Alice, and $U_3$ and $U_4$ denote the messages output by ${\mathrm{Eva}}_1$ and ${\mathrm{Eva}}_2$ , corresponding to rates $R_3^s$ and $R_4^s$ , respectively. The sequences received by ${\mathrm{Eva}}_1$ and ${\mathrm{Eva}}_2$ have errors with a certain probability, meaning ( $U\ne U_3$ $U\ne U_4$ ), according to concept of lossy DF. Despite these errors, ${\mathrm{Eva}}_1$ and ${\mathrm{Eva}}_2$ re-transmit the erroneous messages to Bob. Consequently, the analyses of the ${\mathrm{Eva}}_1$ -Bob and ${\mathrm{Eva}}_2$ -Bob transmissions belongs to the CEO problem category in network information theory [21]. The structure of the CEO encoding is shown in Figure 5. $U_1$ and $U_2$ are omitted in this paper, in order to maintain notation consistency. Under the assumption of block fading, the error probabilities in the Alice- ${\mathrm{Eva}}_1$ and Alice- ${\mathrm{Eva}}_2$ links remain constant throughout a transmission block.

Since $U_3$ and $U_4$ both originate from Alice, $U_3$ and $U_4$ are correlated. We use the bit-flipping model to represent the correlation with: $U_3=U_4\oplus {\epsilon }_0$ , where a random variable ${\epsilon }_0$ is defined $\mathrm{Pr}({\epsilon }_0=1)=1-\mathrm{Pr}({\epsilon }_0=0)={\rho }_0$ . Here, ${\rho }_0$ represents the occurrence probability for flipped bit of $U_3$ and $U_4$ .

According to the Slepian-Wolf theorem, since $U_3$ and $U_4$ have correlation, successful decoding of $U_3$ and $U_4$ can be achieved if the source rates of $U_3$ and $U_4$ , $(R_3^s,R_4^s)$ , satisfy

In this context, ${\widehat{U}}_3$ and ${\widehat{U}}_4$ are the estimates of $U_3$ and $U_4$ , respectively, based on the decoding output at Bob. The terms $H\left(U_3\right|{\widehat{U}}_4)$ and $H\left(U_4\right|{\widehat{U}}_3)$ represent conditional entropy. The relationships between $U_3$ and ${\widehat{U}}_3$ , as well as between $U_4$ and ${\widehat{U}}_4$ , are represent by a bit-flipping pattern. The erroneous probabilities in the ${\mathrm{Eva}}_1$ -Bob and ${\mathrm{Eva}}_2$ -Bob links remain steady during one transmission phase, under the assumption of block fading. Thus, ${\epsilon }_3$ and ${\epsilon }_4$ are treated as constant parameters in each block. We consider i.i.d. source, $H\left(U_3\right|{\widehat{U}}_4)=H({\epsilon }_0\ast {\epsilon }_3)$ and $H\left(U_4\right|{\widehat{U}}_3)=H({\epsilon }_0\ast {\epsilon }_4)$ , where $\alpha \ast \beta =(1-\beta )\alpha +(1-\alpha )\beta$ .

In the case when both $U_3$ and $U_4$ can be completely decoded by Bob, meaning $U_3={\widehat{U}}_3$ and $U_4={\widehat{U}}_4$ , with ${\epsilon }_3=0$ and ${\epsilon }_4=0$ . This scenario corresponds to $(R_3^s,R_4^s)$ falling into regions 1 and 2 in Figure 6. In the second situation, $U_3$ (or $U_4$ ) is decoded with an arbitrarily low error probability, whereas, $U_4$ (or $U_3$ ) is fully erroneous. In this situation, ${\widehat{U}}_4$ (or ${\widehat{U}}_3$ ) does not provide any meaningful information of $U_4$ (or $U_3$ ). The criteria then changes to $\left[R_k^s\ge H\left(U_3\right),R_k^s\ge 0\right]$ ( $k\in \{3,4\}$ ), which corresponds to region 3 and 4, as shown in Figure 6.

4.7. Formulation of CEO Problem

For a binary source, $H\left(U_3\right)=H\left(U_4\right)=1$ , $H\left(U_3\right|U_4)=H\left(U_4\right|U_3)=H\left({\rho }_0\right)$ , and $H(U_3,U_4)=H\left(U_4\right)+H\left(U_3\right|$ $U_4)=H\left(U_3\right)+H\left(U_4\right|U_3)=1+H\left({\rho }_0\right)$ , where $H\left({\rho }_0\right)=-{\rho }_0{log}_2\left({\rho }_0\right)-(1-{\rho }_0){log}_2(1-{\rho }_0)$ is the binary entropy function. It is important to note that ${\rho }_0$ is related to ${\mathcal{D}}_1$ and ${\mathcal{D}}_2$ . When distortion does not exist in the Alice- ${\mathrm{Eva}}_1$ (Alice- ${\mathrm{Eva}}_2$ ) transmissions, i.e., ${\mathcal{D}}_1=0$ and ${\mathcal{D}}_2=0$ , $U_3$ and $U_4$ become identical, meaning ${\rho }_0=0$ .

For a Gaussian source $N\sim (0,{\sigma }^2)$ , we have $h\left(U_3\right)=h\left(U_4\right)={\displaystyle \frac{1}{2}}{log}_{2}2\pi e{\sigma }^2$ , $h\left(U_4\right|U_3)=h(U_4,U_3)-h\left(U_3\right)$ ( $h\left(U_3\right|U_4)=h(U_3,U_4)-h\left(U_4\right)$ ). Here, $h(·)$ , $h(·,·)$ , and $h(·|·)$ represent differential entropy, differential entropy, and conditional entropy.

Let ${\mathcal{D}}_3$ and ${\mathcal{D}}_4$ denote the distortion levels for $\mathrm{Pr}(U_3\ne {\widehat{U}}_3)$ and $\mathrm{Pr}(U_4\ne {\widehat{U}}_4)$ , respectively. Since the lowest distortion $min\left\{{\mathcal{D}}_3\right\}$ and $min\left\{{\mathcal{D}}_4\right\}$ are the same as $p_3$ and $p_4$ , the Hamming distortion of L symbols is represented as

to assess the probability of error propagation using

$\delta$ represents a positive number that is arbitrarily smaller, and $E[·]$ denotes the expectation. Eventually, Bob estimates the message from Alice, by utilizing either majority decoding ([29], Section 4.1) or the optimal decision method [30].

4.8. Outage Probability at Bob

$P_{\mathrm{out}}^{\mathrm{B}}$ in Equation (24) is represented as

$\tilde{\mathcal{D}}=f(·,·)$ represents a function for sequential decoding associated with the decoding scheme. For a detailed explanation of the function $f(·,·)$ , readers may refer to ([30], IV-A).

5.1. Reliable-and-Secure Probability Calculation

Since Alice-Eva transmission error is allowed, the ${\mathrm{Eva}}_1$ -Bob ( ${\mathrm{Eva}}_2$ -Bob) transmission is equal to a CEO problem. With $P_k$ $(k\in 1,2,3,4)$ defining the probability that the rate $(R_3^s,R_4^s)$ falls into region k in Figure 6 for a certain value of $\gamma$ , n, and $ϵ$ . Then, $P_{\mathrm{out}}^{\mathrm{B}}$ is defined as

Please note that, the outage happens when the distortion $\tilde{\mathcal{D}}$ outstrips $min\left\{{\mathcal{D}}_1,{\mathcal{D}}_2\right\}$ as denoted in Equation (34). Hence, we can also treat regions 3 and 4 in Figure 6 as admissible regions if $R_3^s\ge H\left(U_3\right)$ and $R_4^s\ge H\left(U_3\right)$ . Distortion level can satisfy $D_1\le p_1$ and $D_2\le p_2$ , respectively.

When connect the rates $R_3^s$ and $R_4^s$ with the instantaneous channel SNRs ${\gamma }_3$ and ${\gamma }_4$ , block-lengths $n_3$ and $n_4$ , and error probabilities ${ϵ}_3$ and ${ϵ}_4$ , $P_k$ $(k\in 1,2,3,4)$ can be defined by averaging over the transmissions in ${\mathrm{Eva}}_1$ -Bob and ${\mathrm{Eva}}_2$ -Bob channels,

and

The derivation of the explicit expressions for Equations (36)–(39) may prove to be extremely challenging, due to the computational complexity of $C_s(\gamma ,n,ϵ)$ . However, we introduce Monte Carlo simulations to compute $P^{\mathrm{E}}\mathrm{out}$ and $P^{\mathrm{B}}\mathrm{out}$ numerically.

$P_{\mathrm{out}}^{\mathrm{B}}$ is also influenced by the levels of ${\mathcal{D}}_1$ and ${\mathcal{D}}_2$ , which vary with changes in ${\gamma }_1$ and ${\gamma }_2$ , respectively, from phase to phase according to the assumption of block fading.

Figure 7 illustrates the relationship between the RSP and the average SNR, with varying values of error probability $ϵ$ as a key parameter. Additionally, reliable-and-secure probabilities are shown for the infinite block length scenario as a comparative reference. The infinite block length performance curves are also influenced by $ϵ$ (error probability) due to the utilization of rate-distortion function. When $ϵ=0.5$ , the RSP remains identical for both short packets and infinite block length, reflecting the same performance. Here, $ϵ$ is specified as the BSC’s crossover probability; thus, for $ϵ=0.5$ , the transmitted message sent from Alice and that received at Eva are entirely uncorrelated. The capacities of the short packet and Gaussian channels converge as $ϵ$ approaches 0.5. When $ϵ\ne 0.5$ , the RSP for short packets is lower than that achieved RSP with infinite block length. Whereas, for a fixed n, the secrecy capacity increases consistently with rising values of $ϵ$ from 0 to 0.5. Consequently, by allowing for a degree of decoding inaccuracy, a higher RSP can be attained in short packet transmissions.

As observed in Figure 7, the RSP with short packets initially increases and subsequently declines as the average SNR increases. This trend reveals that enhancing the SNR does not always lead to a higher RSP. With increasing SNR, both the first and second terms $C^s(\gamma ,n,ϵ)$ in Equation (18) increase simultaneously. Furthermore, an increase in the SNR of the Alice-Eva(s) links leads to higher probabilities of correct decoding at the relays, ultimately decreasing the total RSP. However, the RSP attains the peak value when the Alice-Eva and Eva-Bob link contributions are balanced.

5.2. Relays Location with RSP

We focus on the relationship between the untrusted relay locations and the RSP. We reformulate the RSP equation as functions of the relay positions by incorporating the geometric gain. With $d_k$ $(k\in 1,2,3,4)$ representing the length of the each channel, the SNR is proportional to the channel length inversely, as

$\rho$ represents the path loss exponent Suppose, the distance between Alice and Bob is $d_0$ and an average SNR of ${\Gamma }_0$ . We have each link’s average SNRs as

By substituting Equation (41) into Equations (36)–(39), we obtain the RSP expressions as functions of the untrusted relay positions. Comprehensive optimization has been formulated with respect to $d_j$ as:

5.3. Optimization Strategy with Cooperative Jamming

Here, we demonstrate an approach to improve the RSP with allocating transmission power between Alice, ${\mathrm{Eva}}_1$ optimally, ${\mathrm{Eva}}_2$ , and Bob, under an overall transmission power restriction $En$ condition. Firstly, the variance of noise in each link to a unity has been normalized. Hence, the transmit power is equal to the average SNR for S, ${\mathrm{UR}}_1$ , ${\mathrm{UR}}_2$ , and D is denoted as $\alpha En$ , $(1-\alpha \beta )E_T$ , and $(1-\alpha )(1-\beta )En$ , respectively. Here, $\alpha$ ( $0<\alpha \le 1$ ) and $\beta$ ( $0\le \alpha \le 1$ ) are the ratios for power allocation. Since the two untrusted relays are symmetric, ${\mathrm{Eva}}_1$ and ${\mathrm{Eva}}_2$ are treated as a single entity in the power allocation analysis.

The main challenge is determining the optimal values for $\alpha$ and $\beta$ that maximize the system’s RSP. This involves improving the ability of Bob to recover the message transmitted by the source in a reliable way, whereas, reducing the potential for untrusted relays to intercept and decode the information sequences, simultaneously. The goal is to optimize the security of the transmission by effectively settling these two respects, as represented in the RSP maximization metric formally.

Then, we formulate the optimization issue to:

Notionally, by computing the partial derivatives to $\alpha$ and $\beta$ , we can identify the apices of $P_{\mathrm{RSP}}$ , and determine the eigenvalues and determinant with respect to Hessian matrix. However, due to the complexity in deriving an explicit expression for $P_{\mathrm{RSP}}$ , a numerical approach is employed for searching the optimal allocated power ratio.

6.1. Implementation Considerations and Challenges

The optimization of RSP involves solving non-linear equations with multiple constraints, which may require significant computational resources. Real-time deployment demands efficient algorithms capable of rapid convergence. Variations in channel conditions, such as fading and mobility, require the optimization process to adapt dynamically. Traditional methods may not respond effectively to such rapid changes, necessitating adaptive approaches. Deploying these strategies on resource-constrained devices (e.g., IoT sensors) may pose challenges due to limited processing power and energy availability. Lightweight approximations or machine learning models could mitigate these issues.

To validate the proposed RSP metric under practical conditions, future work will leverage software-defined radios (SDRs) to simulate real-world environments. SDRs will allow testing of dynamic network scenarios, including varying relay positions, finite blocklength constraints, and power allocation strategies, while accounting for hardware-induced noise and interference. A testbed using platforms like GNU Radio and USRP will emulate the roles of source, untrusted relay, and destination to measure RSP and optimize performance. Insights from these experiments will guide the design of full-scale field tests, ensuring the robustness and practicality of RSP-based strategies for 5G/6G applications.

6.2. Open Issues in Secure Relay Networks

There are several unsolved challenges in secure relay networks, particularly when it comes to achieving efficient and secure communication in the presence of untrusted relays. One major issue is the development of efficient relay selection algorithms. Current methods often do not account for the complexity of the environment, such as varying channel conditions and the dynamic behavior of untrusted nodes. Additionally, the optimal relay selection under power and security constraints remains computationally intensive, especially when the network grows in scale.

Another critical open issue is the implementation of PLS in complex environments, such as those characterized by block fading, short-packet communications, or high mobility (e.g., vehicular networks). These scenarios introduce new challenges for maintaining high secrecy performance due to rapid channel variations and limited transmission opportunities. Addressing these issues requires more adaptive and efficient approaches that can dynamically optimize security and reliability in real-time.

6.3. Future Directions in PLS

The future of PLS lies in expanding its applicability and robustness in more demanding and dynamic environments. One key direction is the development of adaptive power allocation strategies for secure communication. As outlined in the optimization problems discussed earlier, balancing transmit power between trusted and untrusted nodes while maximizing RSP is non-trivial. Future research should focus on efficient numerical methods or approximate closed-form solutions for real-time optimization, especially in the presence of finite block-length constraints.

Machine learning (ML) and AI-based approaches are likely to play a significant role in advancing PLS. These techniques could be leveraged to predict channel conditions and optimize secure communication strategies dynamically. The integration of reinforcement learning to adaptively select relay nodes and power allocations based on real-time network conditions is another direction worth exploring.

Reinforcement learning algorithms could dynamically optimize power allocation and relay selection by continuously learning from real-time network conditions. These methods can significantly reduce computational overhead while adapting to channel variations, making them suitable for high-mobility scenarios [31,32].

Non-orthogonal multiple access (NOMA) is another key area for future research, particularly in improving spectral efficiency and security for short-packet communications. The integration of PLS with NOMA schemes could further enhance secure communications in highly dense IoT environments, which are expected to proliferate with the advent of 6G. Combining PLS with NOMA could improve spectral efficiency and security in densely connected environments like IoT networks.

As the adoption of emerging technologies like 5G/6G and edge computing accelerates, physical layer security must evolve to keep pace with the increasing complexity and demands of these systems. One key research direction is integrating PLS with edge computing to offload security-sensitive computations and leverage distributed resources for enhanced secrecy. This could involve designing secure offloading mechanisms that ensure confidentiality even when processing data near untrusted nodes or networks.