Distributed Control of Discrete-Time Linear Multi-Agent Systems With Optimal Energy Performance

1 Introduction

Inspired by biological motion in nature, the cooperative motion of multi-agent systems (MASs) has been studied extensively in the past decade (Wang et al., 2017, 2019; Wang and Sun, 2018; Wang et al., 2020b; Koru et al., 2021; Wang and Sun, 2021). Compared to a single agent, networked MASs have the advantages of fast command response and robustness. Due to the distributed network computing system having the characteristics of strong scalability and fast computing speed, the study of distributed cooperative control problems for multi-agent systems has attracted increasing attention of control scientists and robotics engineers by virtue of its extensive applications in many cases, such as mobile robots (Mu et al., 2017; Zhao et al., 2019), autonomous underwater vehicles (AUVs) (Zuo et al., 2016; Li et al., 2019), and spacecrafts (Zhang et al., 2018; 2021a). A classical framework for the cooperative control of MASs with switching topologies is discussed in the study by Olfati-Saber and Murray (2004). Ren and Beard (2005) have further relaxed the conditions given by Olfati-Saber and Murray (2004), which present some new results with regard to the consensus of linear MASs.

In practice, it is necessary to investigate the control problem of multi-agent systems in discrete time with most computer systems being discrete structures. In the study by Liang et al. (2017), the cooperative containment control problem of a nonuniform discrete-time linear multi-agent is studied, and a novel internal mode compensator is designed to deal with the uncertain part of system dynamics. A solution method of the discrete-time MAS decentralized consensus problem based on linear matrix inequality (LMI) is given in the study by Liang et al. (2018). The problem of multi-agent consensus control based on the terminal iterative learning framework is discussed by Lv et al. (2018) where an adaptive control method based on time-varying control input is proposed to improve the control performance of the system. Su et al. (2019) proposed a distributed control algorithm based on the low-gain feedback method and the modified algebraic Riccati equation to achieve a semi-global consensus of discrete-time MASs under input saturation conditions. A multi-agent consensus framework based on the distributed model predictive controller is proposed by Li and Li (2020), while the self-triggering mechanism is adopted to reduce communication cost and solve the problem of asynchronous discrete-time information exchange. Liu et al. (2020) proposed a distributed state feedback control algorithm based on the Markov state compensator to solve the problem, that is, some followers cannot directly obtain the leader’s own state information. For MASs with unknown system parameters, a distributed adaptive control protocol containing local information was designed by Li et al. (2020) to ensure the inclusiveness of the system. In the study by Li et al. (2021), a class of discrete-time MASs adaptive fault-tolerant tracking problem based on reinforcement learning is studied, in which an adaptive auxiliary signal variable is designed to compensate the effect of actuator faults on the control system.

In practical applications, the energy cost performance of the designed protocols should be considered carefully, especially for the systems with low loadability, for example, autonomous underwater vehicles and spacecrafts. In the study by Zhang et al. (2017), the discrete-time MAS optimal consensus problem is discussed, and a data-driven adaptive dynamic programming method is proposed to solve the problem, that is, it is difficult to obtain an accurate mathematical model of the system. Wen et al. (2018) constructed a reinforced learning framework based on fuzzy logic system (FLS) approximators for the identifier–actor–critic system to achieve optimal tracking control of MASs. An optimal signal generator is presented in the study by Tang et al. (2018), where an embedded control scheme by embedding the generator in the feedback loop is adopted to realize the optimal output consensus of multi-agent networks. Tan (2020) transformed the distributed H _*∞* optimal tracking problem of a class of physically interconnected large-scale systems with a strict feedback form and saturated actuators into the equivalent control problem of MASs; meanwhile, a feedback control algorithm is designed to learn the optimal control input of the system. In the study by Wang et al. (2020a), the optimal consensus problem of MASs is decomposed into three sub-problems: input optimization, consensus state optimization, and dual optimization, and a distributed control algorithm is proposed to achieve the optimal consensus of the system. The nonuniform MAS distributed optimal steady-state regulation is investigated in the study by Tang (2020), and the results are extended to the case where the system only has real-time gradient information using high gain control techniques. The single-agent goal representation heuristic dynamic programming (GrHDP) technique is extended to the multi-agent consensus control problem in by Zhong and He (2020), and an iterative learning algorithm based on GrHDP is designed to make the local performance indexes of the system converge to the optimal value. In the study by Xu et al. (2021), the optimal control problem with piecewise-constant controller gain in a random environment is solved, and an improved Hamilton–Jacobi –Bellman (HJB) partial differential equation is obtained by the splitting method and Feynman-KAC formula.

However, to the authors’ best knowledge, there are very few studies focusing on the optimal control of discrete-time MASs only containing a directed spanning tree. In this study, the leader-based distributed optimal control problem of discrete-time linear MASs on directed communication topologies is investigated. A distributed discrete-time consensus protocol based on the directed graph is designed, and it is proved that the optimization of energy cost performance can be satisfied with the presented consensus protocol. Furthermore, the optimal solution can be obtained by solving the algebraic Riccati equation (ARE), and the design of the protocol presented in this study does not require global communication topology information and relies on only the agent dynamics and relative states of neighboring agents, which means that every agent manages its protocol in a fully distributed way.

Notation. R N stands for the Euclidean space with N-dimension; I _*n* is an identity matrix of n-order; 1_*N* is a N-dimensional vector with all elements equaling 1, and 0_*m*×*n* denotes a zero matrix of order m × n; ‖x‖ stands for the Euclidean norm of the vector x; A ⊗ B denotes the Kronecker product between the matrices A and B; P > 0 represents the positive definiteness of the matrix P, and P ≥ 0 represents the positive semi-definiteness of P; P ⁻¹ and P ^T are the inverse matrix and transpose matrix of P, respectively.

2 Preliminaries

2.1 Algebraic Graph Theory

A digraph G = { V , E } is used to describe the communication topology of MASs, where V = { ν 1 , … , ν N } denotes the set of nodes. An edge (ν _*j*, ν _*i*) is included in the set E if the relative information can be transfered from ν _*i* to ν _*j*. A path from ν _*i* to ν _*j* is made up of a set of edges (ν _*i*, ν _*l*1), … (ν _ln, ν _*j*). A graph is supposed to be connected if a path from ν _*i* to ν _*j* for all pairs of (ν _*i*, ν _*j*) existed. An adjacency matrix A = [ a i j ] N × N is used to describe the digraph G, where a _*ii* = 0, and a _*ij* = 1, i ≠ j, if ( ν j , ν i ) ∈ E, but 0 otherwise. Let L = [ l i j ] N × N denote the Laplacian matrix of G such that l _*ij* = −a _*ij* for i ≠ j and l i i = ∑ j = 1 N a i j.

2.2 Problem Formulation

Considering a group of N agents with the discrete-time system presented by the following equation.where x i ( k ) ∈ R p denotes the state variable and u i ( k ) ∈ R q denotes the control input; A and B are constant matrices with suitable dimensions. The purpose of this study is to design a protocol that guarantees the states of N agents in Eq. 1 to achieve an asymptotic consensus. i.e., lim k → ∞ x i ( k ) − x 0 ( k ) = 0 and optimizes the cost function (which will be defined later).

x i k + 1 = A x i k + B u i k x 0 k + 1 = A x 0 k i = 1,2 , … , N (1)

Assumption 1 the leader agent’s index is defined as 0, and the leader agent’s and the follower agent’s index are defined as 1, … , N,. The digraph G contains a directed spanning tree with the leader as the root node.

Lemma 1 (Matrix Inversion Lemma (Horn and Johnson, 1996)): For any nonsingular matrices E ∈ C ^*N*×*N*, G ∈ C ^*N*×*M* and the general matrices F ∈ C ^*N*×*N*, H ∈ C ^*N*×*M* holds. Then, the inverse of the matrix(E + FGH) is as follows.

3 Main Results

In this section, a distributed optimal controller is designed to solve the consensus of the system in Eq. 1, and the optimization of cost function is achieved with the presented protocol.

Since Assumption 1 holds, the Laplacian matrix L can be regraded as (Zhang and Lewis, 2012)where L 1 is a nonsingular matrix.

L = 0 0 1 × N L 2 L 1 , (2)

Let ξ i ( k ) = ∑ j = 0 N a i j ( x i ( k ) − x j ( k ) ), then we havewhere x ̃ 0 ( k ) = 1 N ⊗ x 0 ( k ) x ( k ) = x 1 T ( k ) , … , x N T ( k ) T, and ξ ( k ) = ξ 1 T ( k ) , … , ξ N T ( k ) T; it implies that the leader-following consensus of the system in Eq. 1 can be achieved i.e., lim k → ∞ x i ( k ) − x 0 ( k ) = 0 and i ∈ 1 , … , N can be achieved if and only if lim k → ∞ ξ ( k ) = 0.

ξ k = L 1 ⊗ I p x k − x ̃ 0 k , (3)

A distributed optimal controller is developed as follows.where c represents the coupling strength, and K denotes the control gain matrix.

u i k = ∑ j = 0 N a i j − 1 ∑ j = 1 N a i j u j k − c K ξ i k , i = 1 , … , N , j = 0 , … , N (4)

The error system can be obtained by taking the difference of Eq. 3 as follows.where U ( k ) = u 1 T ( k ) , … , u N T ( k ) T.

ξ k + 1 = I N ⊗ A ξ k + L 1 ⊗ B U k , (5)

Inspired by reference given by Zhang et al. (2021b), the cost function is chosen to bewhere the matrices Q = Q T > 0 and R = R T > 0 denote the appropriate weighting matrices. In addition, the energy-cost function constraint performance for the system in Eq. 5 is considered as follows.

L k = 1 2 ξ T k Q ξ k + 1 2 U T k R U k , (6)

J = ∑ k = 0 ∞ L k = ∑ k = 0 ∞ 1 2 ξ T k Q ξ k + 1 2 U T k R U k . (7)

In Eq. 7, 1 2 ξ T ( k ) Q ξ ( k ) represents the process cost and 1 2 U T ( k ) R U ( k ) represents the control cost. Therefore, J can be considered the goal of comprehensive optimization of control energy and error quantity. Furthermore, a Hamiltonian equation is utilized to optimize the cost function L(k) aswhere λ^*T* (k + 1) represents the costate variable and f ( k ) = ( I N ⊗ A ) + ( L 1 ⊗ B ) U ( K ).

H k = − L k + λ T k + 1 f k , (8)

Next, the protocol presented in Eq. 4 is proved to guarantee the optimization of the energy cost performance and stability of system in Eq. 5.

Theorem 1 For the given matrices Q = Q ^*T* > 0 and R = R ^*T* > 0, the cost function L(k) is optimized and the stability of system in Eq. 5 can be achieved if and only if the following ARE holds:where P is the positive definite solution of Eq. 9, and the control gain matrix is K = ( R + c B T P B ) − 1 B T P A; c is a constant value satisfying the condition c > 1.

Proof i) Optimization of Cost Function The optimal control input can be solved from the equation as follows. Let λ(k) = −(I _*N* ⊗ P)ξ(k), then the optimal controller can be obtained in Eq. 10 as follows. Let R = 1 c ( L 1 T L 1 ⊗ R ), where R = R ^T > 0. Since Eq. 11 holds true, Eq. 5 can be rewritten as According to Lemma 1, it indicates that the expression of U* can be rewritten aswhere K = ( R + c B T P B ) − 1 B T P A is the control gain of the optimal controller in Eq. 4, which can guarantee the optimization of the cost function L(k). Considering the costate variable λ(k) = −(I _*N* ⊗ P)ξ(k), it can be obtained by the following equation.which indicates that Let Q = I N ⊗ A T P B ( c − 1 R + B T P B ) B T P A − I N ⊗ A T P B ( R + B T P B ) B T P A + I N ⊗ Q be a positive definite matrix, and Q = Q ^T > 0 holds true, then we havewhich is identical to the ARE presented in Eq. 9. It is to be noted that if c ≥ 1, we have ( c − 1 R + B T P B ) − 1 ≥ ( R + B T P B ) − 1, which implies that Q ≥ 0. Then, the positive definiteness of Q > 0 is achieved by c > 1. Considering the following equation: Based on K = ( R + c B T P B ) − 1 B T P A and U ( k ) = U * ( k ) = − c ( L 1 − 1 ⊗ K ) ξ ( k ), the abovementioned Eq. 17 can be rewritten as Since Eq. 5 and Eq. 15 hold true, we haveAccording to the following conditional equationThen, Eq. 19 can be regarded as Let V ( k ) = 1 2 ξ T ( k ) ( I N ⊗ P ) ξ ( k ) denote the Lyapunov function, then, substituting Eq. 21 into Eq. 18, we have Let ϕ = [ U T ( k ) + ξ T ( k ) ( L 1 T ⊗ c K T ) ] [ L 1 T L 1 ⊗ ( R c ) + B T P B ] [ U ( k ) + ( L 1 ⊗ c K ) ξ ( k ) ], that is ϕ = 0 holds true if and only if U(k) = U(k*) holds, and the cost function L(k) can be rewritten asThen, the cost function can be optimized, that is, L(k) = −ΔV(k*) with the controller U * ( k ) = − c ( L 1 − 1 ⊗ K ) ξ ( k ). Hence, it indicates that the optimal performance index J* is derived as follows.where V (0) represents the initial value of V(k). Based on the expressions of U(k), we have It is inferred from Eq. 25 that − lim_k→∞* V(k) = 0. Then, the optimal performance index J* can be rewritten as As a consequence, the conditions in Theorem 1 are all satisfied, which completes the proof.

Remark 1 Based on Theorem 1, it is obvious that the value of the control gain matrix K mainly depends on the matrix P and the coupling strength c, where the value of P is directly solved by Eq. 9, and c is a constant value satisfying the condition c > 1. Therefore, the design of the control protocol u _*i*(k) in Eq. 4 does not require global communication topology information and relies only on the agent dynamics and relative states of neighboring agents, that is, every agent manages its control protocol u _*i*(k) in a fully distributed way.

Remark 2 The topology considered in this study is a structure containing only one directed spanning tree, which means that the agent can only obtain the information of a single neighbor, and we prove the effectiveness of the proposed distributed optimal controller under the abovementioned conditions. In fact, the proposed controller is also suitable for the case with the general case, such as reference given by Wang et al. (2017), Wang et al. (2019).

4 Numerical Example

In this section, a numerical example is provided to demonstrate the effectiveness of the proposed controller.

Considering a network with seven agents, the communication topology is described by Figure 1. Moreover, the system parameters of each agent are given as follows (Xi et al., 2020).

A = 1.005 2 0.010 2 − 0.099 8 0.046 1 1.041 1 0.099 8 − 0.104 9 − 0.204 7 0.995 0 , B = − 0.067 7 − 0.024 6 0.155 9 T .

Let R = 10, Q = 10*I ₃, and the coupling strength c = 2, then the matrix P and the control gain K can be calculated by Theorem 1. The initial conditions are given by x ₀ (0) = [0.2–0.2 0.3]^T, x ₁ (0) = [0.1 0.2 0.2]^T, x ₂ (0) = [−0.15–0.1 0.1]^T, x ₃ (0) = [0.3 0.2 0.1]^T, x ₄ (0) = [−0.2 0.2–1.1]^T, x ₅ (0) = [1.3 0.1–0.1]^T, and x ₆ (0) = [1.0 0.5 1.5]^T. Then, the trajectories of the state norm and tracking error norm are shown in Figures 2, 3.

It can be seen from Figures 2, 3 that six followers can track the leader successfully within about 11 s by using the proposed optimal controller, and the steady-state tracking error is less than 2.0. In addition, it is shown in Figure 4 that the control input of six agents will nearly reach zero at about 13 s.

Moreover, the trajectories of energy cost performance J are displayed in Figure 5, which shows that the optimal performance of J equals 924. It can be acquired from Theorem 1 that the theoretical value of the optimal performance is J * = V ( 0 ) = 1 2 ξ T ( 0 ) ( I N ⊗ P ) ξ ( 0 ) = 924.066. Consequently, the simulated value of J* is consistent with its theoretical value, which proves that the controller proposed in this study satisfies the optimality requirements.

5 Conclusion

In this study, the leader-based distributed optimal control of discrete-time linear MASs only containing a directed spanning tree has been investigated. A distributed optimal consensus control protocol is presented to guarantee that multiple followers can successfully track the leader. It can be proved that the proposed protocol can ensure the optimization of the energy performance index with the optimal gain parameters which can be realized by solving the ARE. Moreover, the design of the protocol presented in this study is independent with the global information of topologies, which indicates that every agent manages its protocol in a fully distributed way. Finally, a numerical example which illustrates the effectiveness of the designed protocol is reported.

Data Availability Statement

The original contributions presented in the study are included in the article/Supplementary Material, further inquiries can be directed to the corresponding author.

Author Contributions

GH is responsible for the simulation and the writing of this manuscript. ZZ is responsible for the design idea of this study. WY is responsible for the revision of this manuscript.

Funding

This study was supported in part by the National Key R&D Program of China 2019YFB1310303, in part by the Key R&D program of Shaanxi Province, 2021GY-289, in part by the National Natural Science Foundation of China under Grant U21B2047, Grant U1813225, Grant 61733014, and Grant 51979228.

Publisher’s Note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.