Adaptive dynamic programming for coordinated control of a dual-arm reconfigurable manipulator with unknown object constraints

1 Introduction

A dual-arm manipulator demonstrates formidable capabilities in collaborative work and multitasking, exhibiting exceptional coordination and flexibility (Cao et al., 2024). A dual-arm manipulator showcases promising application prospects across diverse fields, such as industrial automation, minimally invasive surgery, precise harvesting of fruits and vegetables, household management, object transportation, and multi-object assembly. To accomplish these tasks, enhancing the flexibility, autonomy, and versatility of the dual-arm manipulator is essential. A conventional dual-arm manipulator, however, is typically designed with a fixed configuration to execute specific tasks, often requiring designers to manually derive and formulate a system model based on a predetermined structure. Due to the modular design, a reconfigurable manipulator (An et al., 2025b; Ji et al., 2025) can be assembled in various configurations. By partially replacing components and reconfiguring the system, a reconfigurable manipulator can effectively enhance fault tolerance while increasing overall flexibility and versatility. Consequently, the dual-arm reconfigurable manipulator system holds significant research value.

During object transportation, the dual-arm manipulator often forms a closed-chain constraint, which significantly enhances precision and stability but may also lead to mutual interference. Consequently, coordinated manipulation in the dual-arm and the increased degrees of freedom of the manipulator itself compound system complexity, thereby increasing the demand for coordinated control. The traditional coordinated control method relies heavily on a precise system dynamic model, computationally intensive kinematic calculations, and extensive offline training, which limits applicability in real-time scenarios and adaptability to dynamic tasks. For dual-arm transportation tasks, adjusting traditional impedance control parameters proves particularly challenging due to environmental uncertainties. Although impedance control (Song et al., 2025) can achieve interactive tasks required by an end-effector and object, and it can be applied without any switching operation between contact and non-contact conditions, its precision in controlling both the position of the grasped object and the contact force is lower than hybrid force/position control. It may limit impedance control applicability in certain scenarios, such as surgery, manufacturing, precision welding, and similar fields. Hybrid force/position control (Zhao et al., 2022; Sardarmehni and Song, 2023) enables simultaneous coordination of force and displacement in different directions, thereby finding broad application in dual-arm coordinated control. Additionally, when a dual-arm manipulator system grasps tool of varying sizes, the geometric uncertainty can alter the grasp matrix. This, in turn, affects the closed-loop kinematics of the interconnected system, making it difficult to achieve measurable decoupling of grasping force, let alone internal force tracking. Therefore, estimation of the tool becomes necessary.

To enhance the control performance of the manipulator system while reducing the energy consumption required by the controller, optimal control (Sun and Yuan, 2026; Luo et al., 2025) for a manipulator is of comparable significance. As a crucial component of modern control theory, optimal control focuses on the fundamental problem of selecting the control policy that optimizes certain performance metrics of a given controlled system. For the linear quadratic regulator problem, which involves a linear system as the controlled plant and a quadratic cost function, an optimal control strategy is primarily derived by addressing the Riccati equation. The approach has now become a mature and well-established component of optimal control theory. However, for the numerous nonlinear systems prevalent in practical engineering applications, obtaining the optimal control strategy requires addressing the Hamilton–Jacobi–Bellman (HJB) equation that constitutes a class of nonlinear partial differential (or difference) equations, making it difficult to derive analytical optimal solution.

Adaptive dynamic programming (ADP) (An et al., 2023; Ting and Aiguo, 2019; Yuan et al., 2025) serves as an efficient approach for addressing optimal control problems in nonlinear systems. In an ADP framework, neural networks are designed to approximate the performance index function and estimate a solution to the HJB equation, thereby providing an effective strategy for tackling the analytical challenges associated with nonlinear optimal control. Due to its strong advantages in solving nonlinear optimal control problems, ADP has attracted widespread attention from scholars in recent years. ADP has yielded substantial results in addressing challenges such as discrete-time (Sun and Xu, 2024; Wei et al., 2019; Luo et al., 2026), continuous-time (Sun et al., 2023; Hyatt et al., 2020; An et al., 2025a) data-driven (Liu et al., 2026; Zhang et al., 2022; Zhang and Liu, 2025), event-triggered (Diao et al., 2021; Qin et al., 2026; Conlon et al., 2024), vehicle platoon (Wang et al., 2026), and interconnected (Xia et al., 2025) optimal control for complex nonlinear systems.

The contributions of this study are mainly embodied in two dimensions:

This article addresses the scenario of a dual-arm reconfigurable manipulator system grasping an unknown object by designing an optimal force/position controller that enhances control accuracy while reducing energy consumption. Through estimation of the system’s grasp matrix, decoupling of grasping force is achieved, thereby enabling internal force tracking.

The adaptive dynamic programming method is introduced into force/position control. Based on the designed fusion error function, the approach simultaneously optimizes position tracking error and internal force error. In addition, only the critic network is used without slowing the response of the system or accumulating error. The proposed control method is further implemented and validated on the actual experimental platform.

2 System description and preliminaries

2.1 Kinematic analysis

Denote x o ∈ R p as the coordinate vector of the object’s center of mass (COM). Denote x e i ∈ R p × 1 as the end-effector position of the ith reconfigurable manipulator. The relationship between x o and x e i can be described as in Equation 1 where J o i x o represents the grasping matrix. It can be obtained by J o i x o = I 3 0 3 S i I 3 for i = 1,2. S i = 0 − S i z S i y S i z 0 − S i x − S i y S i x 0, r ori = S i x , S i y , S i z T is vector from the COM of the object being grasped under the world coordinate system to the end of the manipulator. The relationship between x e i and q i can be described as in Equation 2:where q i ∈ R n i × 1 denotes the joint position of the reconfigurable manipulator, q ̇ i ∈ R n i × 1 is the velocity of ith reconfigurable manipulator, n i is the degrees of freedom (DOF) of the manipulator, and J s i ∈ R p × n i is the Jacobian matrix from the end-effector to q i.

x ̇ e i = J o i x ̇ o , (1)

x ̇ e i = J s i q ̇ i , (2)

The forward kinematic of ith reconfigurable manipulator is

x o = ϕ i q i , i = 1,2 . (3)

Under the assumption that the point of force application between the grasped object and the end-effector coincides with the origin of the kinematic constraint coordinate frame and neglecting the volume of the end-effector, the partial derivative of Equation 3 can be expressed as follows: where J ϕ i q i ∈ R p × n i is the Jacobian matrix from q i to the COM of the grasped object, q ̈ i ∈ R n i × 1 denotes the joint acceleration of the ith reconfigurable manipulator, and the Jacobian matrix J ϕ i q i and the grasp matrix are in Equation 6

x ̇ o = ϕ ̇ i q i = J ϕ i q i q ̇ i , (4)

x ̈ o = J ̇ ϕ i q i q ̇ i + J ϕ i q i q ̈ i , (5)

J ϕ i q i = J o i x o + J s i q i . (6)

2.2 Force analysis of a dual-arm reconfigurable manipulator system

Let F o ∈ R p represent the resultant force that the manipulators exert on the COM of the grasped object. This force can be expressed as in Equation 7 where F cei ∈ R p is force acting on centroid of grasped object by ith reconfigurable manipulator. The relationship between end-effector force F e i and F cei is in Equation 8

F o = − ∑ i = 1 2 F cei , (7)

F cei = J o i T F e i . (8)

F e i consists of the external force used to drive the object’s motion and the internal force for maintaining the object’s stability during grasping as in Equation 9:where F J i = J o i T x o + F o , F I i = I F e i, and I = I − J o i T x o + J o i T x o denotes the zero space matrix of J o i T x o. According to the principle of load distribution, the relationship between the combined force F o and component F o i of the object is expressed as in Equation 10:

F e i = F J i + F I i , (9)

F o i = 1 2 F o . (10)

2.3 Dynamic analysis

Combining load distribution and force analysis, the dynamic of the ith reconfigurable manipulator is as follows:where I m i ∈ R n i × n i indicates the rotational inertia of the rotor, γ i ∈ R n i × n i denotes the gear ratio, f i q i , q ̇ i ∈ R n i is the joint friction torque, τ S i ∈ R n i represents the coupling torque, and τ i ∈ R n i is the output torque of the DC motor.

I m i γ i 2 q ̈ i + γ i f i q i , q ̇ i + τ S i = γ i τ i − J s i T 1 2 J o i T + F o + F I i , (11)

The following equation describes the model of the object in the workspace:where M o x o ∈ R p × p is the positive definite symmetry inertia matrix, C o x o , x ̇ o ∈ R p × p is the centrifugal torque matrix and the Coriolis torque matrix of the object, and g o x o ∈ R p is the gravitational term of the object.

M o x o x ̈ o + C o x o , x ̇ o x ̇ o + g o x o = F o , (12)

Given the tool reference model parameters M D, C D, and g D, Equation 12 can be rewritten as follows:where e M i = M ̃ x o x ̈ o + C ̃ x o , x ̇ o x ̇ o + g ̃ x o = M ̃ x o J ϕ i q ̈ i + M ̃ x o J ̇ ϕ i + C ̃ x o , x ̇ o J ϕ i q ̇ i + g ̃ x o is the reference model error. M ̃ x o = M o x o − M D x o , C ̃ x o , x ̇ o = C o x o , x ̇ o − C D x o , x ̇ o , g ̃ x o = g o x o − g D x o is the error term.

F o = M D x o + M ̃ x o x ̈ o + C D x o , x ̇ o + C ̃ x o , x ̇ o x ̇ o + g D x o + g ̃ x o = M D x o x ̈ o + C D x o , x ̇ o x ̇ o + g D x o + e M i , (13)

Combining Equations 11, 13, kinematic Equation 4, and Equation 5, the dynamic equation of a single manipulator with an object can be expressed as follows:where Γ = 1 2 J s i T J o i T +, τ i = J s i T F I i, F i = γ i f i q i , q ̇ i and u i = γ i τ i.

u i = I m i γ i 2 + Γ M D x o J ϕ i q ̈ i + Γ M D x o J ̇ ϕ i + Γ C D x o , x ̇ o J ϕ i q ̇ i + F i + τ S i + τ i + Γ g D x o + Γ e M i , (14)

Based on Equation 14, the state space equation of the dynamic for a dual-arm reconfigurable manipulator is as in Equation 15:where x i = x 1 , x 2 T = q i , q ̇ i T is the state vector. N D i = I m i γ i 2 + Γ i M D x o J ϕ i − 1, and f i = − F i − Γ i e M i is an unknown term.

x ̇ 1 i = x 2 i x ̇ 2 i = ▵ D i + N D i f i + N D i u i − τ S i − τ i , (15)

▵ D i = − I m i γ i 2 + Γ i M D x o J ϕ i − 1 ⁢ Γ i ⁢ M D ⁢ x o ⁢ J ̇ ϕ i + Γ i ⁢ C D ⁢ x o , x ̇ o ⁢ J ϕ i ⁢ q ̇ i + Γ i ⁢ g D ⁢ x o .

2.4 Estimation of the object’s COM

First, assume that the initial position of the target object’s COM is random and calculate the estimated grasping matrix J ̂ o i based on this initial value. The estimated acceleration X ̈ ̂ o of the target object’s COM is obtained from the COM dynamic equation of the target object as follows:where F ̂ o is the estimated resultant force on the COM of the target object.

M o x o x ̈ ̂ o + g o x o = F ̂ o , (16)

According to Equation 16, the estimated acceleration x ̈ ̂ o of the COM and the component vector F ̂ o ° of the resultant force F ̂ o acting on COM of the target tool satisfy the following relationship. where m o is the weight of the target tool; F e 1 and F e 2 represent the contact forces exerted by the left and right arms of the dual-arm reconfigurable manipulator (DRM) on the target object, respectively.

m o x ̈ ̂ o = F ̂ o ° − g o x o , (17)

F ̂ o ° = F e 1 + F e 2 , (18)

Combining Equation 17 and Equation 18, we obtain in Equation 19

m o x ̈ ̂ o = F e 1 + F e 2 − g o x o . (19)

Therefore, the estimated acceleration x ̈ ̂ o of object’s COM can be expressed as in Equation 20:

x ̈ ̂ o = m o − 1 F e 1 + F e 2 − g o x o . (20)

Define the estimated target object’s COM velocity x ̇ ̂ o k and the estimated target object’s COM position x ̂ o k as in Equation 21

x ̇ ̂ o k = x ̈ ̂ o k − 1 Δ t + x ̇ ̂ o k − 1 , (21)

x ̂ o k = x ̇ ̂ o k − 1 Δ t + x ̂ o k − 1 . (22)

Through the given kinematic relationship, the estimated values of the two contact points between the two end-effectors of the dual-arm reconfigurable manipulator and the target object are obtained:where R o is the transformation matrix from the target object coordinate system to the world coordinate; r ̂ ori = S ̂ i x , S ̂ i y , S ̂ i z T is the estimated value of the direction vector of the end-effector from the COM of the target object to the end of the ith arm in the world coordinate.

x ̂ c 1 k = x ̂ o k + R o r ̂ or1 x ̂ c 2 k = x ̂ o k + R o r ̂ or2 , (23)

The estimated error in defining the position of the two contact points is defined as follows:

e 1 = x c 1 − x ̂ c 1 e 2 = x c 2 − x ̂ c 2 (24)

Combining Equations 22-24, we obtain as Equation 25

e 1 = x c 1 − x ̇ ̂ o k − 1 Δ t + x ̂ o k − 1 + R o r ̂ or1 e 2 = x c 2 − x ̇ ̂ o k − 1 Δ t + x ̂ o k − 1 + R o r ̂ or2 (25)

Subsequently, we use the gradient descent algorithm to minimize the above estimation error, thereby obtaining the estimated direction vector r ̂ ori and giving its update rule as follows:where ρ and η are both positive constants, and we obtain in Equation 27

r ̇ ̂ ori = ρ R o T e i − η r ori , (26)

x ̂ o 1 = x c 1 − R o r ̂ or1 x ̂ o 2 = x c 2 − R o r ̂ or2 (27)

According to the above formula, the estimated value of the COM position of the target object is obtained as Equation 28 x ̂ o k = 1 2 ∑ i = 1 2 x ̂ o i k. Through continuous iteration, the estimated value of the target tool’s COM position can ultimately be approximated to a small neighborhood of its true value. The following is a convergence analysis of the object COM position estimator based on the gradient model designed in this section:

First, define a cost function E i = 1 2 e i T e i. According to the Newton–Raphson algorithm, when the update rule for the estimated direction vector r ̂ ori is a partial derivative of the cost function to r ̂ ori, E i, and e i can be minimized, thereby obtaining as Equation 28

r ̇ ̂ ori = − ρ ∂ E i ∂ r ̂ ori = − ρ ∂ e i T e i 2 ∂ r ̂ ori = − ρ ∂ e i T ∂ r ̂ ori e i = − ρ ∂ x c i − x ̂ o i − R o r ̂ ori ∂ r ̂ ori e i = ρ R o T e i . (28)

The term η r ori is added to Equation 26 to increase the convergence velocity of the estimation error, and the analysis is completed.

2.5 Dynamic analysis based on the object’s center of mass estimation

Combining the estimated grasping matrix J ̂ o i T, the dynamic equation of a single manipulator combined with an object can be expressed as in Equations 29, 30: where Γ ̂ = 1 2 J s i T J ̂ o i T +, J ̂ ϕ i = J ̂ o i + J s i. e J i is the error compensation term estimated by the grasping matrix.

u i = I m i γ i 2 + Γ i ̂ M D x o J ̂ ϕ i q ̈ i + Γ i ̂ M D x o J ̂ ̇ ϕ i + Γ i ̂ C D x o , x ̇ o J ̂ ϕ i q ̇ i + F i + τ S i + τ i + Γ i ̂ g D x o + Γ i ̂ e ̂ M i + e J i , (29)

e J i = 1 2 J s i T J ̃ o i T + M D J ̂ ϕ i q ̈ i + M D J ̂ ̇ ϕ i + C D J ̂ ϕ i q ̇ i + g D + e ̂ M i , (30)

Based on Equation 29, the state space equation of the dynamic of the DRM system can be described as follows:

x ̇ 1 i J ̂ ϕ i = x 2 i J ̂ ϕ i x ̇ 2 i J ̂ ϕ i = ▵ ̂ D i + N ̂ D i f a i + N ̂ D i u i − τ S i − τ i , (31)

where x 2 i J ̂ ϕ i denotes the system state of the grasping matrix following substitution estimation.

▵ ̂ D i = − I m i γ i 2 + Γ i ̂ M D x o J ̂ ϕ i − 1 Γ i ̂ M D x o J ̂ ̇ ϕ i + Γ i ̂ C D x o , x ̇ o J ̂ ϕ i q ̇ i + Γ i ̂ g D x o N ̂ D i = I m i γ i 2 + Γ i ̂ M D x o J ̂ ϕ i − 1, f a i = − F i − Γ i ̂ e ̂ M i − e J i is an unknown term.

3 Design of the ADP-based coordinated control scheme

3.1 Derivation of optimal coordinated control

Define a reference desired velocity q ̇ ijd in Equation 32:where x o d is the desired position of the object; x ̇ o d is the desired velocity; and β is a positive constant.

q ̇ ijd = J ̂ + ϕ i x ̇ o d − β x o − x o d , (32)

Define the joint racking error in Equation 33:

e q i = J ̂ + ϕ i x o − x o d = q i − q ijd . (33)

F I i can be parameterized by the vector of the Lagrange multiplier as in Equation 34:where ξ T is the direction vector of the internal force and λ I i represents the magnitude of the internal force.

F I i = ξ T λ I i , (34)

To ensure that the joint tracking error e q i and the internal force error e λ I i remain sufficiently small to obtain optimal control u i *, the deviation of the fusion performance index function is in Equation 36 where N i s i , u i = s i T Q i s i + u i T R i u i denotes the utility function. s i τ i is the tracking error fusion function with e ̇ q i = q ̇ i − q ̇ ijd, K d j , K q j , K c j are control parameters, and the initial state of this function is s 0 τ i = s i 0. Q i, R i are determined positive definite matrices. Ψ Ω are the admissible control policy set.

J i s i u i ∈ Ψ Ω = ∫ t ∞ N i s i , u i d τ , (35)

s i τ i = K d j e ̇ q i + K q j e q i + K c j ∫ 0 t k e λ I i d τ , (36)

Definition 1: The control strategy u i is admissible via the performance index function (Equation 35), if u i stabilizes the DRM (Equation 11) on the compact set C and J i is finite for ∀ x ∈ C.

Based on Equation 35, the Hamilton function iswhere ∇ J i s i = ∂ J i s i / ∂ s i , v j i = − K d j q ̈ ijd + K q j e ̇ q i + K c j e λ I i. Define the optimal performance index function:

H i s i , u i , ∇ J i = N i s i , u i + ∇ J i T ⋅ s ̇ i = s i T Q i s i + u i T R i u i + ∇ J i T ⋅ K d j ▵ ̂ D i + N ̂ D i f a i + N ̂ D i u i − τ S i − τ i + v j i , (37)

J i * s i = min u i ∫ t ∞ N i s i , u i d τ . (38)

Combining the above Hamilton function (Equation 37) and the optimal performance index function (Equation 38) and applying optimization principles, the optimal performance index function satisfies the HJB equation as Equation 39:

0 = min u i H i s i , u i , ∇ J i * . (39)

If J i * exists and is continuously differentiable, we can obtain the optimal control law as follows:

u i * = − 1 2 K d j R i − 1 N ̂ D i T ∇ J i * s i . (40)

Next, employ the policy iteration (PI) algorithm to identify the optimal control strategy. The specific steps are as follows:

Input: k = 0, initial feasible control u i 0, and a small positive constant ς i;

1) Solve J i kfrom 0 = N i s i , u i k + ∇ J i k + 1 T ⋅ K d j ▵ ̂ D i + N ̂ D i f a i + N ̂ D i u i k − τ S i − τ i + v j i, with J i k + 1 0 = 0.

2) Update u i k via u i k + 1 = − 1 2 K d j R i − 1 N ̂ D i T ∇ J i k + 1.

If J i k + 1 − J i k ≤ ς i then stop and obtain the approximate optimal control.

Else let k = k + 1 and return to procedure 1.

Output: The optimal solution u i *.

Through iterative computation of this algorithm, as the iteration index k → ∞, the optimal performance index function and optimal control strategy can be approximated.

3.2 Adaptive observer design

Next, we approximate the unknown model term in real time by designing an adaptive observer:where x ̂ i = x ̂ 1 , x ̂ 2 T is the observed value of the state of the reconfigurable manipulator system; k 1 i and k 2 i are positive gains given by the observer; and e F i = E 1 i , E 2 i T = x 1 i − x ̂ 1 i , x 2 i − x ̂ 2 i T represents the system state observation error. Combining Equation 31 and Equation 41, we obtain as Equation 42 where e f i = f a i − f ̂ a i denotes the observational error of the model uncertainty term.

x ̂ ̇ 1 i = x ̂ 2 i + k 1 i x 1 i − x ̂ 1 i x ̂ ̇ 2 i = ▵ ̂ D i + N ̂ D i f ̂ a i + N ̂ D i u i − τ S i − τ i + k 2 i x 2 i − x ̂ 2 i (41)

e ̇ F i = E ̇ 1 i E ̇ 2 i = = x ̇ 1 i − x ̂ ̇ 1 i x ̇ 2 i − x ̂ ̇ 2 i = = E 2 i − k 1 i E 1 i N ̂ D i e f i − k 2 i E 2 i , (42)

Estimation of model uncertainty can be updated in Equation 43:where α F i is a positive constant.

f ̂ ̇ a i = α F i N ̂ D i E 2 i , (43)

Theorem 1. Consider the adaptive observer (Equation 41) approximating the unknown model terms in Equation 31. The observation error is uniformly ultimately bounded (UUB) based on the adaptive update strategy (Equation 43).

Proof. As follows, the candidate Lyapunov function is defined:

V 1 i t = 1 2 e F i T e F i + 1 2 e f i T α F i − 1 e f i . (44)

The derivative of Equation 44 with respect to time is

V ̇ 1 i t = e F i T e ̇ F i + e f i T α F i − 1 e ̇ f i = E 1 i E ̇ 1 i + E 2 i E ̇ 2 i − f ̂ ̇ i T α F i − 1 e f i = E 1 i E 2 i − k 1 i E 1 i + E 2 i N ̂ D i e f i − k 2 i E 2 i − f ̂ ̇ i T α F i − 1 e f i = E 1 i E 2 i − k 1 i E 1 i 2 − k 2 i E 2 i 2 + N ̂ D i E 2 i − f ̂ ̇ i T α F i − 1 e f i ≤ − k 1 i − 1 2 E 1 i 2 − k 2 i − 1 2 E 2 i 2 − f ̂ ̇ i T α F i − 1 − N ̂ D i E 2 i e f i . (45)

Substitute the updated law Equation 40 into Equation 45. When k 1 i ≥ 1 2 and k 2 i ≥ 1 2 are satisfied, V ̇ 1 i t ≤ 0. Therefore, the observational error of the above model is UUB, which completes the proof.

3.3 Critic neural network implementation

Combining Equations 35, 41, we obtain the Hamiltonian function as Equation 46:

H i s ̂ i , u i , ∇ J i = N i s ̂ i , u i + ∇ J i T ⋅ s ̂ ̇ i = N i s ̂ i , u i + ∇ J i T ⋅ K d j ▵ ̂ D i + N ̂ D i f a i + N ̂ D i u i − τ S i − τ i + v j i . (46)

To obtain the optimal control strategy, the Hamiltonian function (Equation 46) should be solved to obtain the optimal performance index function J i *. The optimal control strategy is then obtained through the PI algorithm and the approximation ability of the neural network. Therefore, a critic neural network is constructed in Equation 47:where W i j ∈ R N i denotes the ideal weight of neural network, N i is the number of hidden neurons, δ i j s ̂ i is the active function, and ε i j is the critic NN approximation error.

J i s ̂ i = W i j T δ i j s ̂ i + ε i j , (47)

Therefore, the partial derivative of J i s ̂ i is expressed as in Equation 48:where ∇ δ i j s ̂ i = ∂ δ i j s ̂ i / ∂ s ̂ i and ∇ ε i j are the partial derivatives of the active function and the critic NN approximation error. Combining Equation 46 and Equation 48 yieldswhere e ijh is the residual obtained by approximating the HJB function using an ideal critic neural network. Because we cannot directly obtain the ideal critic neural network weight W i j, we approximate the critic neural network as in Equation 50:where W ̂ i j is the approximate value of the neural network weight. Combining Equation 49, ∇ J ̂ i s i is expressed as follows:

∇ J i s ̂ i = ∇ δ i j s ̂ i T W i j + ∇ ε i j , (48)

H i s ̂ i , u i , W i j = N i s ̂ i , u i + ∇ δ i j s ̂ i T W i j + ∇ ε i j T ⋅ s ̂ ̇ i = N i s ̂ i , u i + W i j T ∇ δ i j s ̂ i + ∇ ε i j T ⋅ K d j ▵ ̂ D i + N ̂ D i f ̂ a i + N ̂ D i u i − τ S i − τ i + v j i = − ∇ ε i j T s ̂ ̇ i = e ijh , (49)

J ̂ i s ̂ i = W ̂ i j T δ i j s ̂ i , (50)

∇ J ̂ i s ̂ i = ∇ δ i j s ̂ i T W ̂ i j . (51)

Therefore, by utilizing Equations 49, 51, we can derive the approximate Hamiltonian function:where e i j is the estimated approximate error of the Hamiltonian function. Next, we minimize objective function to adjust the neural network weight vector E i j = 1 2 e i j T e i j using the gradient descent algorithm and design its update strategy as follows:where α i j is learning rate of critic NN. Define v i = ∇ δ i j s ̂ i s ̂ ̇ i. Suppose that v i is bounded and there exists a positive constant v a i such that v i ≤ v a i holds.

H ̂ i s ̂ i , u i , W ̂ i j = N i s ̂ i , u i + ∇ δ i j s ̂ i T W ̂ i j T ⋅ s ̂ ̇ i = N i s ̂ i , u i + W ̂ i j T ∇ δ i j s ̂ i ⋅ K d j ▵ ̂ D i + N ̂ D i f ̂ a i + N ̂ D i u i − τ S i − τ i + v j i = e i j , (52)

W ̂ ̇ i j = − α i j e i j ∇ δ i j s ̂ i s ̂ ̇ i , (53)

The estimation error associated with the weights of the critic neural network is formulated as follows:

W ̃ i j = W i j − W ̂ i j . (54)

Combining Equations 52–54, we obtain

e i j = e ijh − W ̃ i j T v i . (55)

Combining the critic neural network update rule (Equation 53) with Equations 54, 55, we obtain the update rule for the weight estimation error as in Equation 56:

W ̃ ̇ i j = − W ̂ ̇ i j = α i j e i j v i = α i j e ijh − W ̃ i j T v i v i . (56)

The final approximate optimal control law is in Equation 57

u ̂ i * = − 1 2 K d j R i − 1 N ̂ D i T ∇ δ i j s ̂ i T W ̂ i j . (57)

3.4 Proof of stability

The stability of the developed optimal control for reconfigurable manipulators is analyzed using Lyapunov functions.

Provided that the persistent excitation condition holds, the weight vector can be iteratively refined until it converges toward its optimal value.

Theorem 2. The dynamic model of the DRM system and the object are illustrated in Equation 11 and Equation 12, and the state space equation is formulated in Equation 31. The critic NN weight approximation error, the internal force error, and the position tracking error are UUB via the presented distributed coordinated optimal control derived in Equation 57 via the weight update strategy (Equation 53).

Proof. Select the Lyapunov candidate function as in Equation 58

V t = ∑ i = 1 2 V i t = ∑ i = 1 2 1 2 s i T s i + J i * s i + 1 2 α i j W ̃ i j T W ̃ i j . (58)

The time derivative of V t as Equation 59

V ̇ t = ∑ i = 1 2 V ̇ i t = ∑ i = 1 2 s i T s ̇ i + ∇ J i * s i T s ̇ i + 1 α i j W ̃ i j T W ̃ ̇ i j = ∑ i = 1 2 s i T K d j ▵ ̂ D i + N ̂ D i f ̂ a i + N ̂ D i u i − τ S i − τ i + v j i + ∇ J i * s i T s ̇ i + 1 α i j W ̃ i j T W ̃ ̇ i j = ∑ i = 1 2 s i T K d j ▵ ̂ D i + N ̂ D i f ̂ a i + N ̂ D i u i − τ S i − τ i + v j i − s i T Q i s i − u i T R i u i + W ̃ i j T e ijh − W ̃ i j T v i v i = ∑ i = 1 2 s i T K d j ▵ ̂ D i + N ̂ D i f ̂ a i + N ̂ D i u i − τ S i − τ i + v j i − s i T Q i s i − u i T R i u i + W ̃ i j T e ijh v i − W ̃ i j T v i 2 . (59)

According to the properties of reconfigurable manipulator, there exist positive parameters L i f, w i g, and L i l such that inequalities ▵ ̂ D i ≤ L i f s i, N ̂ D i ≤ w i g, and f ̂ a i ≤ L i l hold. Combining these with Young’s inequality, the above equation can be derived as Equation 60:

V ̇ t ≤ ∑ i = 1 2 K d j L i f s i 2 + K d j w i g L i l s i + K d j w i g s i u i − K d j w i g τ S i s i − K d j w i g τ i s i + v i j s i − s i T Q i s i − u i T R i u i + W ̃ i j T e ijh v i − W ̃ i j T v i 2 ≤ ∑ i = 1 2 K d j L i f s i 2 + K d j w i g L i l s i + 1 2 K d j 2 s i 2 + 1 2 w i g 2 u i 2 − K d j w i g τ S i s i − K d j w i g τ i s i + v i j s i − λ min Q i s i 2 − λ min R i u i 2 + 1 2 e ijh 2 − 1 2 W ̃ i j T v i 2 ≤ ∑ i = 1 2 − λ min Q i − K d j L i f − 1 2 K d j 2 s i 2 − − K d j w i g L i l + K d j w i g τ S i + K d j w i g τ i − v i j s i − λ min R i − 1 2 w i g 2 u i 2 − 1 2 W ̃ i j T v i 2 − e ijh 2 . (60)

The angular acceleration q ̈ ijd, position tracking error e q i of the known desired trajectory, and internal force error e λ I i are bounded, and q ̈ ijd ≤ σ i, e q i ≤ ξ i, and e λ I i ≤ η i exist, where σ i, ξ i, and η i are upper bound normal numbers. The above equation can be summarized as in Equation 61:where Φ i 1 = λ min Q i − K d j L i f − 1 2 K d j 2 and Φ i 2 = λ min R i − 1 2 w i g 2.

V ̇ t ≤ ∑ i = 1 2 − Φ i 1 s i + − K d j w i g L i l + K d j w i g τ S i + K d j w i g τ i − v i j s i − Φ i 2 u i 2 − 1 2 W ̃ i j T v i 2 − e ijh 2 , (61)

Therefore, we can conclude that when s i satisfies outside Ω i 1 = s i : s i ≤ K d j w i g L i l − K d j w i g τ S i − K d j w i g τ i + K d j σ i + K q j ξ i + K c j η i Φ i 1 and W i j satisfies outside Ω i 2 = W ̃ i j : W ̃ i j ≤ e ijh v i, then V ̇ t ≤ 0 can be guaranteed. The parameters must satisfy the following conditions as in Equation 62:

λ min Q i ≥ K d j L i f + 1 2 K d j 2 λ min R i ≥ 1 2 w i g 2 (62)

Through the above stability proof and analysis, under the designed control strategy, the tracking error and the internal force tracking error of the dual-arm reconfigurable manipulator can be UUB.

4 Experimental study

4.1 Experiment settings

Distributed coordinated optimal control is experimentally verified using a DRM system that consists of a five-degree-of-freedom (5-DoF) reconfigurable manipulator and a 7-DoF manipulator (cf Figure 1). Two reconfigurable manipulators jointly perform the experiment of object grasping, and the trajectory of the target tool is a reciprocating linear motion along the Z-axis. The critic NN is chosen as the radial basis function neural network (RBFNN), and the activation function of Equation 47 is δ i j = e − ( s ̂ ̇ i − ζ i ) T ( s ̂ ̇ i − ζ i ) ℓ i with certain parameters ζ i as well as ℓ i. The learning rate of the critic neural network can yield solutions that diverge from the optimal control result. When parameter α i j is set too small, overtraining occurs. The iterative process becomes excessively time-consuming, while the performance index improves only marginally. In robotic systems, fast response is as critical a performance index as accuracy. Conversely, selecting an excessively large learning rate leads to undertraining, which compromises tracking accuracy. Based on various experimental investigations, an appropriate learning rate is chosen as α i j = 0.85. The effectiveness of the proposed method is verified through the position tracking in Cartesian space and joint space error, as well as the control torque and NN weight compared with the existing learning-based optimal method (Qin et al., 2023).

4.2 Experiment analysis

Figure 2 shows the position trajectory in Cartesian space. Because the end-effector of the manipulator only moves along the Z-axis, the curve depicting the actual position of the Z-axis changing over time is presented. Figures 3–6 show position tracking error in joint space. From the figures, the desired position and the actual position are highly overlapping; hence, the position error is within a small range via the proposed method. Therefore, the effectiveness of the proposed optimal control method has been verified. Figures 7–10 show control torque curves of two manipulators by different methods. The control torque curves are smooth and gentle, and the amplitude of the control torque is within an acceptable range, ensuring good output performance of the motor. Therefore, the developed method is effective. Figure 11 is the critic NN curves. As demonstrated by the curves, owing to the learning mechanism of the neural network, the curves converge to zero within a few seconds, thereby ensuring the stability of the system.

5 Conclusion

This work introduces a distributed coordinated optimal control approach employing adaptive dynamic programming for dual-arm reconfigurable manipulators handling diverse objects. Kinematic and force analyses are conducted to derive the dynamics of the individual arm and the object using the joint torque feedback technique and the Newton–Euler method, respectively. A fusion error function with unknown target constraints is then formulated, which simultaneously maintains satisfactory position and internal force tracking while enabling accurate estimation of the target location. An adaptive observer is utilized to realize the distributed coordinated optimal control strategy within the adaptive dynamic programming framework. Lyapunov stability analysis demonstrates that the critic neural network weight approximation error, internal force error, and position tracking error are all uniformly ultimately bounded. Experimental results confirm the validity of the proposed methodology. The control mechanism proposed in this article is distributed control. However, for a reconfigurable robot, a decentralized control mechanism that can adapt to different configuration changes is more important. How to apply the decentralized control mechanism to the dual-arm reconfigurable robot system will be the content of our next research work.

Generative AI statement

The author(s) declared that generative AI was not used in the creation of this manuscript.

Any alternative text (alt text) provided alongside figures in this article has been generated by Frontiers with the support of artificial intelligence and reasonable efforts have been made to ensure accuracy, including review by the authors wherever possible. If you identify any issues, please contact us.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Footnotes

Edited by: Yongwei Zhang, South China Agricultural University, China

Reviewed by: Li Zan, Shenyang University of Chemical Technology, China

Ke Wang, Tianjin University, China