ADMM and linearized ADMM for weakly convex minimization

1 Introduction

Optimization is a fundamental tool in applied mathematics and scientific computing, with broad applications in machine learning, signal processing, statistics, and operations research. Within the diverse landscape of optimization methods, convex optimization has become particularly prominent and influential. It rests on a mature theoretical foundation with strong guarantees (e.g., any local minimizer is globally optimal) and supports the design of efficient, reliable algorithms that scale to high-dimensional problems [1–3]. Consequently, convex optimization has become both a standard modeling approach and a practical computational technique across a wide range of applications.

In practice, however, convexity can be overly restrictive: many modern modeling choices involve objective functions that are non-convex, yet still possess structure that algorithms can exploit. Consequently, a variety of structured non-convex optimization approaches along with convergence analysis have also been developed [4–6]. An important class that lies between convex and fully non-convex functions is that of weakly convex (WC) functions, namely functions that become convex after adding a quadratic term [7]. On the theoretical side, weak convexity admits a well-developed first-order variational framework, e.g., directional derivatives and generalized subdifferentials, which gives rise to rigorous stationarity concepts and supports convergence analysis for optimization algorithms. At the same time, the WC assumption is prevalent in large-scale machine learning, for instance in the analysis of stochastic and proximal-type methods for non-smooth non-convex objectives [5], as well as in min-max formulations motivated by modern machine-learning applications [8].

In this paper, we study a broad class of composite optimization problems whose objective can be written as the sum of two components: a smooth convex function and a (potentially non-smooth) WC function. The smooth convex term typically models data fidelity or physics-based consistency, while the WC term is used to encode additional modeling structure, such as sparsity, robustness, or other forms of regularization. This composite formulation encompasses many widely used models in statistical learning, inverse problems, and signal/image processing [9, 10]. For example, non-linear regression and classification losses combined with weakly convex penalties fall naturally into this framework. Popular WC regularizers include smoothly clipped absolute deviation (SCAD) [11], the minimax concave penalty (MCP) [12], logarithmic (LOG) regularization [13, 14], and the transformed ℓ₁ penalty [15–17].

We employ a variable-splitting method, namely the alternating direction method of multipliers (ADMM) [18, 19], taking advantage of the two-term structure: the smooth term supplies gradient information, while the WC term preserves useful variational properties. Although the original formulation is unconstrained, introducing an auxiliary variable allows the convex term and the weakly convex term to be handled separately. Majorization-minimization (MM) [20], difference-of-convex algorithm (DCA) [21], and related first-order methods such as proximal gradient are also natural alternatives for unconstrained composite problems; however, these methods typically take a single forward step on the smooth term with a size capped by its Lipschitz constant, which can be conservative when this constant is large or difficult to estimate tightly. ADMM instead solves the smooth-term subproblem to high accuracy at each iteration, which can yield faster practical progress in such settings.

Despite its flexibility, a practical bottleneck in applying ADMM to our problem setting is that the subproblem associated with the smooth term may not admit a closed-form solution. Consequently, it is often necessary to run an inner iterative solver at each outer iteration, which can dominate the overall computational cost. To mitigate this issue, we further adopt a linearized ADMM (LADMM) [22] scheme, in which the smooth term in the corresponding subproblem is replaced by its first-order approximation. This modification preserves the splitting structure while yielding an inexpensive closed-form update, thereby significantly reducing the per-iteration cost.

Prior convergence analyses for ADMM were largely limited to convex problem settings, but there has been growing interest in extending such guarantees to non-convex problems. For example, Hong et al. [23] established convergence of the classical ADMM for certain classes of linearly constrained non-convex problems with possibly non-smooth terms, and Li and Pong [24] proved global convergence of splitting methods (including ADMM) for non-convex composite problems under a semi-algebraic assumption. Analogous guarantees have since been developed for the linearized variant (LADMM): Liu et al. [25] analyzed a linearly constrained non-convex non-smooth setting and proved that both the constraint violation and first-order optimality residuals vanish; Yashtini [26] proved global convergence of a proximal linearized ADMM under the Kurdyka–Łojasiewicz (KL) assumption; and, under the same KL assumption, Sun et al. [6] proposed an ADMM-based method that convexifies the resulting non-convex subproblems via a linearization technique inspired by the difference-of-convex algorithm (DCA), establishing convergence to a critical point. For problems with non-linear constraints, El Bourkhissi and Necoara [27] investigated an inexact, linearized ADMM in which both the smooth objective term and the non-linear constraints are linearized within the augmented Lagrangian, with one of the resulting subproblems solved only approximately at each iteration.

There has been prior work on ADMM-type convergence analysis for strongly and weakly convex composite models with linear constraints [23, 28]. In contrast, we focus on an unconstrained formulation and establish convergence to a directional stationary (d-stationary) point (see Definition 4). In general non-convex optimization, d-stationary points can be a stronger notion of optimality than critical points because it requires the directional derivative to be non-negative in every direction. For example, it has been shown that d-stationarity implies critical points for certain DC programs, and serves as a necessary condition for local optimality [29, 30]. For the specific weakly convex composite structure studied here, these notions in fact coincide (Section 3.1); we discuss the implications of this equivalence for our contribution there.

Although the unconstrained setting may appear simpler, it eliminates the consensus structure induced by linear constraints that is heavily exploited in prior analyses such as Hong et al. [23]. As a result, our analysis requires a different sufficient-decrease argument that directly leverages the weak convexity modulus. For ADMM, we establish a sufficient-decrease property of the augmented Lagrangian under explicit parameter conditions. For LADMM, we further introduce a modified augmented Lagrangian to control the lagged-gradient effect arising from linearization, which is critical for the convergence analysis. These developments lead to explicit parameter conditions under which the sequences generated by both ADMM and LADMM admit convergent subsequences whose limit points are d-stationary solutions.

We further corroborate the theory with three numerical case studies, including smooth low-dimensional test functions and a high-dimensional sparse logistic regression model [9]. These experiments illustrate (i) the effectiveness of ADMM/LADMM on the two-term structure, (ii) the practical benefit of linearization in reducing per-iteration cost when inner solves are expensive, and (iii) the resulting trade-off between computational time and solution quality across methods, showing that ADMM/LADMM achieve comparable accuracy with significantly reduced runtime in our test problems.

We summarize the main contributions of this work as follows:

• We apply ADMM to the unconstrained WC composite problem. Since this eliminates the consensus structure exploited by existing linearly constrained non-convex ADMM analyses [e.g., Hong et al. [23]], we develop a sufficient-decrease argument that directly leverages the weak convexity modulus.
• We develop a linearized variant (LADMM) that replaces the smooth-term subproblem with an inexpensive closed-form update. To control the resulting lagged-gradient effect, we introduce a Lyapunov correction sequence that avoids the additional structural assumptions, required by existing linearized/proximal ADMM analyses.
• We establish subsequential convergence of both ADMM and LADMM to a d-stationary point of the original problem, under explicit, verifiable parameter conditions on ρ, avoiding the KL or semi-algebraic assumptions typically required for non-convex ADMM-type convergence guarantees.
• We conduct numerical experiments on three case studies, ranging from low-dimensional smooth test functions to a high-dimensional sparse logistic regression model, which confirm the practical efficiency of both methods.

The remainder of the paper is organized as follows. In Section 2, we introduce the weakly convex setting, presenting the ADMM and LADMM algorithms. Section 3 develops the optimality notions, characterizes weak convexity used in our analysis, and establishes convergence guarantees for both methods. Section 4 provides a detailed account of the numerical experiments to validate the performance of ADMM and LADMM. Finally, Section 5 summarizes the paper.

2 Formulation and algorithms

We consider the problem

minx∈ℝn ζ(x)≜f(x)+g(x),(1)

where f is convex and differentiable and g is a weakly convex function that is not necessarily differentiable. The notion of weak convexity is defined in Definition 1.

Definition 1 (weakly convex function). A function g(x) is weakly convex if there exists a constant σ_*g*>0 such that g(x)+σg2||x||22 is convex. The constant σ_*g* is often referred to as the weakly convex modulus of the function g.

To take advantage of the special structure of the two functions f and g, we apply ADMM to solve Equation 1, as detailed in Section 2.1, and further accelerate it using LADMM in Section 2.2. The convergence analysis for both methods is given in Section 3.

2.1 ADMM

We rewrite Equation 1 into an equivalent constrained formulation,

minx,z f(z)+g(x) s.t. x=z,(2)

with its corresponding augmented Lagrangian function given by

L(x,z;v)≜f(z)+g(x)+ρ〈v,x-z〉+ρ2||x-z||22,(3)

where v∈ℝ^*n* is a dual variable and ρ>0 is a parameter. The ADMM iterations proceed as follows:

{xt+1∈ argminxL(x, zt; vt)zt+1∈ argminzL(xt+1, z; vt)vt+1= vt+ xt+1− zt+1,(4)

where the superscript t counts the iteration number. By algebraic manipulation, the x-subproblem in Equation 4 can be expressed equivalently as

minx g^(x)≜g(x)+ρ2||x-zt+vt||22.(5)

Proposition 1 guarantees that Equation 5 is a convex problem if the parameter ρ is greater than or equal to the weakly convexity modulus of g (defined in Definition 1).

Proposition 1. Suppose g is weakly convex with modulus σ_*g*>0. If ρ≥σ_*g*, then the problem (Equation 5) is convex.

Proof. We can rewrite g^ as follows,

g^(x)=g(x)+ρ2‖x‖22+ρ〈x,−zt+vt〉+ρ2‖−zt+vt‖22 =(g(x)+σg2‖x‖22)+ρ−σg2‖ x ‖22+ρ〈x,−zt+vt〉+ρ2‖ −zt+vt‖22.

Since g is weakly convex with modulus σ_*g*>0, the first term in the above equation is convex, and hence g^ is a convex function if ρ≥σ_*g*.

Likewise, the z-subproblem in Equation 4 can be written as

minz f^(z)≜f(z)+ρ2||xt+1-z+vt||22,

which is a convex problem, due to the convexity assumption of f(·). Consequently, if ρ≥σ_*g*, both the x- and z-subproblems in Equation 4 are convex, and their global optima can be obtained by standard optimization methods such as gradient descent (GD) [31] or Newton's method [1]. The overall procedure is summarized in Algorithm 1. Please refer to Supplementary Appendices A.1, B.1, and C.1 for additional algorithmic details for specific choices of f and g.

Algorithm 1 ADMM for weakly convex minimization. Algorithm pseudocode for an iterative optimization method is shown, using parameters rho and tMAX, initializing variables x, z, v, and t, and performing updates on x, z, and v within a while loop until t reaches tMAX before returning the final x.

2.2 Linearized ADMM

In many applications, the x-subproblem in Equation 4 can be computed in a single step using a proximal operator, whereas the z-subproblem involves the smooth term f and typically has no closed-form solution; as a result, it may require an inner iterative routine at each outer iteration, which can dominate the overall runtime. To avoid nested loops, we consider a linearized ADMM variant that replaces f(z) in the z-subproblem with a first-order approximation at z^*t*, i.e.,

f(z)≈f(zt)+〈∇f(zt),z-zt〉.

LADMM preserves the ADMM splitting structure, so the x-subproblem and the dual update remain unchanged. It iterates as follows,

{xt+1∈ argminxL(x, zt; vt)zt+1∈ argminzf^(z; zt)vt+1= vt+ xt+1− zt+1,(6)

where f^(z;zt)=f(zt)+〈∇f(zt),z-zt〉+ρ2||xt+1-z+vt||22. The z-update from Equation 6 can be computed in a single closed-form step,

zt+1=xt+1+vt-∇f(zt)ρ.(7)

Compared with ADMM, LADMM avoids iterative inner solves for the z-subproblem, often reducing per-iteration cost while preserving the same x-subproblem and dual step. The overall procedure is summarized in Algorithm 2. Detailed updates for the specified forms of f and g are provided in Supplementary Appendices A.2, B.2, and C.2.

Algorithm 2 LADMM for weakly convex minimization. Algorithm pseudocode written in plain text outlines an iterative optimization process with parameters rho and tMAX, initialization of iterates x, z, v, and t, a while loop to update variables using an argmin computation, an update referencing Equation 7, and a final return of x star as x t.

3 Theoretical analysis

This section is devoted to the convergence analysis of ADMM and LADMM. Specifically, in Section 3.1, we characterize weak convexity of a function and review the notion of a stationary solution, which forms the foundation of our subsequent analysis. With these tools in place, we establish convergence guarantees for ADMM in Section 3.2 and extend the analysis to the LADMM scheme in Section 3.3.

3.1 Characterizing weak convexity and stationarity

In the literature, there are a number of stationary solutions used for analyzing non-convex and non-differentiable problems. It turns out some of widely used stationary solutions are equivalent for the problem (Equation 1). This can be shown by rewriting Equation 1 as a difference-of-convex (DC) function (see Definition 2, then apply the results from the literature [29, 30, 32]. To do so, we introduce a DC function in Definition 2.

Definition 2 (DC function/program). A function is called difference-of-convex (DC) if one can rewrite the function as a difference of two convex functions. A DC program is defined by

minx∈ℝnθ(x)≜ϕ(x)-ψ(x),(8)

where both functions ϕ and ψ are convex.

It is straightforward that weakly convex functions are a special case of DC functions: if g is weakly convex with modulus σ_*g*, then

ζ(x)=ζ(x)+σg2||x||22︸convex-σg2||x||22,(9)

so the objective function ζ admits a DC decomposition. Since the problem (Equation 1) has a trivial DC representation (Equation 9) with a differentiable concave component, one can apply [29] (Proposition 6.1.10), which establishes the equivalence among directional stationary solutions, critical points, and Clarke stationary solutions of Equation 9. We refer to Cui and Pang [29] for a detailed discussion of these notions. These stationary concepts, defined respectively via the subdifferential, directional derivatives, and the Clarke subdifferential [33, 34], are standard in the DC programming literature. Leveraging their equivalence, we adopt the directional stationary solution for our analysis, as formalized in Definition 4. Consequently, proving that ADMM or LADMM produces a directional stationary point immediately establishes the other two stationarity properties.

We provide an equivalent characterization of weakly convex functions (see Definition 1). As shown in Proposition 2, this result is a specialization of Vial [35], (Proposition 4.8) and is stated using the directional derivative (Definition 3) so that the existing result can be applied directly in our analysis.

Definition 3 (Directional derivative). The directional derivative of a function g at a point x in the direction d, denoted g′(x; d), is defined as

g′(x;d)=limτ↓0g(x+τd)-g(x)τ,

if the limit exists. In this case, the function g(x) is directionally differentiable.

Definition 4 (directional stationary point). A point x^*∈ℝ^*n* is a directional-stationary solution (or d-stationary for short) of Equation 1 if ζ′(x^*; x−x^*)≥0 for all x∈ℝ^*n*.

For any directionally differentiable function, we present an equivalence condition of the weakly convexity of the function. We include a proof to make the paper self-contained.

Proposition 2. The function g is weakly convex if and only if g is directionally differentiable and there exists σ_*g*>0 for which

-g(x)+g(y)+g′(y;x-y)≤σg2||x-y||22,for allx,y∈ℝn.(10)

Proof. If g is weakly convex, then it is DC and therefore directionally differentiable, since the directional derivative of a DC function equals the difference of the directional derivatives of its convex components [33]. By applying the first-order condition of a convex function to g(x)+σg2||x||22, we have

g(y)+σg2||y||22+g′(y;x-y)+σg〈y,x-y〉≤g(x) + σg2||x||22 ∀x,y,

which is equivalent to Equation 10.

For the other direction, we assume g is directionally differentiable and there exists σ_*g*>0 such that Equation 10 holds. Define g~(x)=g(x)+σg2||x||22. Since σg2||x||22 is differentiable, g~ is directionally differentiable and

g~′(y;x-y)=g′(y;x-y)+σg〈y,x-y〉.

Rearranging Equation 10 gives

g(x)≥g(y)+g′(y;x-y)-σg2||x-y||22.

Adding σg2||x||22 to both sides and using ||x||22-||x-y||22=||y||22+2〈y,x-y〉 yields

g~(x)≥g~(y)+g~′(y;x-y), ∀x,y.

By the standard directional-derivative characterization of convexity, g~ is convex. Therefore g is weakly convex with modulus σ_*g*.

Before proceeding, we highlight the technical features that distinguish our analysis from existing convergence results for non-convex ADMM and DC programming methods. First, the weakly convex structure of g makes the x-subproblem non-convex unless the penalty parameter satisfies ρ≥σ_*g*; this parameter condition is essential and enters explicitly in our sufficient-decrease argument (Lemma 1). Our splitting constraint x = z in Equation 2 is a special case of the general linear constraint studied for non-convex ADMM by Hong et al. [23]; however, their convergence theory requires the non-convex term to be differentiable with a Lipschitz-continuous gradient, an assumption our possibly non-smooth weakly convex g need not satisfy. Consequently, introducing the splitting variable in Equation 2 does not by itself bring problem (Equation 1) within the scope of Hong et al. [23]; our sufficient-decrease argument instead directly exploits the weak convexity modulus σ_*g*.

Second, for LADMM, linearizing f in the z-subproblem introduces a lagged-gradient term that prevents the augmented Lagrangian from decreasing monotonically. To address this, we introduce the auxiliary sequence {κ_*t*} (defined in Equation 32), which absorbs the lagged-gradient effect and restores a sufficient-decrease property (Lemma 3). This differs in both mechanism and assumptions from existing linearized and proximal ADMM analyses: Sun et al. [6] convexifies the non-convex subproblem via a DCA-inspired linearization, but requires the concave component of the DC decomposition to be Lipschitz, which our weakly convex g need not satisfy; Yashtini [26] establishes global convergence of a proximal linearized ADMM under the Kurdyka–Łojasiewicz (KL) property, but requires the non-smooth term to decompose as the sum of a proper lower-semicontinuous function and a coercive function, a structural assumption we do not impose.

Third, although directional stationarity coincides with critical-point and Clarke stationarity for DC problem [Proposition 6.1.10, Cui and Pang [29]], our analysis reaches this common set of points directly via directional derivatives, under the explicit parameter conditions above, rather than through the Kurdyka–Łojasiewicz or semi-algebraic assumptions required by Li and Pong [24], Sun et al. [6], and Yashtini [26] to obtain their critical-point guarantees.

3.2 Convergence of ADMM

We establish the convergence of ADMM in Equation 4 based on the following assumptions about the objective function in Equation 1,

• (A1) The function ζ is coercive, i.e., ζ(x) → ∞ as ||x||₂ → ∞;
• (A2) The function ζ is lower-bounded;
• (A3) The function g is proper, lower semicontinuous, finite-valued, weakly convex with modulus σ_*g*, and directionally differentiable.
• (A4) The function f has Lipschitz continuous gradient, i.e., there exists a constant Lip∇f>0 such that

‖∇f(x)−∇f(y)‖2≤ Lip∇f‖x−y‖2, ∀x,y.(11)

The roles of these assumptions are as follows. The weak convexity of g and the Lipschitz continuity of ∇f are used to establish the sufficient-decrease estimates for the augmented Lagrangian in ADMM. The lower boundedness assumption ensures that the corresponding sequence {L(xt,zt;vt)}t=0∞ is lower bounded. The coercivity of ζ is used to obtain boundedness of the iterates {(xt,zt,vt)}t=0∞. Together, these descent and boundedness properties allow us to extract convergent subsequences and show that the successive differences vanish. Finally, the properness, lower semicontinuity, finite-valuedness, and directional differentiability of g ensure that the directional derivatives and limiting arguments used in the stationarity proof are well-defined.

We show in Lemma 1 that the augmented Lagrangian function (Equation 3) decreases sufficiently at each ADMM iteration under certain conditions, followed by Theorem 3 for the convergence of the iterations to a d-stationary solution of the problem (Equation 1). The key idea is that each primal update yields a descent controlled by the successive differences ||xt+1-xt||22 and ||zt+1-zt||22; telescoping then forces these differences to vanish.

Lemma 1. (Sufficient decrease) If ρ>max{σg,2Lip∇f} and Assumptions A3–A4 hold, then each ADMM iteration (Equation 4) satisfies

L(xt+1,zt+1;vt+1)−L(xt,zt;vt) ≤(σg−ρ2)‖xt+1−xt‖22 +(Lip∇f2ρ−ρ2)‖zt+1−zt‖22.(12)

Proof. Since g^ is convex by Proposition 1 and xt+1∈argminxg^(x) by the ADMM scheme (Equation 4), we have 0∈∂g^(xt+1), which implies that x^*t*+1 is a d-stationary point of g^, i.e., g^′(xt+1;xt-xt+1)≥0. Therefore, we can get

L(xt+1,zt;vt)-L(xt,zt;vt)=g^(xt+1)-g^(xt) ≤g^(xt+1)-g^(xt) +g^′(xt+1;xt-xt+1).

Using the definition of ĝ in Equation 5, we have

g^(xt+1)−g^(xt)+g^′(xt+1;xt−xt+1)=g(xt+1)+ρ2‖xt+1−zt+vt‖22−g(xt)−ρ2‖xt−zt+vt‖22 +g′(xt+1;xt−xt+1)+ρ〈xt+1−zt+vt, zt−xt+1〉={−g(xt)+g(xt+1)+g′(xt+1;xt−xt+1)} +ρ2{‖xt+1−zt+vt‖22−‖xt−zt+vt‖22 +2〈xt+1−zt+vt, xt−xt+1〉}={−g(xt)+g(xt+1)+g′(xt+1;xt−xt+1)}−ρ2‖xt−xt+1‖22.

It further follows from Assumption A3 that

L(xt+1,zt;vt)-L(xt,zt;vt)≤(σg-ρ)2||xt-xt+1||22.(13)

For the z-subproblem, the optimality of z^*t*+1 yields

∇f(zt+1)-ρ(xt+1-zt+1+vt)=0,(14)

which leads to the following chain of inequalities

L(xt+1,zt+1;vt)−L(xt+1,zt;vt)=f(zt+1)−f(zt)+ρ2(‖xt+1−zt+1+vt‖22−‖xt+1−zt+vt‖22)≤−〈∇f(zt+1),zt−zt+1〉+ρ2〈zt+1+zt −2(xt+1+vt),zt+1−zt〉 by convexity of f=ρ〈xt+1−zt+1+vt,zt+1−zt〉+ρ2〈zt+1+zt − 2(xt+1+vt),zt+1−zt〉 by Equation 14=−ρ2‖zt−zt+1‖22.

=-ρ2||zt-zt+1||22.(15)

Lastly, using the v-update (Equation 4) and the optimality of z^*t*+1 in Equation 14, we obtain

vt+1=vt+xt+1-zt+1=∇f(zt+1)ρ,(16)

which then yields

L(xt+1,zt+1;vt+1)-L(xt+1,zt+1;vt)=ρ〈vt+1-vt,xt+1-zt+1〉=ρ||vt+1-vt||22

=1ρ||∇f(zt+1)-∇f(zt)||22≤Lip∇f2ρ||zt+1-zt||22.(17)

The last inequality is obtained by Assumption A4 where the Lipschitz constant of the gradient of f is defined in Equation 11. Combining the inequalities (Equations 13, 15, and17), we complete the proof.

Theorem 3. If ρ>max{σg,2Lip∇f} and Assumptions A1–A4 hold, then the iterates {(xt,zt,vt)}t=1∞ generated by ADMM in Equation 4 has a subsequence convergent to a d-stationary point of problem (Equation 1).

Proof. Using the assumption that ρ>max{σg,2Lip∇f}, we have σg-ρ2<0 and Lip∇f2ρ-ρ2<0. The telescoping summation of inequality (Equation 12) from t = 0 to T for an arbitrary integer T = 0, 1, ⋯ leads to

L(xT+1,zT+1;vT+1)-L(x0,z0;v0) ≤(σg-ρ2)∑t=0T||xt+1-xt||22+(Lip∇f2ρ-ρ2)∑t=0T||zt+1 -zt||22≤0,(18)

which means that the sequence {L(xt,zt;vt)}t=0∞ is upper-bounded by L(x0,z0;v0).

Additionally, using Assumptions A2 and A4 together with Equation 16, we obtain a lower bound of L(xT+1,zT+1;vT+1), i.e.,

L(xT+1,zT+1;vT+1)=f(zT+1)−f(xT+1)+f(xT+1)+g(xT+1)+ρ〈vT+1,xT+1 −zT+1〉+ρ2‖xT+1−zT+1‖22 ≥f(xT+1)+g(xT+1)+〈vT+1−∇f(zT+1),xT+1−zT+1〉 +ρ−Lip∇f2‖xT+1−zT+1‖22

=f(xT+1)+g(xT+1)+ρ-Lip∇f2||xT+1-zT+1||22,(19)

which also implies that the sequence {f(xt)+g(xt)}t=0∞ is upper-bounded if ρ>Lip∇f. Assumption A1 regarding the coerciveness of ζ(x) guarantees that {xt}t=1∞ is bounded, so is {zt}t=1∞ by Equation 19. To show the boundedness of {vt}t=1∞, we use Equation 16 to obtain

thus leading to

The boundedness of {zt}t=1∞ guarantees the boundedness of {vt}t=1∞. Consequently, we show that the sequence {(xt,zt,vt)}t=1∞ is bounded. By the Bolzano-Weierstrass Theorem, it therefore admits a convergent subsequence, denoted by (xtj,ztj,vtj)→(x,z,v*) as t_*j* → ∞.

L(xt,zt;vt) is monotonically decreasing and lower bounded due to Lemma 1 and Assumption A2. By letting T → ∞ in Equation 18, we obtain that ∑t=0∞||xt+1-xt||22 and ∑t=0∞||zt+1-zt||22 are finite, and hence x^*t*+1−x^*t* → 0 and z^*t*+1−z^*t* → 0 as t → ∞. By Equation 17, each term ||vt+1-vt||22 is bounded by ||zt+1-zt||22, which implies that v^*t*+1−v^*t* → 0. Since (xtj,ztj,vtj)→(x,z,v), we have (xtj+1,ztj+1,vtj+1)→(x,z,v) and x^* = z^* due to the v-update.

It remains to show (x^*, z^*, v^*) is a d-stationary point of the problem (Equation 1). By the iterative scheme (Equation 4), we have

L(xtj+1,ztj,vtj)≤L(x,ztj,vtj), ∀x,L(xtj+1,ztj+1,vtj)≤L(xtj+1,z,vtj), ∀z.

Letting t_*j* → ∞, we have L(x,z,v)≤L(x,z,v*) for all x and L(x,z,v)≤L(x,z,v*) for all z, thus leading to

g(x)+ρ2||v||22≤g(x)+ρ2||x-z+v||22, ∀x,(21)

and

f(z)+ρ2||v||22≤f(z)+ρ2||x-z+v||22, ∀z.(22)

We fix a point x and choose z = x in Equation 22. Combining Equations 21, 22 yields

f(x)+g(x)+ρ||v*||22≤f(x)+g(x)

+ ρ2(||x-x+v||22+||x-x+v||22),(23)

where we use x^* = z^*. Let us define ζ^(x)≜f(x)+g(x)+ρ||x-x*||22. It follows from Equation 23 that

ζ^(x)=f(x)+g(x)≤f(x)+g(x)+ρ||x-x||22=ζ^(x),∀x,

implying x^* is a global minimum for ζ^ and hence a d-stationary solution. We show functions ζ and ζ^ have the same directional derivative at the point x = x^* by the following calculations:

ζ^′(x;d)=limh→0+ζ^(x+hd)-ζ^(x)h =limh→0+ζ(x+hd)+ρ||x+hd-x||22h -limh→0+ζ(x)+ρ||x-x||22h =limh→0+ζ(x+hd)-ζ(x)+ρh2||d||22h =limh→0+ζ(x+hd)-ζ(x)h =ζ′(x;d), ∀d.

Since x^* is a d-stationary solution of ζ^(x), we have 0≤ζ^′(x;d)=ζ′(x;d), which concludes the proof.

Remark 1. The convergence analysis above for ADMM, and the analysis for LADMM to follow, both assume that the respective subproblems are solved exactly. In practice, however, some subproblems are solved via iterative inner solvers. Extending the analysis to inexact updates would require additional assumptions on the stopping criteria, and we leave this as an important direction for future work.

3.3 Convergence of linearized ADMM

In this section, we establish the convergence of LADMM in Equation 6 under the same Assumptions A1–A4 as those used for ADMM. The main technical challenge is that linearizing f in the z-update breaks the monotonicity of the augmented Lagrangian L(xt,zt;vt), as evident in Lemma 2. To address this issue, we introduce the sequence {κt}t=1∞ in Equation 32, obtained by augmenting L with an additional quadratic term. While linearized ADMM has been studied for non-convex problems, the present analysis is tailored to weakly convex minimization, where the non-convexity of g is quantified by the weak convexity modulus σ_*g*. Building on Lemma 2, we further introduce Lemma 3 to prove that κ_*t* admits a sufficient decrease at each iteration, which implies boundedness of the iterates and vanishing successive differences. Theorem 4 establishes that the LADMM iterates admit a subsequence converging to a d-stationary point of Equation 1 under a suitably stronger condition on ρ compared to the requirement in ADMM.

Lemma 2. Under Assumptions A3–A4, each LADMM iteration (Equation 6) satisfies

L(xt+1,zt+1;vt+1)-L(xt,zt;vt) ≤(σg-ρ2)||xt+1-xt||22+(Lip∇f-ρ2)||zt+1 -zt||22+Lip∇f2ρ||zt-zt-1||22.(24)

Proof. We start with the z-subproblem in Equation 6 by analyzing the following difference

L(xt+1,zt+1;vt)−L(xt+1,zt;vt)=f(zt+1)−f(zt)+ρ2(‖xt+1−zt+1+vt‖22−‖xt+1−zt+vt‖22)≤〈∇f(zt),zt+1−zt〉+Lip∇f2‖zt+1−zt‖22 +ρ2(‖xt+1−zt+1+vt‖22−‖xt+1−zt+vt‖22)

=f^(zt+1;zt)-f^(zt;zt)+Lip∇f2||zt+1-zt||22,(25)

where we use the definition of f^ and the Lipschitz Assumption A4 that leads to

f(zt+1)≤f(zt)+〈∇f(zt),zt+1-zt〉+Lip∇f2||zt+1-zt||22.(26)

Using the strong convexity of f^(z;zt) with respect to z and the optimality condition ∇f^(zt+1;zt)=0, we have

f^(zt;zt)≥f^(zt+1;zt)+〈∇f^(zt+1;zt),zt-zt+1〉+ρ2||zt

-zt+1||22=f^(zt+1;zt)+ρ2||zt-zt+1||22.(27)

It follows from Equations 25, 27 that

L(xt+1,zt+1;vt)-L(xt+1,zt;vt)≤Lip∇f-ρ2||zt+1-zt||22.(28)

Lastly, using the optimality condition of the z-subproblem

∇f(zt)-ρ(xt+1-zt+1+vt)=0,(29)

we have

vt+1=vt+xt+1-zt+1=∇f(zt)ρ,(30)

from which the last term in Equation 24 follows by applying the following inequality

L(xt+1,zt+1;vt+1)-L(xt+1,zt+1;vt)≤Lip∇f2ρ||zt-zt-1||22.(31)

Combining Equations 13, 28, and31 completes the proof.

Unlike ADMM, the descent argument in Lemma 1 cannot be applied directly to LADMM, because the v-update in LADMM involves ∇f(z^*t*) in Equation 30 rather than ∇f(z^*t*+1) in Equation 16. As a result, an additional lagged-gradient term appears, and the augmented Lagrangian L(xt,zt,vt) alone no longer yields a sufficient decrease. To address this issue, we introduce an auxiliary function:

κt≜L(xt,zt,vt)+Lip∇f2||zt-zt-1||22.(32)

The additional squared-difference term allows us to absorb the lagged-gradient effect and recover a sufficient-decrease property for LADMM; see Lemma 3.

Lemma 3. If ρ>max{σ_*g*, 1+Lip∇f, Lip∇f(1+2Lip∇f)} and Assumptions A2–A4 hold, then {κt}t=1∞ defined in Equation 32 is monotonically decreasing and lower bounded.

Proof. Using the definition of κ_*t* in Equation 32 together with Lemma 2, we obtain

κt+1-κt =L(xt+1,zt+1,vt+1)-L(xt,zt,vt)+Lip∇f2||zt+1-zt||22 -Lip∇f2||zt-zt-1||22 ≤(σg-ρ2)||xt+1-xt||22+(2Lip∇f2+Lip∇f-ρ2)||zt+1

-zt||22(33)

+((1-ρ)Lip∇f2ρ)||zt-zt-1||22,(34)

where the right side must be non-positive due to the choices of parameters. This shows that {κt}t=1∞ is monotonically decreasing. Next, we show that {κ_*t*} is lower bounded. To do so, we first observe

ρ〈vt,xt−zt〉=〈∇f(zt−1),xt−zt〉 by Equation 30 =〈∇f(zt−1)−∇f(zt),xt−zt〉+〈∇f(zt),xt−zt〉 ≥−12(‖∇f(zt−1)−∇f(zt)‖22+‖xt−zt‖22) +〈∇f(zt),xt−zt〉 ≥−Lip∇f22‖zt−zt−1‖22−12‖xt−zt‖22+f(xt) −f(zt)−Lip∇f2‖xt−zt‖22,

where the last inequality is by Equation 26. Combining the above with the definition of κ_*t* yields

κt=f(zt)+g(xt)+ρ〈vt,xt-zt〉+ρ2||xt-zt||22 +Lip∇f2||zt-zt-1||22 ≥f(xt)+g(xt)+Lip∇f22||zt-zt-1||22

+(ρ-1-Lip∇f2)||xt-zt||22.(35)

Due to Assumption A2, the sum f(x^*t*)+g(x^*t*) is lower bounded, which implies κ_*t* is lower bounded with the choice of parameters. This concludes the proof.

Note that the condition ρ>max{σ_*g*, 1+Lip∇f, Lip∇f(1+2Lip∇f)} is stronger than the ADMM condition ρ>max{σg,2Lip∇f}, reflecting the additional control needed to compensate for the lagged-gradient term introduced by linearization. In practice, we can estimate a lower bound of ρ based on the explicit forms of both f and g; see the experimental section and Supplementary material for more details.

Theorem 4. If ρ>max{σ_*g*, 1+Lip∇f, Lip∇f(1+2Lip∇f)} and Assumptions A1–A4 hold, then the iterates {(xt,zt,vt)}t=1∞ generated by Equation 6 has a subsequence convergent to a d-stationary point of problem (Equation 1).

Proof. The proof is similar to that of Theorem 3. The only difference is that we use the sequence {_{κ_*t*}t≥1} in place of {L(xt,zt;vt)}t≥0. In particular, by telescoping summation of the inequality (Equation 33) from t = 1 to an arbitrary integer T leads to

κT+1-κ1≤(σg-ρ2)∑t=1T||xt+1-xt||22+(2Lip∇f2+Lip∇f-ρ2)∑t=1T||zt+1-zt||22

+(Lip∇f2-ρLip∇f2ρ)∑t=1T||zt-zt-1||22,(36)

which means that {κt}t=1∞ is upper bounded by κ₁. Therefore, with Equation 35, the sequence {f(xt)+g(xt)}t=1∞ is upper bounded. The coerciveness of ζ(x) in Assumption A1 guarantees that {xt}t=1∞ is bounded, and so is {zt}t=1∞ by Equation 35. To show the boundedness of {vt}t=1∞, we use Equation 30 to obtain

leading to

The boundedness of {zt}t=1∞ guarantees the boundedness of {vt}t=1∞. The remainder of the proof follows verbatim from that of Theorem 3.

Remark 2. The convergence results in Theorems 3 and 4 establish subsequential convergence of the generated iterates. Establishing an analogous full-sequence convergence result for the present weakly convex ADMM/LADMM framework would require additional assumptions such as the Kurdyka–Łojasiewicz (KL) property [36] beyond those used here. We therefore leave this extension for future work.

Remark 3. The lower bounds on the penalty parameter ρ in Theorems 3 and 4 are sufficient conditions for convergence. These bounds depend on the weak convexity modulus σ_*g* and the Lipschitz constant Lip∇f, which can be estimated from the analytical form of the objective function. In particular, the weak convexity modulus σ_*g* is obtained from the regularizer or non-convex component, while the Lipschitz constant Lip∇f is estimated from the smooth term f. The corresponding estimates used in the numerical experiments are reported in the Supplementary material. We note that these theoretical bounds may be conservative in practice; and convergence was observed even for values of ρ below the theoretical threshold in our experiments. Developing adaptive or problem-dependent strategies for choosing ρ is an important direction for future work.

4 Numerical experiments

In this section, we present numerical experiments to demonstrate the efficiency of both the ADMM and LADMM methods across three case studies. All the numerical experiments were conducted in MATLAB R2021a on a machine equipped with an 11th-Gen Intel Core i7-11370H CPU and 8 GB of memory. The first two involve two-dimensional (2D) smooth functions, where we compare ADMM/LADMM with gradient descent (GD) [31], heavy ball (HB) [37], and Nesterov's accelerated gradient (NAG) [38]. Specifically, the update of GD is given by:

{pt+1=−∇ζ(xt)xt+1=xt+αt+1pt+1,(37)

where the stepsize α^*t*+1>0 is updated iteratively using the Armijo-Goldstein condition [39]. The iterations of HB method are

{pt+1=−∇ζ(xt)+βt+1ptxt+1=xt+αt+1pt+1,(38)

with α^*t*+1 adaptively chosen by the Armijo-Goldstein condition and a fixed β^*t*+1 = β. We choose the optimal value of β = 0.1 among {10⁻³, 10⁻², 10⁻¹, 1, 10¹} that gives the lowest relative errors to the ground-truth. Starting with q⁰ = 1 and an initial point x⁰, the NAG method is formulated as:

{qt+1=1+4(qt)2+12pt+1=−∇ζ(xt)yt+1=xt+αt+1pt+1xt+1=yt+1+qt−1qt+1(yt+1−yt).(39)

Similar to other gradient-based methods, the step size α^*t*+1 in NAG is updated adaptively by the Armijo-Goldstein condition.

Lastly, we investigate a sparse logistic regression problem in high dimensions in Section 4.3. Since the objective function is non-differentiable, the gradient-based methods described above are not directly applicable; Instead, in addition to ADMM and LADMM, we compare with several representative baselines, including a majorization-minimization (MM) approach [20], proximal linearized ADMM (PLADMM) [26], a proximal alternating linearized minimization (PALM) method [40], and a standard DC algorithm (DCA) [21].

For subproblems without a closed-form solution, we use iterative inner solvers. In the two low-dimensional test functions (Sections 4.1 and 4.2), the x-subproblem of ADMM is solved by gradient descent, and terminated when the gradient norm falls below 10⁻² or after 8,000 iterations. In the logistic regression experiment (Section 4.3), the z-subproblem of ADMM is solved by Newton's method, and terminated when ||znew-zold||<10-3, or after 100 iterations. Additional implementation details are provided in the Supplementary material.

Remark 4. We emphasize that two different LADMM variants are used in this paper, only one of which is covered by the convergence theory in Section 3.3. For the two low-dimensional test functions (Sections 4.1 and 4.2), the z-subproblem in ADMM admits a closed-form solution, while the x-subproblem does not. Accordingly, the LADMM variant implemented for these experiments linearizes g in the x-subproblem, rather than f in the z-subproblem as presented in Section 2.2. This is a problem-specific deviation from the analyzed algorithms: the convergence analysis in Section 3.3, which relies on the Lipschitz continuity of ∇f, does not directly apply to this variant; extending the theory to cover linearization of the weakly convex term is left for future work. By contrast, the logistic regression experiment in Section 4.3 implements the LADMM scheme exactly as analyzed in Section 2.2, with the linearization applied to the smooth logistic loss f in the z-subproblem.

4.1 Three-Hump Camel Function

We start with a valley-shaped testing function, called the Three-Hump Camel Function [41], defined by,

ζ(x1,x2)=2x12-1.05x14+x166+x1x2+x22.(40)

This function has a global minimum of ζ(x^*) = 0 at x^* = (0, 0) and is weakly convex. We set ρ = 100 for ADMM, which satisfies the sufficient condition in Theorem 3. We use the same parameter for LADMM; although our convergence analysis does not cover the chosen splitting strategy, empirical results indicate that LADMM still converges. Further details on the weakly convex formulation, the calculation of the relevant constants, and the implementation of ADMM and LADMM for Equation 40 are provided in Supplementary Appendix A.

All competing methods start from the same initial point, chosen randomly from [−2, 2] × [−2, 2]. The stopping criteria are identical across methods: the algorithm terminates when the gradient norm satisfies ||∇ζ(x)||2<10-8 or when the number of outer iterations reaches 15,000.

We evaluate performance based on the decay of the objective function defined in Equation 40 and the decay of the norm of the iterates, i.e., ||xt||2, with respect to the elapsed time, measured in s. Since the global optimum is attained at the origin, ||xt||2 quantifies the error to the optimal solution. Figure 1 presents the objective function values and the corresponding errors, demonstrating that LADMM and ADMM achieve the fastest descent while maintaining the highest accuracy among competing methods.

We further examine computation time, the number of outer iterations, the number of inner iterations (for ADMM only), and the total number of iterations across 100 random initializations. Table 1 reports the mean values over these 100 trials, with standard deviations shown in parentheses. The total number of iterations for ADMM (including both outer and inner iterations) is nearly twice that of GD and HB; nevertheless, ADMM is faster. In particular, LADMM is the fastest method and requires fewer outer iterations than ADMM, reflecting the efficiency of its linearized subproblem updates and the absence of inner solves. NAG consistently reaches the maximum number of iterations before stopping, making it the slowest method. We note that the Armijo-based step-size strategy used for NAG may not be optimal for these non-convex objectives; a different step-size rule could improve its performance.

Table 1. Computation time and (inner/outer) iteration numbers for minimizing the Three-Hump Camel Function (Equation 40), highlighting LADMM and ADMM as the most efficient.

Results are reported as the mean over 100 random initializations, with standard deviations shown in parentheses. Other parameter settings are the same as in Figure 1.

To understand why LADMM and ADMM appear faster than other gradient-based methods, we plot the trajectories of ADMM and GD on the contour of the Three-Hump Camel Function, illustrating a case where ADMM outperforms GD in terms of speed. As shown in Figure 2, the trajectories of ADMM and GD are nearly identical; however, in the beginning, the points along ADMM's trajectory are more widely spaced than those of GD. This indicates that while ADMM requires inner iterations to solve the subproblem, resulting in a higher total iteration count, its alternating updates of two variables nonetheless lead to more effective minimization than standard gradient-based approaches that only involve a single variable.

4.2 Six-Hump Camel Function

Next, we examine the Six-Hump Camel Function [42], defined by,

ζ(x1,x2)=(4-2.1x12+x143)x12+x1x2+(-4+4x22)x22.(41)

This function possesses two symmetric global minima at approximately (0.0898, −0.7126) and (−0.0898, 0.7126), where the minimum function value is about −1.0316. The corresponding function f for the the Six-Hump Camel is not globally Lipschitz continuous, and hence our theoretical guidelines for choosing ρ are not applicable. We therefore adopt an empirical choice ρ = 100 for both ADMM and LADMM. Despite the lack of theoretical guarantees, both methods exhibit stable convergence in our experiments. More details on the ADMM/LADMM formulation of Equation 41 can be found in Supplementary Appendix B.

Initial points for all methods are randomly sampled from the domain [−2, 2] × [−2, 2], ensuring a diverse set of starting conditions. The stopping criteria are nearly identical to those in Section 4.1, except that the maximum number of outer iterations is set to 1,500 to keep the method difference visible, since using 15,000 iterations (as in Section 4.1) would make curves nearly indistinguishable. These criteria are applied uniformly across all methods.

We study the convergence behavior of each method by tracking the decay of the gradient norm, ||∇ζ(xt)||2, and the relative error, defined as ||xt-x||2||x||2, over time. A gradient norm of ||∇ζ(x)||₂ = 0 indicates a stationary point. The relative error quantifies the deviation between the computed solution and the true global minimum. As shown in Figure 3, the evolution of both metrics demonstrates that LADMM and ADMM converge more rapidly than the other competing methods.

We conduct a quantitative evaluation of each method's overall performance. The aggregated results, including means and standard deviations, are reported in Table 2. Both Tables 1, 2 further indicate that although ADMM requires more total iterations, it achieves higher computational efficiency in these experiments. LADMM is again the fastest method, requiring fewer outer iterations than ADMM.

Table 2. Computation time and (inner/outer) iteration numbers for minimizing the Six-Hump Camel function (Equation 41), highlighting LADMM and ADMM as the most efficient.

Results are reported as the mean over 100 random initializations, with standard deviations shown in parentheses. Other parameter settings are the same as in Figure 3.

Figure 4 shows the trajectories of ADMM and HB methods on the contour of the Six-Hump Camel Function, starting from a particular initial point. The widely spaced points along ADMM's trajectory, noted in Figure 4, are even more evident than in Figure 2. Both Figures 2, 4 together suggest that ADMM identifies a descent direction more effectively in the early stages than HB.

4.3 Logistic regression with LOG regularization

Logistic regression, which produces both a binary decision and an estimated class probability, is a widely used classification model in statistics and machine learning, with applications in many areas including medical diagnosis [43] and finance [44]. Yet in high-dimensional settings, such as gene expression analysis [45] and large-vocabulary text classification [46], the number of features often exceeds the number of samples, where classical logistic regression may become unstable and prone to overfitting. Enforcing sparsity on the coefficients helps address these challenges, ensuring both statistical robustness and practical interpretability. In this section, we investigate a simulated scenario for sparse logistic regression.

We start by generating a sparse coefficient vector x^*∈ℝ^*n* with a sparsity level of 10%, which serves as the ground-truth. In particular, for each index j∈{1, ⋯ , n}, we include j in the index set S independently with probability 0.1. If j∈S, we assign xj* a non-zero value drawn from the standard normal distribution; otherwise, we set xj*=0. Then we generate an i.i.d. Gaussian matrix xtr∈ℝmtr×n with entries (xtr)ij~iidN(0,1), where m_tr denotes the number of training samples and n denotes the number of features. We set (m_tr, n) = (200, 50) in our experiment. We standardize the data matrix by subtracting column means. Specifically, we compute training column means μ=1mtr1⊤xtr∈ℝ1×n and define A_tr = X_tr−1μ. Then the binary labels are generated by

btr,i=sign(atr,i⊤ x*+ηtr,i), i=1,…,mtr,

where ηtr,i~N(0,ν2) acts as noise and ν>0 controls the noise level. We set its variance ν² = 0.1 in our experiments. Here, the subscript tr indicates that the matrix A_tr and vector b_tr correspond to the training data.

Given training data (Atr,btr)∈ℝmtr×n×{-1,1}mtr with rows ai⊤ and labels b_*i*∈{−1, 1}, the logistic regression function is defined as

L(x)=1mtr∑i=1mtrlog(1+exp(-bi(ai⊤x))),(42)

To mitigate overfitting, sparse logistic regression [9] estimates a coefficient vector x by solving

minx L(x)+λR(x),(43)

where R(x) promotes sparsity and λ>0 controls the strength of regularization. In this work, we adopt a log-composite (LOG) function [13] defined on a univariable x∈ℝ,

Plog(x;ϵ)=log(x2+ϵ+|x|),(44)

with ϵ>0 to enforce sparsity, thus leading to the following optimization problem:

minx L(x)+λ∑j=1nPlog(xj;ϵ).(45)

Since L(·) is convex and the LOG penalty function is weakly convex, the problem (Equation 45) aligns with our formulation (Equation 1). See Supplementary Appendix C for more details on the minimization of Equation 45 using ADMM and LADMM.

We compare the proposed ADMM/LADMM schemes with several representative baselines, including a majorization-minimization (MM) method [20], a proximal linearized ADMM method (PLADMM) [26], a proximal alternating linearized minimization (PALM) method [40], and a standard DC algorithm (DCA) [21]. We first describe the MM baseline in detail. In particular, we replace P_log(x_*j*; ϵ) at each iteration t by its concave tangent at xjt, thus leading to

xt+1∈argminxL(x)+λ∑j=1nwjt|xj|,(46)

where the weights are defined by wjt=1(xjt)2+ϵ.

The subproblem (Equation 46) is convex, which can be solved via ADMM.

For the PLADMM baseline, we use the same variable-splitting formulation as in ADMM/LADMM and add proximal terms to the subproblems; moreover, the smooth term L(x) is linearized, leading to a proximal linearized ADMM update [26]. For the PALM baseline, we apply a PALM scheme [40] to a quadratic penalty reformulation of the split problem. For the DCA baseline, we apply a standard DC algorithm [21] based on a difference-of-convex decomposition of the LOG regularizer, so that each DCA iteration requires the solution of a convex subproblem.

All algorithms start from the origin and share the same stopping rule: the algorithm is terminated when ||xt+1-xt||2||xt||2<10-4 or when the prescribed maximum number of outer iterations is reached.

We choose the regularization parameter λ in Equation 45 by K-fold (we set K = 5) cross validation on the training set A_tr. Following a conventional choice of the set [13], the candidate set {λi}i=150 is generated by

λi={10−5,i=1,5λi−1,2≤i≤6,1.1λi−1,7≤i≤44,2λi−1,45≤i≤50,(47)

which produces 50 values starting at 10⁻⁵, with coarse growth early, fine spacing in the middle, and a few larger steps at the end. We partition the index set {1, …, m_tr} into K disjoint folds {I(k)}k=1K of (approximately) equal size. For each k, define the training indices T(k)=⋃ℓ≠kI(ℓ) and the validation indices V(k)=I(k). Given a candidate λ, let x^λ(k) denote the coefficient estimate obtained by fitting on T(k). We evaluate the performance of the validation fold using the average logistic regression [47], i.e., by computing Equation 42 on the set V(k):

Lval(k)(λ)=1|V(k)|∑i∈V(k)log(1+exp(−btr,i atr,i⊤x^λ(k))),

and select the optimal value of λ that minimizes this loss,

λ*∈argminλ∈{λi}1K∑k=1KLval(k)(λ),

then refit the model on the full training set using λ^*. We observe that the optimal λ value is relatively small, so we fix ρ = 1 and ϵ = 0.01 for both ADMM and LADMM in the implementation.

After selecting the optimal λ, we refit the model on the full training set. Notably, although our analysis provides a lower bound on ρ as a function of λ (see Supplementary Appendix C), we adopt the simpler choice ρ = 1 in all runs. This is motivated by the empirical observation that the optimal λ values are relatively small, for which ρ = 1 is sufficient to ensure stable convergence in practice.

We assess computational efficiency by examining the objective function value (Equation 45) vs. elapsed time. Figure 5 shows that LADMM descends the fastest. The computational advantage of LADMM is consistent with its closed-form z-update, which avoids the iterative inner solves required by ADMM.

Then we evaluate the training performance on an independent test dataset. Consider a matrix: xte∈ℝmte×n with entries (xte)ij~iidN(0,1), where m_te is the number of test samples and we set m_te = 50. Using the training means μ, we center the data matrix [48] by A_te = X_te−1μ. Then the true test labels are generated from the ground truth:

bte,i=sign(ate,i⊤ x*), i=1,…,mte.

Suppose x is an estimated solution, we obtain the predicted labels b^ given by

b^i={+1,(Atex)i≥0,−1,(Atex)i<0,(48)

for i = 1, …m_te. Using the true labels b_te and predicted values b^, we define the confusion matrix, i.e.,

TP=|{i:b^i=+1,bte,i=+1}|,FP=|{i:b^i=+1,bte,i=-1}|,TN=|{i:b^i=-1,bte,i=-1}|,FN=|{i:b^i=-1,bte,i=+1}|,(49)

based on which we calculate the following evaluation metrics:

Accuracy=TP+TNmte, Precision=TPTP+FP

Recall=TPTP+FN.(50)

To further connect the numerical results with the stationarity guarantees in Section 3, we also report a proximal-gradient mapping residual [49] at the final iterate. For a final iterate x^* returned by each method, we define

ResPG(x)=1n||1η[x-proxlog(x-η∇L(x);ηλ)]||2,

where η is the step size and the closed form expression of prox_log is provided in Supplementary Appendix C. This residual measures how much the final iterate would change after one proximal-gradient step. If the residual is zero, then one proximal-gradient step leaves the point unchanged. Thus, smaller values of the residual indicate that the final iterate is closer to satisfying d-stationarity.

These metrics, together with computation time, and the proximal-gradient mapping residual are reported in Table 3 as mean (standard deviation) over 30 independent train/test instances. The results show that our method is the fastest (in alignment with Figure 5), while still achieving comparable classification performance, with accuracy, precision, and recall all around 0.97 and standard deviations in the range of 0.02–0.05. In addition, LADMM attains a comparable proximal-gradient mapping residual. Among the newly included baselines, PLADMM performs similarly to LADMM in terms of accuracy, recall, precision, and Res_PG, but has a slightly larger computational time. PALM also remains faster than ADMM and MM in this experiment, although its stationarity residual is larger. DCA attains slightly higher average classification metrics, but the differences are small relative to the reported standard deviations, and its average computational time is higher than that of LADMM. Taken together, these findings demonstrate that LADMM achieves competitive predictive quality and comparable stationarity residuals while delivering the lowest computational cost.

Table 3. Computation time and performance metrics for LOG-regularized logistic regression (Equation 45) with m_tr = 200 training samples, n = 50 features, and m_te = 50 test samples.

Values are mean over 30 random instances, with standard deviations shown in parentheses. For each random instance, the regularization parameter λ is selected by K-fold (we set K = 5) cross validation over a prescribed candidate set. Other parameter settings are the same as in Figure 5.

To further examine the computational behavior of the methods as the problem size increases, we perform a scaling study for LOG-regularized logistic regression (Equation 45). We set m = 4n and test n∈{50, 100, 200, 300} and we fix ρ = 1, λ = 0.0063, and ϵ = 0.01. Table 4 reports the average computational time over 10 random instances. The results show that LADMM remains among the fastest methods over the tested range. These results provide additional evidence of the computational efficiency of LADMM.

Table 4. Scalability comparison for LOG-regularized logistic regression (Equation 45) with m = 4n and n∈{50, 100, 200, 300}.

Values are mean over 10 random instances, with standard deviations shown in parentheses. Parameters are fixed at ρ = 1, λ = 0.0063, and ϵ = 0.01. Other implementation settings are the same as in Figure 5.

5 Conclusion

We studied an optimization problem involving the sum of two functions (Equation 1), where one function is convex and differentiable and the other is weakly convex (but not necessarily differentiable). This setting naturally captures a broader family of optimization models that go beyond the convex regime. Numerically, we applied both ADMM and its linearized variant to solve the problem, with the latter reducing the need for nested loops. Theoretically, we showed that both ADMM and LADMM iterates admit a subsequence converging to a directional stationary solution under mild conditions. Both convergence guarantees rely on explicit and verifiable conditions on the penalty parameter: ρ>max{σg,2Lip∇f} for ADMM (Theorem 3) and ρ>max{σ_*g*, 1+Lip∇f, Lip∇f(1+2Lip∇f)} for LADMM (Theorem 4). The stronger condition for LADMM reflects the additional control needed to absorb the lagged-gradient effect introduced by linearization. Numerical experiments on two low-dimensional smooth functions and a high-dimensional logistic regression problem demonstrate the practical efficiency of the proposed ADMM and LADMM algorithms compared with baseline methods. In the LOG-regularized logistic regression setting, LADMM achieved classification performance and proximal-gradient mapping residuals comparable to the competing baselines, while having the lowest average computational time. These results suggest that LADMM provides an efficient strategy for this weakly convex regularized logistic-regression model.

Some promising extensions include inexact updates that are attractive for large-scale learning problems, acceleration and preconditioning techniques within the weakly convex ADMM framework, and extending the convergence analysis to cover linearization of the weakly convex term (as employed in Sections 4.1 and 4.2).

Generative AI statement

The author(s) declared that Generative AI was used in the creation of this manuscript. The authors used OpenAI's ChatGPT to assist with language editing, improving clarity, and revising portions of the manuscript text. All AI-assisted content was reviewed and verified by the authors, who take full responsibility for the accuracy and integrity of the manuscript.

Any alternative text (alt text) provided alongside figures in this article has been generated by Frontiers with the support of artificial intelligence and reasonable efforts have been made to ensure accuracy, including review by the authors wherever possible. If you identify any issues, please contact us.

Publisher's note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fams.2026.1898659/full#supplementary-material

Supplementary material Supplementary material at the source

Footnotes

Edited by: Lixin Shen, Syracuse University, United States

Reviewed by: Valentina De Simone, University of Campania Luigi Vanvitelli, Italy

Zhengwei Shen, University of Science and Technology Beijing, China