Mean-Field Stochastic Linear-Quadratic Optimal Control Problems: Weak Closed-Loop Solvability

This paper is concerned with mean-field stochastic linear-quadratic (MF-SLQ, for short) optimal control problems with deterministic coefficients. The notion of weak closed-loop optimal strategy is introduced. It is shown that the open-loop solvability is equivalent to the existence of a weak closed-loop optimal strategy. Moreover, when open-loop optimal controls exist, there is at least one of them admitting a state feedback representation, which is the outcome of a weak closed-loop optimal strategy. Finally, an example is presented to illustrate the procedure for finding weak closed-loop optimal strategies.

According to the standard results of mean-field SDEs (see [15], for example), under some mild conditions, for any (t, ξ) ∈ D and u(·) ∈ U[t, T ], equation (1) admits a unique solution X(·) ≡ X(· ; t, ξ, u(·)). To measure the performance of the control u(·), we introduce the following quadratic cost functional: J(t, ξ; u(·)) = E GX(T ), X(T ) + 2 g, X(T ) where G,Ḡ ∈ R n×n are symmetric constant matrices; g is an F T -measurable R nvalued random vector andḡ is a (deterministic) ds , (6) respectively. In this case, we denote the corresponding mean-field stochastic LQ problem and its value function by Problem (MF-SLQ) 0 and V 0 (· , ·), respectively. When the mean-field part vanishes, Problem (MF-SLQ) becomes a classical stochastic LQ optimal control problem, which has been well studied by many researchers; see, for example, [13,1,2,16,4,7,3,9,11] and the references cited therein. LQ optimal control problems for MF-SDEs over a finite horizon were first studied by Yong [14], and were later extended to the infinite horizon by Huang, Li, and Yong [5]. Recently, based on the idea of [10,9], Sun [8] and Li, Sun, and Yong [6] investigated the open-loop and closed-loop solvabilities for Problem (MF-SLQ) and found that these two types of solvabilities are essentially different. More precisely, they showed in [6] that the closed-loop solvability of Problem (MF-SLQ) is equivalent to the existence of a regular solution to the following generalized Riccati equation (GRE, for short): (where M † denotes the Moore-Penrose pseudoinverse of a matrix M and the argument s is suppressed), and that the closed-loop solvability implies the open-loop solvability of Problem (MF-SLQ), but not vice-versa. The advantage of the existence of a closed-loop optimal strategy is that a state feedback optimal control, which is the outcome of some closed-loop optimal strategy, can be explicitly constructed in terms of the solution to (7). However, as just mentioned, Problem (MF-SLQ) might be merely open-loop solvable, in which case solving the GRE (7) will fail to produce a state feedback optimal control. To see this, let us consider the following example.
In this example, the associated GRE reads Ṗ (s) + 3P (s) = 0,Π(s) + 4Π(s) + 4P (s) = 0, s ∈ [t, 1], It is easy to verify that the unique solution of (8) is (P (s), Π(s)) ≡ (0, e 4−4s ), which, however, is not regular according to the definition in [6]. If we use the usual Riccati equation approach to construct the state feedback optimal control u * (·), then u * (·) should be given by the following (noting that R(·) = 0,R(·) = 0, D(·) = 0,D(·) = 0 and 0 † = 0): Such a control is not open-loop optimal if the initial state ξ satisfies E[ξ] = 0. Indeed, by the variation of constants formula, the expectation of the state process X * (·) corresponding to (t, ξ) and u * (·) is given by On the other hand, letū(·) be the control defined bȳ By the variation of constants formula, the expectation of the state processX(·) corresponding to (t, ξ) andū(·) is given by which satisfies E[X(1)] = 0. Hence, Since the cost functional is nonnegative,ū(·) is an open-loop optimal control for the initial pair (t, ξ), but u * (·) is not. Now some questions arise naturally: When Problem (MF-SLQ) is merely openloop solvable, does state feedback optimal control exists? If yes, how can we find such an optimal control? The objective of this paper is to answer these questions. We shall first provide an alternative characterization of the open-loop solvability for Problem (MF-SLQ) using the perturbation approach introduced by Sun, Li, and Yong [9]. This characterization, which avoids the subsequence extraction, is a refinement of [8,Theorem 3.2]. Then we generalize the notion of weak closed-loop strategies, which was first introduced by Wang, Sun, and Yong [11] for classical stochastic LQ problems and then investigated by Wen, Li, and Xiong [12] for Markovian regime switching system, to the mean-field case. We shall show that the existence of a weak closed-loop optimal strategy is equivalent to the existence of an open-loop optimal control, and that as long as Problem (MF-SLQ) is open-loop solvable, a state feedback optimal control always exists and can be represented as the outcome of a weak closed-loop optimal strategy. Moreover, our constructive proof provides a procedure for finding weak closed-loop optimal strategies. The rest of this paper is organized as follows. In section 2, we collect some preliminary results and introduce a few elementary notions for Problem (MF-SLQ). section 3 is devoted to the study of open-loop solvability by a perturbation method. In section 4, we show how to obtain a weak closed-loop optimal strategy and establish the equivalence between open-loop and weak closed-loop solvabilities. An example is presented in section 5 to illustrate the results we obtained.
For M, N ∈ S n , we use the notation M N (respectively, M > N ) to indicate that M − N is positive semi-definite (respectively, positive definite). Further, for any S n -valued measurable function F on [t, T ], we denote For the state equation (1) and cost functional (2), we introduce the following assumptions: (H1). The coefficients and the nonhomogeneous terms of the state equation (1) satisfy . The weighting coefficients in the cost functional (2) satisfy Under assumption (H1), we have the following well-posedness of state equation (1), whose proof is standard and can be found in [15, Proposition 2.1].
Motivated by Example 1.1 and [11], we now introduce the notion of weak closedloop strategies for mean-field stochastic LQ problems.

Remark 1.
We emphasize that the closed-loop strategies of Problem (MF-SLQ) need to be square-integrable but the weak closed-loop strategies are only required to be locally square-integrable. Thus the weak closed-loop strategies need less regularities than closed-loop strategies.
Similar to the case of classical stochastic LQ problem (see [11], for example), we have the following equivalence: In the sequel, we shall use the following result, which is concerned with the open-loop and closed-loop solvabilities of Problem (MF-SLQ). For more details and proofs of this result, the interested reader is referred to Sun [8].

A perturbation approach to open-loop solvability. In this section, we
shall study the open-loop solvability of Problem (MF-SLQ) under the following convexity condition: By Theorem 2.5 (i), the above condition is necessary for the open-loop solvability of Problem (MF-SLQ). Condition (18) means that the mapping u(·) → J 0 (0, 0; u(·)) is convex. Indeed, under condition (18), one can obtain that the mapping u(·) → J(t, ξ; u(·)) is convex for any initial pair (t, ξ) ∈ D. We point out that the condition (18)  For any ε > 0, let us consider the LQ problem of minimizing the perturbed cost functional subject to the state equation (1). We denote this perturbed LQ problem by Problem (MF-SLQ) ε and its value function by V ε (· , ·). Then the cost functional of the homogeneous LQ problem associated with Problem (MF-SLQ) ε is give by It follows from (18) that By Theorem 2.5 (ii), the above implies that the Riccati equation where with (η ε (·), ζ ε (·)) being the unique adapted solution to the BSDE: andη ε (·) being the solution to the following ODE: (26) Let X ε (·) be the unique solution to the closed-loop system then the unique open-loop optimal control of Problem (MF-SLQ) ε for (t, ξ) is given by We now are ready to state and prove the main result of this section, which provides a characterization of the open-loop solvability of Problem (MF-SLQ) in terms of the family {u ε (·)} ε>0 defined by (28).
The implication (iii) ⇒ (ii) is trivially true. Finally, we prove the implication (ii) ⇒ (iii). The proof is divided into two steps.

This means that
is also an optimal control of Problem (MF-SLQ) for (t, ξ). Then we can repeat the argument employed in the proof of (i) ⇒ (ii), replacing v * (·) by , to obtain (see (30)) Taking inferior limits on the both sides of the above inequality then yields Adding the above two inequalities and then multiplying by 2, we get which is equivalent to Thus, u * 1 (·) = u * 2 (·) and our claim is proved.

Remark 2.
A similar result first appeared in [9] for the classical stochastic LQ problem. Afterwards, Sun [8] generalized it to the mean-field case. More precisely, they found that if Problem (MF-SLQ) is open-loop solvable at (t, ξ), then the limit of any weakly/strongly convergent subsequence of {u ε (·)} ε>0 is an open-loop optimal control. Our result refines that in [8] by showing the family {u ε (·)} ε>0 itself is strongly convergent when Problem (MF-SLQ) is open-loop solvable.
The following result shows that {Θ ε (·)} ε>0 defined by (21) is locally convergent in (0, T ). Proof. To do this, we only need to show that for any 0 < t < T < T , the family {Θ ε (·)} ε>0 is Cauchy in L 2 (t, T ; R m×n ). For any 0 < t < T and ξ ∈ L 2 Ft (Ω; R n ) with E[ξ] = 0, let X ε (·) be the unique solution to the closed-loop system Taking an expectation on the both sides of the above, we have which implies that E[X ε (s)] ≡ 0; t s T . Then the state equation (37) can be rewritten as Let Φ ε (·) ∈ L 2 F (Ω; C([t, T ]; R n×n ) be the solution to the following SDE: Clearly, for any initial state ξ, the solution of (39) (or (37)) can be expressed by Since Problem (MF-SLQ) 0 is open-loop solvable, by Theorem 3.1, the family . Note that Φ ε (·) is independent of F t , then for any ε 1 , ε 2 > 0, ]} ε>0 converges strongly in L 2 (t, T ; R m×n ) as ε → 0. Denote the limit of U ε (·) by U * (·). One sees that E[Φ ε (·)] satisfies the following ODE: which shows that {Θ ε (·)} ε>0 is Cauchy in L 2 (t, t + ∆ t ; R m×n ). Similar to the last paragraph in the proof of Proposition 1, we can obtain that {Θ ε (·)} ε>0 is Cauchy in L 2 (t, T ; R m×n ) by using the compactness argument.