- Research
- Open Access
- Published:
Mean-field stochastic linear quadratic optimal control problems: closed-loop solvability
Probability, Uncertainty and Quantitative Risk volume 1, Article number: 2 (2016)
Abstract
An optimal control problem is studied for a linear mean-field stochastic differential equation with a quadratic cost functional. The coefficients and the weighting matrices in the cost functional are all assumed to be deterministic. Closed-loop strategies are introduced, which require to be independent of initial states; and such a nature makes it very useful and convenient in applications. In this paper, the existence of an optimal closed-loop strategy for the system (also called the closed-loop solvability of the problem) is characterized by the existence of a regular solution to the coupled two (generalized) Riccati equations, together with some constraints on the adapted solution to a linear backward stochastic differential equation and a linear terminal value problem of an ordinary differential equation.
Introduction
Let \((\Omega,{\mathcal {F}},\mathbb {F},\mathbb {P})\) be a complete filtered probability space on which a standard one-dimensional Brownian motion W={W(t);0≤t<∞} is defined, where \(\mathbb {F}=\{{\mathcal {F}}_{t}\}_{t\geqslant 0}\) is the natural filtration of W augmented by all the \(\mathbb {P}\)-null sets in \({\mathcal {F}}\). Consider the following controlled linear mean-field stochastic differential equation (MF-SDE, for short) on a finite time horizon [t,T]:
where \(A(\cdot), \bar {A}(\cdot), B(\cdot), \bar {B}(\cdot), C(\cdot), \bar {C}(\cdot), D(\cdot), \bar {D}(\cdot)\) are given deterministic matrix-valued functions; b(·),σ(·) are vector-valued \(\mathbb {F}\)-progressively measurable processes and ξ is an \({\mathcal {F}}_{t}\)-measurable random vector. In the above, u(·) is the control process and X(·) is the corresponding state process with initial pair (t,ξ). For any t∈[0,T), we define
Any \(u(\cdot)\in {\mathcal {U}}[t,T]\) is called an admissible control (on [t,T]). Under some mild conditions, for any initial pair (t,ξ) with \(\xi \in L^{2}_{{\mathcal {F}}_{t}}(\Omega ;\mathbb {R}^{n})\) (the set of all \({\mathcal {F}}_{t}\)-measurable, square-integrable \(\mathbb {R}^{n}\)-valued processes), and any admissible control \(u(\cdot)\in {\mathcal {U}}[t,T]\), (1.1) admits a unique square-integrable solution X(·)≡X(· ;t,ξ,u(·)). Now we introduce the following cost functional:
where G, \(\bar {G}\) are symmetric matrices and \(Q(\cdot), \bar {Q}(\cdot), S(\cdot), \bar {S}(\cdot), R(\cdot), \bar {R}(\cdot)\) are deterministic matrix-valued functions with Q(·), \(\bar {Q}(\cdot)\), R(·), and \(\bar {R}(\cdot)\) being symmetric; g is an \({\mathcal {F}}_{t}\)-measurable random vector and \(\bar {G}\) is a deterministic vector; q(·), ρ(·) are vector-valued \(\mathbb {F}\)-progressively measurable processes and \(\bar {Q}(\cdot)\), \(\bar {\rho }(\cdot)\) are vector-valued deterministic functions. Our mean-field stochastic linear quadratic (LQ, for short) optimal control problem can be stated as follows:
Problem (MF-LQ). For any given initial pair \((t,\xi)\in [0,T)\times L^{2}_{{\mathcal {F}}_{t}}(\Omega ;\mathbb {R}^{n})\), find a \(u^{*}(\cdot)\in {\mathcal {U}}[t,T]\) such that
Any \(u^{*}(\cdot)\in {\mathcal {U}}[t,T]\) satisfying (1.3) is called an optimal open-loop control of Problem (MF-LQ) for the initial pair (t,ξ), and the corresponding X ∗(·)≡X(· ;t,ξ,u ∗(·)) is called an optimal open-loop state process. The function V(·,·) is called the value function of Problem (MF-LQ). In the special case where b(·), σ(·), g, \(\bar g\), q(·), \(\bar q(\cdot)\), ρ(·), and \(\bar \rho (\cdot)\) vanish, we denote the corresponding mean-field LQ problem, cost functional, and value function by Problem (MF-LQ)0, J 0(t,ξ;u(·)), and V 0(t,ξ), respectively.
The theory of MF-SDEs can be traced back to Kac who presented a stochastic toy model for the Vlasov kinetic equation of plasma in (Kac 1956) which leads to the so-called McKean-Vlasov stochastic differential equation. Since then, the research on the related topics and their applications has become a notable and serious endeavor among researchers in applied probability and optimal stochastic controls, including financial engineering. See, for examples, Andersson and Djehiche (2011), Buckdahn et al. (2009), Buckdahn et al. (2009), Buckdahn et al. (2011), Cui et al. (2014), Elliott et al. (2013), Huang et al. (2015), Huang et al. (2015), McKean (1966), Meyer-Brandis et al. (2012), Yong (2013). Note that when the mean-field part is absent, Problem (MF-LQ) is reduced to the classical stochastic LQ optimal control problem. For relevant results and historic remarks on this subject, the reader is further referred to, for examples, (Ait Rami et al. 2001; Chen and Yong 2000; Chen et al. 1998; Tang 2003; Wonham 1968) and the book of Yong and Zhou (1999).
Recently, Sun and Yong introduced the notions of open-loop and closed-loop solvabilities for stochastic LQ problems (Sun and Yong 2014). See also (Sun et al. 2016; Sun et al. 2015). It turns out that these two notions are essentially different for stochastic LQ problem on finite time horizon. Roughly speaking, the open-loop solvability is equivalent to the solvability of a forward-backward stochastic differential equation (FBSDE, for short), and the closed-loop solvability is equivalent to the existence of a regular solution to a Riccati equation. Open-loop solvability was studied for Problem (MF-LQ) in (Sun 2016). The current work is therefore a continuation of the above-mentioned works.
The rest of the paper is organized as follows. In Section “Preliminaries” we give some preliminaries, carefully explain the closed-loop strategies, and introduce the regular solution to the generalized Riccati equation. Section “Necessary Conditions for Closed-Loop Solvability” is devoted to the necessary conditions for the existence of an optimal closed-loop strategy. In Section “Characterization of Closed-Loop Solvability”, we present our main result, in which the closed-loop solvability of the mean-field LQ problems is characterized. Finally, some concluding remarks are given in Section “Conclusion”.
Preliminaries
We begin with some notations that will be used throughout the paper:
-
\(\mathbb {R}^{n\times m}\): the Euclidean space of all n×m real matrices.
-
\(\mathbb {R}^{n}=\mathbb {R}^{n\times 1}\) and \(\mathbb {R}=\mathbb {R}^{1}\).
-
\(\mathbb {S}^{n}\): the space of all symmetric n×n real matrices.
-
M ⊤: the transpose of a matrix M.
-
M †: the Moore–Penrose pseudoinverse of a matrix M (Penrose 1955).
-
tr(M): the sum of diagonal elements of a square matrix M.
-
〈·,·〉: the inner product on a Euclidean space given by 〈M,N〉↦tr(M ⊤ N).
-
\(|M|\triangleq \sqrt {\text {tr}(MM^{\top })}\): the Frobenius norm of a matrix M.
-
\({\mathcal {R}}(M)\): the range of a matrix M.
For \(M,N\in \mathbb {S}^{n}\), we use the notation \(M\geqslant N\) (respectively, M>N) to indicate that M−N is positive semi-definite (respectively, positive definite). For any \(\mathbb {S}^{n}\)-valued measurable function F on [t,T], we write
-
\(F\geqslant 0 \quad \Longleftrightarrow \quad F(s)\geqslant 0, \quad \text {a.e.}~s\in [t,T]\),
-
F>0 ⇔ F(s)>0, a.e. s∈[t,T],
-
\(F\gg 0 \quad \Longleftrightarrow \quad F(s)\geqslant \delta I, \quad \text {a.e.}~s\in [t,T],\text {for some}~\delta >0\).
Let f(·) be a function from \(\mathbb {R}^{n\times m}\) into \(\mathbb {R}\). Recall that the gradient of f at X=(x ij ), denoted by \(\frac {{\partial } f(X)}{{\partial } X}\), is an n×m matrix whose (i,j)-th entry is given by \(\left [{{\partial } f(X)\over {\partial } X}\right ]_{ij}={{\partial } f(X)\over {\partial } x_{ij}}\). For matrices L, M, and N of proper dimensions, the following formulae hold:
For a Euclidean space \(\mathbb {H}\), let \(L^{p}(t,T;\mathbb {H}) (1\leqslant p\leqslant \infty)\) be the space of all \(\mathbb {H}\)-valued functions that are L p-integrable on [t,T], and let \(C([t,T];\mathbb {H})\) be the space of all \(\mathbb {H}\)-valued continuous functions on [t,T]. We denote
The following assumptions will be in force throughout this paper.
(H1) The coefficients of the state equation satisfy the following:
(H2) The weighting coefficients in the cost functional satisfy the following:
A standard argument using the contraction mapping theorem shows that under (H1), for any initial pair \((t,\xi)\in [0,T)\times L^{2}_{{\mathcal {F}}_{t}}(\Omega ;\mathbb {R}^{n})\) and any admissible control \(u(\cdot)\in {\mathcal {U}}[t,T]\), (1.1) admits a unique (strong) solution \(X(\cdot)\equiv X(\cdot \,; t,\xi,u(\cdot))\in L_{\mathbb {F}}^{2}(\Omega ;C([t,T];\mathbb {R}^{n}))\). Hence, under (H1)–(H2), the cost functional (1.2) is well-defined, and Problem (MF-LQ) makes sense.
Let us now recall the notion of open-loop solvability from Sun (2016), which was inspired by (Sun and Yong 2014).
Definition 2.1
Problem (MF-LQ) is said to be (uniquely) open-loop solvable at initial pair \((t,\xi)\in [0,T]\times L^{2}_{{\mathcal {F}}_{t}}(\Omega ;\mathbb {R}^{n})\) if there exists a (unique) \(u^{*}(\cdot)\in {\mathcal {U}}[t,T]\) satisfying (1.3). Problem (MF-LQ) is said to be (uniquely) open-loop solvable at t if for any \(\xi \in L^{2}_{{\mathcal {F}}_{t}}(\Omega ;\mathbb {R}^{n})\), there exists a (unique) \(u^{*}(\cdot)\in {\mathcal {U}}[t,T]\) satisfying (1.3), and Problem (MF-LQ) is said to be (uniquely) open-loop solvable on [t,T) if it is (uniquely) open-loop solvable at all s∈[t,T).
Next, we introduce the following definition.
Definition 2.2
(i) Let
Any triple 
is called a closed-loop strategy of Problem (MF-LQ) on [t,T].
(ii) For any 
and \(\xi \in L^{2}_{{\mathcal {F}}_{t}}(\Omega ;\mathbb {R}^{n})\), let \(X(\cdot)\equiv X(\cdot \,;t,\xi,\Theta (\cdot),\bar \Theta (\cdot),v(\cdot))\) be the solution to the following closed-loop system:
and let
Then (X(·),u(·)) is called the outcome pair of \((\Theta (\cdot),\bar {\Theta }(\cdot),v(\cdot))\)on [t,T] corresponding to the initial state ξ; X(·) and u(·) are called the corresponding closed-loop state process and closed-loop outcome control process, respectively.
Note that if (X(·),u(·)) is the outcome pair of a closed-loop strategy 
corresponding to some \(\xi \in L^{2}_{{\mathcal {F}}_{t}}(\Omega ;\mathbb {R}^{n})\), then \((X(\cdot),u(\cdot))\in L^{2}_{\mathbb {F}}(\Omega ;C([t,T];\mathbb {R}^{n})\times {\mathcal {U}}[t,T]\) is actually a state-control pair of the state Eq. (1.1). In fact, (2.1) is equivalent to the following:
Therefore, for any 
, the meaning of \(J(t,\xi ;\Theta (\cdot)X(\cdot)+\bar {\Theta }(\cdot)\mathbb {E}[X(\cdot)]+v(\cdot))\) is clear. We point out that a closed-loop strategy 
is not related to any initial state \(\xi \in L^{2}_{{\mathcal {F}}_{t}}(\Omega ;\mathbb {R}^{n})\), whereas, an outcome pair (X(·),u(·)) of a closed-loop strategy \((\Theta (\cdot),\bar {\Theta }(\cdot),v(\cdot))\) depends not only on the closed-loop strategy, but also on the initial state ξ. Hence, we should carefully distinguish the closed-loop strategy and the corresponding outcome control. Now, we are ready to introduce the following notion.
Definition 2.3
(i) A closed-loop strategy 
is said to be optimal on [t,T] if
where X ∗(·) and X(·) are the closed-loop state processes corresponding to \((\Theta ^{*}(\cdot),\bar {\Theta }^{*}(\cdot),v^{*}(\cdot),\xi)\) and (Θ(·), \(\bar {\Theta }(\cdot),v(\cdot),\xi)\), respectively. If an optimal closed-loop strategy (uniquely) exists on [t,T], Problem (MF-LQ) is said to be (uniquely) closed-loop solvable on [t,T].
(ii) A closed-loop strategy 
is said to be weakly optimal on [t,T] if (??) holds only for \(\xi =x\in \mathbb {R}^{n}\). If a weakly optimal closed-loop strategy (uniquely) exists on [t,T], Problem (MF-LQ) is said to be (uniquely) weakly closed-loop solvable on [t,T].
Similar to (Sun and Yong 2014), we have the following proposition.
Proposition 2.4
Let (H1)–(H2) hold and let 
. Then the following statements are equivalent:
(i) \((\Theta ^{*}(\cdot),\bar {\Theta }^{*}(\cdot),v^{*}(\cdot))\) is an optimal closed-loop strategy of Problem (MF-LQ) on [t,T];
(ii) The following holds:
where X ∗(·) and X(·) are the closed-loop state processes corresponding to \((\Theta ^{*}(\cdot),\bar {\Theta }^{*}(\cdot),v^{*}(\cdot),\xi)\) and (Θ ∗(·), \(\bar {\Theta }^{*}(\cdot),v(\cdot),\xi)\), respectively;
(iii) The following holds:
where X ∗(·)is the closed-loop state process corresponding to \((\Theta ^{*}(\cdot),\bar \Theta ^{*}(\cdot),v^{*}(\cdot))\) and \(\xi \in L^{2}_{{\mathcal {F}}_{t}}(\Omega ;\mathbb {R}^{n})\).
Proof.
(i) \(\mathop {\Rightarrow }\) (ii) is trivial, by taking Θ(·)=Θ ∗(·) and \(\bar {\Theta }(\cdot)=\bar {\Theta }^{*}(\cdot)\) in (??).
(ii) \(\mathop {\Rightarrow }\) (iii). For any \(\xi \in L^{2}_{{\mathcal {F}}_{t}}(\Omega ;\mathbb {R}^{n})\) and \(u(\cdot)\in {\mathcal {U}}[t,T]\), let X(·) be the solution to the following:
and set
Then X(·) is also the solution to the following MF-SDE:
Therefore,
(iii) \(\mathop {\Rightarrow }\) (i). For any \(\xi \in L^{2}_{{\mathcal {F}}_{t}}(\Omega ;\mathbb {R}^{n})\) and 
, let X(·) be the solution to the following MF-SDE:
Set
Then, by uniqueness, X(·) also solves MF-SDE (2.4). Thus,
This completes the proof. □
With the same proof, we have the following result for the weakly optimal closed-loop strategies.
Proposition 2.5
Let (H1)–(H2) hold and let 
. Then the following statements are equivalent:
(i) \((\Theta ^{*}(\cdot),\bar {\Theta }^{*}(\cdot),v^{*}(\cdot))\) is a weakly optimal closed-loop strategy of Problem (MF-LQ) on [t,T];
(ii) The following holds:
where X ∗(·) and X(·) are the closed-loop state processes corresponding to \((\Theta ^{*}(\cdot),\bar {\Theta }^{*}(\cdot),v^{*}(\cdot),x)\) and (Θ ∗(·), \(\bar {\Theta }^{*}(\cdot),v(\cdot),x)\), respectively;
(iii) The following holds:
where X ∗(·) is the closed-loop state process corresponding to \((\Theta ^{*}(\cdot),\bar {\Theta }^{*}(\cdot),v^{*}(\cdot))\) and \(x\in \mathbb {R}^{n}\).
Remark 2.6
(i) An optimal open-loop control is allowed to depend on the initial state, whereas an optimal closed-loop strategy is required to be independent of the initial state.
(ii) It is clear from Proposition 2.4 (iii) that the outcome control \(u^{*}(\cdot)\equiv \Theta ^{*}(\cdot)X^{*}(\cdot)+\bar {\Theta }^{*}(\cdot)\mathbb {E}[X^{*}(\cdot)]+v^{*}(\cdot)\) of an optimal closed-loop strategy \((\Theta ^{*}(\cdot),\bar {\Theta }^{*}(\cdot),v^{*}(\cdot))\) is an optimal open-loop control of Problem (MF-LQ) for the initial pair (t,X ∗(t)). Hence, closed-loop solvability implies open-loop solvability.
(iii) Obviously, an optimal closed-loop strategy on [t,T]is also weakly optimal. For the classical LQ optimal control problems where \(\mathbb {E}[X(\cdot)]\) and \(\mathbb {E}[u(\cdot)]\) are absent, it can be shown, using the results from Sun and Yong ( 2014 ) and a completion of squares technique, that the two concepts coincide. But for Problem (MF-LQ), the existence of a weakly optimal closed-loop strategy does not guarantee the existence of an optimal closed-loop strategy. To see this, we present the following example.
Example 2.7
Consider the following one-dimensional state equation
and cost functional
For any \(x\in \mathbb {R}\) and \(u(\cdot)\in {\mathcal {U}}[t,1]\), we have
On the other hand, it is clear that \((\Theta ^{*}(\cdot),\bar {\Theta }^{*}(\cdot),v^{*}(\cdot))\equiv (0,0,0)\) satisfies
Thus, by Proposition 2.5, (0,0,0)is a weakly optimal closed-loop strategy of the problem on [t,1].
Let us now show that the above problem does not admit an optimal closed-loop strategy on any [t,1]with 0<t<1. Assume the contrary; i.e., let
satisfy (2.3) on [t,1]for some 0<t<1. For any
\(x\in \mathbb {R}\), take
ξ=W(t)x
and
\(u(s)\equiv {W(t)x\over t-1}\). The corresponding solution of (2.5) is
Note that X(1)=0. Thus, (2.3) implies that
where X ∗(·)is the solution to the following closed-loop system:
It follows that
Note that \(\mathbb {E}[X^{*}(s)]\equiv 0\). Then,
But this is impossible since it has to be true for all \(x\in \mathbb {R}\).
We conclude this section by introducing the coupled generalized Riccati equation (GRE, for short), whose regular solvability will turn out to be necessary and sufficient for the closed-loop solvability of Problem (MF-LQ) in the next two sections.
The GRE associated with Problem (MF-LQ) consists of two coupled nonlinear differential equations of the following form (for simplicity of notation, we will usually suppress the time variable s below):
Definition 2.8
A solution \((P(\cdot),\Pi (\cdot))\in C([t,T];\mathbb {S}^{n})\times C([t,T];\mathbb {S}^{n})\) of (2.6) is said to be regular if
and
The GRE (2.6) is said to be regularly solvable on [t,T] if it admits a regular solution.
Necessary Conditions for Closed-Loop Solvability
In this section we will deduce necessary conditions for the closed-loop solvability of Problem (MF-LQ). In particular, we shall establish the necessity of the regular solvability of GRE (2.6) by a matrix minimum principle.
Let \(\Theta ^{*}(\cdot),\bar \Theta ^{*}(\cdot)\in L^{2}(t,T;\mathbb {R}^{m\times n})\) and consider the following state equation
and cost functional
where
By Proposition 2.4 (ii), 
is an optimal closed-loop strategy of Problem (MF-LQ) on [t,T] if and only if for any \(\xi \in L^{2}_{{\mathcal {F}}_{t}}(\Omega ;\mathbb {R}^{n})\), u
∗(·) is an optimal open-loop control of the problem with the above state equation and cost functional. This leads to the following result.
Proposition 3.1
Let (H1)–(H2) hold. If 
is an optimal closed-loop strategy of Problem (MF-LQ) on [t,T], then \((\Theta ^{*}(\cdot),\bar \Theta ^{*}(\cdot),0)\) is an optimal closed-loop strategy of Problem (MF-LQ)
0 on [t,T].
Proof.
By the preceding discussion and (Theorem 2.3, Sun 2016), we see that \((\Theta ^{*}(\cdot),\bar \Theta ^{*}(\cdot),u^{*}(\cdot))\) is an optimal closed-loop strategy of Problem (MF-LQ) on [t,T] if and only if for any \(\xi \in L^{2}_{{\mathcal {F}}_{t}}(\Omega ;\mathbb {R}^{n})\), the adapted solution (X ∗(·),Y ∗(·),Z ∗(·)) to the following mean-field forward-backward stochastic differential equation (MF-FBSDE, for short):
satisfies
and the following condition holds:
where X(·) is the solution of
Since the MF-FBSDE (3.1) admits a solution for each \(\xi \in L^{2}_{{\mathcal {F}}_{t}}(\Omega ;\mathbb {R}^{n})\) and \((\Theta ^{\ast }(\cdot),\bar \Theta ^{\ast }(\cdot),u^{\ast }(\cdot))\) is independent of ξ, by subtracting solutions corresponding ξ and 0, the latter from the former, we see that for any \(\xi \in L^{2}_{{\mathcal {F}}_{t}}(\Omega ;\mathbb {R}^{n})\), the following MF-FBSDE:
also admits an adapted solution (X(·),Y(·),Z(·)) satisfying
It follows, again from (Theorem 2.3, Sun2016), that \((\Theta ^{\ast }(\cdot),\bar \Theta ^{\ast }(\cdot),0)\) is an optimal closed-loop strategy of Problem (MF-LQ)0 on [t,T]. □
Now let us look at Problem (MF-LQ)0. If we consider only closed-loop strategies of the form \((\Theta (\cdot),\bar \Theta (\cdot),0)\), then the state equation becomes
and \(\mathbb {E}[X(\cdot)]\) satisfies
By Itô’s formula, the matrices \(\mathbf {X}(s)\triangleq \mathbb {E}\left [X(s)X(s)^{\top }\right ]\) and \(\mathbf {Y}(s)\triangleq \mathbb {E}[X(s)]\mathbb {E}[X(s)]^{\top }\) satisfy the matrix-valued ordinary differential equations (ODEs, for short)
and
respectively. The cost functional \(J^{0}(t,\xi ;\Theta (\cdot)X(\cdot)+\bar \Theta (\cdot)\mathbb {E}[X(\cdot)])\) can be expressed equivalently as
where
Then we may pose the following deterministic optimal control problem.
Problem (O). For any given \((t,\xi)\in [0,T)\times L^{2}_{{\mathcal {F}}_{t}}(\Omega ;\mathbb {R}^{n})\), find \(\Theta ^{\ast }(\cdot),\bar \Theta ^{\ast }(\cdot)\in L^{2}(t,T;\mathbb {R}^{m\times n})\) such that
and denote the integrand in (3.5) by \(L(\mathbf {X}(s),\mathbf {Y}(s),\Theta (s),\bar \Theta (s),s)\). We present the following matrix minimum principle for Problem (O). The interested reader is referred to Athans (1968) for a proof.
Lemma 3.2
Let (H1)–(H2) hold. Suppose that \((\Theta ^{\ast }(\cdot),\bar \Theta ^{\ast }(\cdot))\) is an optimal control of Problem (O) for the initial pair (t,ξ) and let (X ∗(·),Y ∗(·)) be the corresponding optimal state process. Then there exist matrix-valued functions P(·) and Λ(·) satisfying the following ODEs (the variable s∈[t,T] is suppressed)
with constraints
Now, we are ready to state and prove the principal result of this section.
Theorem 3.3
Let (H1)–(H2) hold and t∈(0,T). If Problem (MF-LQ) admits an optimal closed-loop strategy on [t,T], then the GRE (2.6) is regularly solvable on [t,T].
Proof.
Suppose that 
is an optimal closed-loop strategy of Problem (MF-LQ) on [t,T]. Then, by Proposition 3.1, \((\Theta ^{\ast }(\cdot),\bar \Theta ^{\ast }(\cdot),0)\) is an optimal closed-loop strategy of Problem (MF-LQ) 0 on [t,T], and it follows from Definition 2.3 (i) that \((\Theta ^{\ast }(\cdot),\bar \Theta ^{\ast }(\cdot))\) is an optimal control of Problem (O) for any \(\xi \in L^{2}_{{\mathcal {F}}_{t}}(\Omega ;\mathbb {R}^{n})\). Thus, by the matrix minimum principle, Lemma 3.2, there exist functions \(P(\cdot),\Lambda (\cdot):[t,T]\to \mathbb {R}^{n}\) such that (3.6)–(3.7) hold. By a straightforward calculation, we see from the first equation in (3.6) that P(·) satisfies
and from the second equation in (3.6), we see that Λ(·) satisfies
Note that P(·)⊤ and Λ(·)⊤ also solve (3.8) and (3.9), respectively. Hence, by uniqueness, we have P(·)=P(·)⊤ and Λ(·)=Λ(·)⊤. Let
Then, \(\Pi (T)=G+\bar G\) and
Also, from the first equality in (3.7), we have (noting that X ∗ and Y ∗ are symmetric)
Likewise, from the second equality in (3.7), we have
Let Φ(·) be the solution to the \(\mathbb {R}^{n\times n}\)-valued ODE
where
Then
Denoting \(\bar \Sigma \equiv R+\bar R+(D+\bar D)^{\top } P(D+\bar D)\), since (3.12) holds for all \(\xi \in L_{{\mathcal {F}}_{t}}^{2}(\Omega ;\mathbb {R}^{n})\) and Φ(s) is invertible for all s∈[t,T], we must have
Now take \(\eta \in L_{{\mathcal {F}}_{t}}^{2}(\Omega ;\mathbb {R})\) with \(\mathbb {E}\eta =0\) and \(\mathbb {E}\eta ^{2}=1\). Then for any \(x\in \mathbb {R}^{n}\), the solution Y ∗ of (3.4) corresponding to \((\Theta,\bar \Theta)=(\Theta ^{\ast },\bar \Theta ^{\ast })\) and ξ=η x is identically zero, and the corresponding solution X ∗ of (3.3) satisfies
Let Ψ(·) be the solution to the following SDE for \(\mathbb {R}^{n\times n}\)-valued process:
Then we have
Hence, denoting Σ≡R+D ⊤ PD, we obtain from (3.11) that
which implies Σ Θ ∗+B ⊤ P+D ⊤ PC+S=0. It follows that \({\mathcal {R}}(B^{\top } P+D^{\top } PC+S)\subseteq {\mathcal {R}}(\Sigma)\). Moreover, since Σ † Σ is an orthogonal projection, we have
and
for some \(\theta (\cdot)\in L^{2}(t,T;\mathbb {R}^{m\times n})\). Similarly, from (3.13) we have
and
for some \(\tau (\cdot)\in L^{2}(t,T;\mathbb {R}^{m\times n})\). Substituting (3.14) and (4.8) back into (3.8) and (3.10), respectively, we see that (P(·),Π(·)) satisfies the GRE (2.6). In order to show that (P(·),Π(·)) is regular, it remains to prove that
For this we take any \(u(\cdot)\in {\mathcal {U}}[t,T]\) and let X(·) be the solution to
Applying Itô’s formula to \(s\mapsto \langle P(s)(X(s)-\mathbb {E}[X(s)]),X(s)-\mathbb {E}[X(s)]\rangle \) and \(s\mapsto \langle \Pi (s)\mathbb {E}[X(s)],\mathbb {E}[X(s)]\rangle \), we have
Since \((\Theta ^{\ast }(\cdot),\bar \Theta ^{\ast }(\cdot),0)\) is an optimal closed-loop strategy of Problem (MF-LQ)0 on [t,T], we have
Note that for any \(u(\cdot)\in {\mathcal {U}}[t,T]\) of the form
the corresponding solution X(·) of (3.16) satisfies \(\mathbb {E}[X(\cdot)]=0\) and hence \(\mathbb {E}[u(\cdot)]=0\). Then (3.17) yields
which implies that \(\Sigma \geqslant 0\). Likewise, for any \(u(\cdot)\in {\mathcal {U}}[t,T]\) of the form
the corresponding solution X(·) of (3.16) satisfies
Then (3.17) yields
which implies that \(\bar \Sigma \geqslant 0\). The proof is completed. □
Characterization of Closed-Loop Solvability
The aim of this section is to provide a characterization of the closed-loop solvability of Problem (MF-LQ) in terms of the GRE (2.6), a linear backward stochastic differential equation (BSDE, for short), and a linear terminal value problem of ODE. In the case of Problem (MF-LQ)0, it turns out that the regular solvability of the GRE (2.6) is not only necessary but also sufficient for the existence of an optimal closed-loop strategy.
Theorem 4.1
Let (H1)–(H2) hold and t∈(0,T). Then Problem (MF-LQ) is closed-loop solvable on [t,T] if and only if the following hold:
(i) The GRE (2.6) admits a regular solution \((P(\cdot),\Pi (\cdot))\in C([t,T];\mathbb {S}^{n})\times C([t,T];\mathbb {S}^{n})\).
(ii) The adapted solution (η(·),ζ(·)) to the BSDE
satisfies
and the solution \(\bar \eta (\cdot)\) to the ODE
satisfies
where
In the above case, the optimal closed-loop strategy \((\Theta ^{\ast }(\cdot),\bar \Theta ^{\ast }(\cdot),u^{\ast }(\cdot))\) admits the following representation:
where \(\theta (\cdot),\tau (\cdot)\in L^{2}(t,T;\mathbb {R}^{m\times n})\), \(\nu (\cdot)\in L_{\mathbb {F}}^{2}(t,T;\mathbb {R}^{m})\), and \(\bar \nu (\cdot)\in L^{2}(t,T;\mathbb {R}^{m})\). Moreover, the value V(t,ξ) is given by
Proof.
Necessity. Suppose that 
is an optimal closed-loop strategy of Problem (MF-LQ) on [t,T]. Then it follows from Theorem 3.3 that the GRE (2.6) admits a regular solution (P(·),Π(·)). To determine u
∗(·), let (X
∗(·),Y
∗(·),Z
∗(·)) be the adapted solution of (3.1). Proceeding as in the proof of Proposition 3.1, we see that (X
∗(·),Y
∗(·),Z
∗(·)) satisfies (3.2). Now, let
and define
By the terminal value of Y ∗(T) (see (3.1)) and P(T)=G, we obtain
Also, making use of \(\Pi (T)=G+\bar G\), we obtain
Further, it is obvious that
and
We now want to show that
To this end, we recall from the proof of Theorem 3.3 that
Then we have
Also, we have
Moreover, we have from (3.2):
Now (4.9), (4.12), (4.16) and (4.17) yield
Hence,
Since \(\bar \Sigma ^{\dag }\left [(B+\bar B)^{\top }\widetilde \eta +(D+\bar D)^{\top }(\mathbb {E}[\beta ]+P\mathbb {E}[\sigma ])+\mathbb {E}[\rho ]+\bar \rho \right ] =-\bar \Sigma ^{\dag }\bar \Sigma \mathbb {E}[u^{\ast }]\) and \(\bar \Sigma ^{\dag }\bar \Sigma \) is an orthogonal projection, we have
for some \(\bar \nu (\cdot)\in L^{2}(t,T;\mathbb {R}^{m})\). Consequently,
Likewise, (4.9), (4.13), (4.15) and (4.18) yield
which implies that
and that
for some \(\nu (\cdot)\in L_{\mathbb {F}}^{2}(t,T;\mathbb {R}^{m})\). Consequently,
Since the adapted solution (η(·),ζ(·)) of (4.1) satisfies
by the uniqueness of solutions, we have proved (4.14). Then, (4.19) and (4.20) yield
This proves the necessity, as well as (4.6).
Sufficiency. The proof is much like that of (Theorem 5.2 Sun2016). Let \((\Theta ^{\ast }(\cdot),\bar \Theta ^{\ast }(\cdot),u^{\ast }(\cdot))\) be defined by (4.6). Then we have
For any \(\xi \in L^{2}_{{\mathcal {F}}_{t}}(\Omega ;\mathbb {R}^{n})\) and \(u(\cdot)\in {\mathcal {U}}[t,T]\), let X(·)≡X(· ;t,ξ,u(·)) be the corresponding solution of (1.1). Proceeding similarly to the proof of (Theorem 5.2 Sun2016) and using (4.21), we obtain
Since \(\Sigma,\bar \Sigma \geqslant 0\), (4.22) implies that
Therefore, \((\Theta ^{\ast }(\cdot),\bar \Theta ^{\ast }(\cdot),u^{\ast }(\cdot))\) is an optimal closed-loop strategy of Problem (MF-LQ) on [t,T] and (4.7) holds. The proof is completed. □
In the special case that b(·), σ(·), g, \(\bar g\), q(·), \(\bar q(\cdot)\), ρ(·), and \(\bar \rho (\cdot)\) vanish, if the GRE (2.6) admits a regular solution (P(·),Π(·)), then condition (ii) of Theorem 4.1 holds automatically. Indeed, one can easily check that (η(·),ζ(·))=(0,0) and \(\bar \eta (\cdot)=0\). Thus, we have the following corollary.
Corollary 4.2
Let (H1)–(H2) hold and t∈(0,T). Then Problem (MF-LQ) 0 is closed-loop solvable on [t,T] if and only if the GRE (2.6) is regularly solvable.
Conclusion
This is an important yet challenging research topic. Recently there has been increasing interest in studying this type of stochastic control problems as well as their applications. Beside this work, the optimal stochastic control problems under MF-SDEs are underdeveloped in the literature, and therefore many fundamental questions remain open and methodologies need to be significantly improved. To establish new theory and hopefully to shed light on financial investment, we expect the findings of this research program to add to various streams of the literature, such as portfolio selection, optimal control techniques, financial risk management, and relative performance evaluation.
References
Ait Rami, M, Moore, JB, Zhou, XY: Indefinite stochastic linear quadratic control and generalized differential Riccati equation. SIAM J. Control Optim. 40, 1296–1311 (2001).
Andersson, D, Djehiche, B: A maximum principle for SDEs of mean-field type. Appl. Math. Optim. 63, 341–356 (2011).
Athans, M: The matrix minimum principle. Inform. Control. 11, 592–606 (1968).
Buckdahn, R, Djehiche, B, Li, J: A general stochastic maximum principle for SDEs of mean-field type. Appl. Math. Optim. 64, 197–216 (2011).
Buckdahn, R, Djehiche, B, Li, J, Peng, S: Mean-field backward stochastic differential equations: a limit approach. Ann. Probab. 37, 1524–1565 (2009).
Buckdahn, R, Li, J, Peng, S: Mean-field backward stochastic differential equations and related partial differential equations. Stoch. Proc. Appl. 119, 3133–3154 (2009).
Chen, S, Li, X, Zhou, XY: Stochastic linear quadratic regulators with indefinite control weight costs. SIAM J. Control Optim. 36, 1685–1702 (1998).
Chen, S, Yong, J: Stochastic linear quadratic optimal control problems with random coefficients. Chin. Ann. Math. 21B, 323–338 (2000).
Cui, XY, Li, X, Li, D: Unified framework of mean-field formulations for optimal multi-period mean-variance portfolio selection. IEEE Trans. Auto. Control. 59, 1833–1844 (2014).
Elliott, R, Li, X, Ni, YH: Discrete time mean-field stochastic linear-quadratic optimal control problems. Automatica. 49, 3222–3233 (2013).
Huang, J, Li, X, Wang, TX: Mean-field linear-quadratic-Gaussian (LQG) games for stochastic integral systems. IEEE Trans. Auto. Control (2015). doi:10.1109/TAC.2015.2506620.
Huang, J, Li, X, Yong, J: A linear-quadratic optimal control problem for mean-field stochastic differential equations in infinite horizon. Math. Control Related Fields. 5, 97–139 (2015).
Kac, M: Foundations of kinetic theory. Proc. Third Berkeley Symp. Math. Stat. Probab. 3, 171–197 (1956).
McKean, HP: A class of Markov processes associated with nonlinear parabolic equations. Proc. Natl. Acad. Sci. USA. 56, 1907–1911 (1966).
Meyer-Brandis, T, Øksendal, B, Zhou, XY: A mean-field stochastic maximum principle via Malliavin calculus. Stochastics. 84(5–6), 643–666 (2012). doi:10.1080/17442508.2011.651619.
Penrose, R: A generalized inverse of matrices. Proc. Cambridge Philos Soc. 52, 17–19 (1955).
Sun, J: Mean-field stochastic linear quadratic optimal control problems: open-loop solvabilities. ESAIM: COCV, 016023 (2016). doi:10.1051/cocv/2.
Sun, J, Li, X, Yong, J: Open-loop and closed-loop solvabilities for stochastic linear quadratic optimal control problems (2015). http://arxiv.org/pdf/1508.02163v1.pdf.
Sun, J, Yong, J: Linear quadratic stochastic differential games: open-loop and closed-loop saddle points. SIAM J. Control Optim. 52, 4082–4121 (2014).
Sun, J, Yong, J, Zhang, S: Linear quadratic stochastic two-person zero-sum differential games in an infinite horizon. ESAIM COCV. 22, 743–769 (2016). doi:10.1051/cocv/2015024.
Tang, S: General linear quadratic optimal stochastic control problems with random coefficients: linear stochastic Hamilton systems and backward stochastic Riccati equations. SIAM J. Control Optim. 42, 53–75 (2003).
Wonham, WM: On a matrix Riccati equation of stochastic control. SIAM J. Control Optim. 6, 681–697 (1968).
Yong, J: Linear-quadratic optimal control problems for mean-field stochastic differential equations. SIAM J. Control Optim. 51, 2809–2838 (2013).
Yong, J, Zhou, XY: Stochastic controls: Hamiltonian systems and HJB equations. Springer-Verlag, New York (1999).
Authors’ contributions
All authors read and approved the final manuscript.
Authors’ information
Xun Li was partially supported by Hong Kong RGC under grants 519913, 15209614 and 15224215. Jingrui Sun was partially supported by the National Natural Science Foundation of China (11401556) and the Fundamental Research Funds for the Central Universities (WK 2040000012). Jiongmin Yong was partially supported by NSF DMS-1406776.
Competing interests
The authors declare that they have no competing interests.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.
About this article
Cite this article
Li, X., Sun, J. & Yong, J. Mean-field stochastic linear quadratic optimal control problems: closed-loop solvability. Probab Uncertain Quant Risk 1, 2 (2016). https://doi.org/10.1186/s41546-016-0002-3
Received:
Accepted:
Published:
DOI: https://doi.org/10.1186/s41546-016-0002-3
Keywords
- Mean-field stochastic differential equation
- Linear quadratic optimal control
- Riccati equation
- Regular solution
- Closed-loop solvability
AMS subject classifications
- 49N10
- 49N35
- 93E20