Mean-field stochastic linear quadratic optimal control problems: closed-loop solvability

An optimal control problem is studied for a linear mean-field stochastic differential equation with a quadratic cost functional. The coefficients and the weighting matrices in the cost functional are all assumed to be deterministic. Closed-loop strategies are introduced, which require to be independent of initial states; and such a nature makes it very useful and convenient in applications. In this paper, the existence of an optimal closed-loop strategy for the system (also called the closed-loop solvability of the problem) is characterized by the existence of a regular solution to the coupled two (generalized) Riccati equations, together with some constraints on the adapted solution to a linear backward stochastic differential equation and a linear terminal value problem of an ordinary differential equation.

Let \((\Omega,{\mathcal {F}},\mathbb {F},\mathbb {P})\) be a complete filtered probability space on which a standard one-dimensional Brownian motion W={W(t);0≤t<∞} is defined, where \(\mathbb {F}=\{{\mathcal {F}}_{t}\}_{t\geqslant 0}\) is the natural filtration of W augmented by all the \(\mathbb {P}\)-null sets in \({\mathcal {F}}\). Consider the following controlled linear mean-field stochastic differential equation (MF-SDE, for short) on a finite time horizon [t,T]:

where \(A(\cdot), \bar {A}(\cdot), B(\cdot), \bar {B}(\cdot), C(\cdot), \bar {C}(\cdot), D(\cdot), \bar {D}(\cdot)\) are given deterministic matrix-valued functions; b(·),σ(·) are vector-valued \(\mathbb {F}\)-progressively measurable processes and ξ is an \({\mathcal {F}}_{t}\)-measurable random vector. In the above, u(·) is the control process and X(·) is the corresponding state process with initial pair (t,ξ). For any t∈[0,T), we define

Any \(u(\cdot)\in {\mathcal {U}}[t,T]\) is called an admissible control (on [t,T]). Under some mild conditions, for any initial pair (t,ξ) with \(\xi \in L^{2}_{{\mathcal {F}}_{t}}(\Omega ;\mathbb {R}^{n})\) (the set of all \({\mathcal {F}}_{t}\)-measurable, square-integrable \(\mathbb {R}^{n}\)-valued processes), and any admissible control \(u(\cdot)\in {\mathcal {U}}[t,T]\), (1.1) admits a unique square-integrable solution X(·)≡X(· ;t,ξ,u(·)). Now we introduce the following cost functional:

where G, \(\bar {G}\) are symmetric matrices and \(Q(\cdot), \bar {Q}(\cdot), S(\cdot), \bar {S}(\cdot), R(\cdot), \bar {R}(\cdot)\) are deterministic matrix-valued functions with Q(·), \(\bar {Q}(\cdot)\), R(·), and \(\bar {R}(\cdot)\) being symmetric; g is an \({\mathcal {F}}_{t}\)-measurable random vector and \(\bar {G}\) is a deterministic vector; q(·), ρ(·) are vector-valued \(\mathbb {F}\)-progressively measurable processes and \(\bar {Q}(\cdot)\), \(\bar {\rho }(\cdot)\) are vector-valued deterministic functions. Our mean-field stochastic linear quadratic (LQ, for short) optimal control problem can be stated as follows:

Problem (MF-LQ). For any given initial pair \((t,\xi)\in [0,T)\times L^{2}_{{\mathcal {F}}_{t}}(\Omega ;\mathbb {R}^{n})\), find a \(u^{*}(\cdot)\in {\mathcal {U}}[t,T]\) such that

Any \(u^{*}(\cdot)\in {\mathcal {U}}[t,T]\) satisfying (1.3) is called an optimal open-loop control of Problem (MF-LQ) for the initial pair (t,ξ), and the corresponding X ^∗(·)≡X(· ;t,ξ,u ^∗(·)) is called an optimal open-loop state process. The function V(·,·) is called the value function of Problem (MF-LQ). In the special case where b(·), σ(·), g, \(\bar g\), q(·), \(\bar q(\cdot)\), ρ(·), and \(\bar \rho (\cdot)\) vanish, we denote the corresponding mean-field LQ problem, cost functional, and value function by Problem (MF-LQ)⁰, J ⁰(t,ξ;u(·)), and V ⁰(t,ξ), respectively.

The theory of MF-SDEs can be traced back to Kac who presented a stochastic toy model for the Vlasov kinetic equation of plasma in (Kac 1956) which leads to the so-called McKean-Vlasov stochastic differential equation. Since then, the research on the related topics and their applications has become a notable and serious endeavor among researchers in applied probability and optimal stochastic controls, including financial engineering. See, for examples, Andersson and Djehiche (2011), Buckdahn et al. (2009), Buckdahn et al. (2009), Buckdahn et al. (2011), Cui et al. (2014), Elliott et al. (2013), Huang et al. (2015), Huang et al. (2015), McKean (1966), Meyer-Brandis et al. (2012), Yong (2013). Note that when the mean-field part is absent, Problem (MF-LQ) is reduced to the classical stochastic LQ optimal control problem. For relevant results and historic remarks on this subject, the reader is further referred to, for examples, (Ait Rami et al. 2001; Chen and Yong 2000; Chen et al. 1998; Tang 2003; Wonham 1968) and the book of Yong and Zhou (1999).

Recently, Sun and Yong introduced the notions of open-loop and closed-loop solvabilities for stochastic LQ problems (Sun and Yong 2014). See also (Sun et al. 2016; Sun et al. 2015). It turns out that these two notions are essentially different for stochastic LQ problem on finite time horizon. Roughly speaking, the open-loop solvability is equivalent to the solvability of a forward-backward stochastic differential equation (FBSDE, for short), and the closed-loop solvability is equivalent to the existence of a regular solution to a Riccati equation. Open-loop solvability was studied for Problem (MF-LQ) in (Sun 2016). The current work is therefore a continuation of the above-mentioned works.

The rest of the paper is organized as follows. In Section “Preliminaries” we give some preliminaries, carefully explain the closed-loop strategies, and introduce the regular solution to the generalized Riccati equation. Section “Necessary Conditions for Closed-Loop Solvability” is devoted to the necessary conditions for the existence of an optimal closed-loop strategy. In Section “Characterization of Closed-Loop Solvability”, we present our main result, in which the closed-loop solvability of the mean-field LQ problems is characterized. Finally, some concluding remarks are given in Section “Conclusion”.

We begin with some notations that will be used throughout the paper:

• \(\mathbb {R}^{n\times m}\): the Euclidean space of all n×m real matrices.

• \(\mathbb {R}^{n}=\mathbb {R}^{n\times 1}\) and \(\mathbb {R}=\mathbb {R}^{1}\).

• \(\mathbb {S}^{n}\): the space of all symmetric n×n real matrices.

• M ^⊤: the transpose of a matrix M.

• M ^†: the Moore–Penrose pseudoinverse of a matrix M (Penrose 1955).

• tr(M): the sum of diagonal elements of a square matrix M.

• 〈·,·〉: the inner product on a Euclidean space given by 〈M,N〉↦tr(M ^⊤ N).

• \(|M|\triangleq \sqrt {\text {tr}(MM^{\top })}\): the Frobenius norm of a matrix M.

• \({\mathcal {R}}(M)\): the range of a matrix M.

For \(M,N\in \mathbb {S}^{n}\), we use the notation \(M\geqslant N\) (respectively, M>N) to indicate that M−N is positive semi-definite (respectively, positive definite). For any \(\mathbb {S}^{n}\)-valued measurable function F on [t,T], we write

• \(F\geqslant 0 \quad \Longleftrightarrow \quad F(s)\geqslant 0, \quad \text {a.e.}~s\in [t,T]\),

• F>0 ⇔ F(s)>0, a.e. s∈[t,T],

• \(F\gg 0 \quad \Longleftrightarrow \quad F(s)\geqslant \delta I, \quad \text {a.e.}~s\in [t,T],\text {for some}~\delta >0\).

Let f(·) be a function from \(\mathbb {R}^{n\times m}\) into \(\mathbb {R}\). Recall that the gradient of f at X=(x _ij ), denoted by \(\frac {{\partial } f(X)}{{\partial } X}\), is an n×m matrix whose (i,j)-th entry is given by \(\left [{{\partial } f(X)\over {\partial } X}\right ]_{ij}={{\partial } f(X)\over {\partial } x_{ij}}\). For matrices L, M, and N of proper dimensions, the following formulae hold:

For a Euclidean space \(\mathbb {H}\), let \(L^{p}(t,T;\mathbb {H}) (1\leqslant p\leqslant \infty)\) be the space of all \(\mathbb {H}\)-valued functions that are L ^p-integrable on [t,T], and let \(C([t,T];\mathbb {H})\) be the space of all \(\mathbb {H}\)-valued continuous functions on [t,T]. We denote

The following assumptions will be in force throughout this paper.

(H1) The coefficients of the state equation satisfy the following:

(H2) The weighting coefficients in the cost functional satisfy the following:

A standard argument using the contraction mapping theorem shows that under (H1), for any initial pair \((t,\xi)\in [0,T)\times L^{2}_{{\mathcal {F}}_{t}}(\Omega ;\mathbb {R}^{n})\) and any admissible control \(u(\cdot)\in {\mathcal {U}}[t,T]\), (1.1) admits a unique (strong) solution \(X(\cdot)\equiv X(\cdot \,; t,\xi,u(\cdot))\in L_{\mathbb {F}}^{2}(\Omega ;C([t,T];\mathbb {R}^{n}))\). Hence, under (H1)–(H2), the cost functional (1.2) is well-defined, and Problem (MF-LQ) makes sense.

Let us now recall the notion of open-loop solvability from Sun (2016), which was inspired by (Sun and Yong 2014).

Definition 2.1

Problem (MF-LQ) is said to be (uniquely) open-loop solvable at initial pair \((t,\xi)\in [0,T]\times L^{2}_{{\mathcal {F}}_{t}}(\Omega ;\mathbb {R}^{n})\) if there exists a (unique) \(u^{*}(\cdot)\in {\mathcal {U}}[t,T]\) satisfying (1.3). Problem (MF-LQ) is said to be (uniquely) open-loop solvable at t if for any \(\xi \in L^{2}_{{\mathcal {F}}_{t}}(\Omega ;\mathbb {R}^{n})\), there exists a (unique) \(u^{*}(\cdot)\in {\mathcal {U}}[t,T]\) satisfying (1.3), and Problem (MF-LQ) is said to be (uniquely) open-loop solvable on [t,T) if it is (uniquely) open-loop solvable at all s∈[t,T).

Next, we introduce the following definition.

Definition 2.2

(i) Let

Any triple is called a closed-loop strategy of Problem (MF-LQ) on [t,T].

(ii) For any and \(\xi \in L^{2}_{{\mathcal {F}}_{t}}(\Omega ;\mathbb {R}^{n})\), let \(X(\cdot)\equiv X(\cdot \,;t,\xi,\Theta (\cdot),\bar \Theta (\cdot),v(\cdot))\) be the solution to the following closed-loop system:

and let

Then (X(·),u(·)) is called the outcome pair of \((\Theta (\cdot),\bar {\Theta }(\cdot),v(\cdot))\)on [t,T] corresponding to the initial state ξ; X(·) and u(·) are called the corresponding closed-loop state process and closed-loop outcome control process, respectively.

Note that if (X(·),u(·)) is the outcome pair of a closed-loop strategy corresponding to some \(\xi \in L^{2}_{{\mathcal {F}}_{t}}(\Omega ;\mathbb {R}^{n})\), then \((X(\cdot),u(\cdot))\in L^{2}_{\mathbb {F}}(\Omega ;C([t,T];\mathbb {R}^{n})\times {\mathcal {U}}[t,T]\) is actually a state-control pair of the state Eq. (1.1). In fact, (2.1) is equivalent to the following:

Therefore, for any , the meaning of \(J(t,\xi ;\Theta (\cdot)X(\cdot)+\bar {\Theta }(\cdot)\mathbb {E}[X(\cdot)]+v(\cdot))\) is clear. We point out that a closed-loop strategy is not related to any initial state \(\xi \in L^{2}_{{\mathcal {F}}_{t}}(\Omega ;\mathbb {R}^{n})\), whereas, an outcome pair (X(·),u(·)) of a closed-loop strategy \((\Theta (\cdot),\bar {\Theta }(\cdot),v(\cdot))\) depends not only on the closed-loop strategy, but also on the initial state ξ. Hence, we should carefully distinguish the closed-loop strategy and the corresponding outcome control. Now, we are ready to introduce the following notion.

Definition 2.3

(i) A closed-loop strategy is said to be optimal on [t,T] if

where X ^∗(·) and X(·) are the closed-loop state processes corresponding to \((\Theta ^{*}(\cdot),\bar {\Theta }^{*}(\cdot),v^{*}(\cdot),\xi)\) and (Θ(·), \(\bar {\Theta }(\cdot),v(\cdot),\xi)\), respectively. If an optimal closed-loop strategy (uniquely) exists on [t,T], Problem (MF-LQ) is said to be (uniquely) closed-loop solvable on [t,T].

(ii) A closed-loop strategy is said to be weakly optimal on [t,T] if (??) holds only for \(\xi =x\in \mathbb {R}^{n}\). If a weakly optimal closed-loop strategy (uniquely) exists on [t,T], Problem (MF-LQ) is said to be (uniquely) weakly closed-loop solvable on [t,T].

Similar to (Sun and Yong 2014), we have the following proposition.

Proposition 2.4

Let (H1)–(H2) hold and let . Then the following statements are equivalent:

(i) \((\Theta ^{*}(\cdot),\bar {\Theta }^{*}(\cdot),v^{*}(\cdot))\) is an optimal closed-loop strategy of Problem (MF-LQ) on [t,T];

(ii) The following holds:

where X ^∗(·) and X(·) are the closed-loop state processes corresponding to \((\Theta ^{*}(\cdot),\bar {\Theta }^{*}(\cdot),v^{*}(\cdot),\xi)\) and (Θ ^∗(·), \(\bar {\Theta }^{*}(\cdot),v(\cdot),\xi)\), respectively;

(iii) The following holds:

where X ^∗(·)is the closed-loop state process corresponding to \((\Theta ^{*}(\cdot),\bar \Theta ^{*}(\cdot),v^{*}(\cdot))\) and \(\xi \in L^{2}_{{\mathcal {F}}_{t}}(\Omega ;\mathbb {R}^{n})\).

Proof.

(i) \(\mathop {\Rightarrow }\) (ii) is trivial, by taking Θ(·)=Θ ^∗(·) and \(\bar {\Theta }(\cdot)=\bar {\Theta }^{*}(\cdot)\) in (??).

(ii) \(\mathop {\Rightarrow }\) (iii). For any \(\xi \in L^{2}_{{\mathcal {F}}_{t}}(\Omega ;\mathbb {R}^{n})\) and \(u(\cdot)\in {\mathcal {U}}[t,T]\), let X(·) be the solution to the following:

and set

Then X(·) is also the solution to the following MF-SDE:

Therefore,

(iii) \(\mathop {\Rightarrow }\) (i). For any \(\xi \in L^{2}_{{\mathcal {F}}_{t}}(\Omega ;\mathbb {R}^{n})\) and , let X(·) be the solution to the following MF-SDE:

Set

Then, by uniqueness, X(·) also solves MF-SDE (2.4). Thus,

This completes the proof. □

With the same proof, we have the following result for the weakly optimal closed-loop strategies.

Proposition 2.5

Let (H1)–(H2) hold and let . Then the following statements are equivalent:

(i) \((\Theta ^{*}(\cdot),\bar {\Theta }^{*}(\cdot),v^{*}(\cdot))\) is a weakly optimal closed-loop strategy of Problem (MF-LQ) on [t,T];

(ii) The following holds:

where X ^∗(·) and X(·) are the closed-loop state processes corresponding to \((\Theta ^{*}(\cdot),\bar {\Theta }^{*}(\cdot),v^{*}(\cdot),x)\) and (Θ ^∗(·), \(\bar {\Theta }^{*}(\cdot),v(\cdot),x)\), respectively;

(iii) The following holds:

where X ^∗(·) is the closed-loop state process corresponding to \((\Theta ^{*}(\cdot),\bar {\Theta }^{*}(\cdot),v^{*}(\cdot))\) and \(x\in \mathbb {R}^{n}\).

Remark 2.6

(i) An optimal open-loop control is allowed to depend on the initial state, whereas an optimal closed-loop strategy is required to be independent of the initial state.

(ii) It is clear from Proposition 2.4 (iii) that the outcome control \(u^{*}(\cdot)\equiv \Theta ^{*}(\cdot)X^{*}(\cdot)+\bar {\Theta }^{*}(\cdot)\mathbb {E}[X^{*}(\cdot)]+v^{*}(\cdot)\) of an optimal closed-loop strategy \((\Theta ^{*}(\cdot),\bar {\Theta }^{*}(\cdot),v^{*}(\cdot))\) is an optimal open-loop control of Problem (MF-LQ) for the initial pair (t,X ^∗(t)). Hence, closed-loop solvability implies open-loop solvability.

(iii) Obviously, an optimal closed-loop strategy on [t,T]is also weakly optimal. For the classical LQ optimal control problems where \(\mathbb {E}[X(\cdot)]\) and \(\mathbb {E}[u(\cdot)]\) are absent, it can be shown, using the results from Sun and Yong ( 2014 ) and a completion of squares technique, that the two concepts coincide. But for Problem (MF-LQ), the existence of a weakly optimal closed-loop strategy does not guarantee the existence of an optimal closed-loop strategy. To see this, we present the following example.

Example 2.7

Consider the following one-dimensional state equation

and cost functional

For any \(x\in \mathbb {R}\) and \(u(\cdot)\in {\mathcal {U}}[t,1]\), we have

On the other hand, it is clear that \((\Theta ^{*}(\cdot),\bar {\Theta }^{*}(\cdot),v^{*}(\cdot))\equiv (0,0,0)\) satisfies

Thus, by Proposition 2.5, (0,0,0)is a weakly optimal closed-loop strategy of the problem on [t,1].

Let us now show that the above problem does not admit an optimal closed-loop strategy on any [t,1]with 0<t<1. Assume the contrary; i.e., let satisfy (2.3) on [t,1]for some 0<t<1. For any \(x\in \mathbb {R}\), take ξ=W(t)x and \(u(s)\equiv {W(t)x\over t-1}\). The corresponding solution of (2.5) is

Note that X(1)=0. Thus, (2.3) implies that

where X ^∗(·)is the solution to the following closed-loop system:

It follows that

Note that \(\mathbb {E}[X^{*}(s)]\equiv 0\). Then,

But this is impossible since it has to be true for all \(x\in \mathbb {R}\).

We conclude this section by introducing the coupled generalized Riccati equation (GRE, for short), whose regular solvability will turn out to be necessary and sufficient for the closed-loop solvability of Problem (MF-LQ) in the next two sections.

The GRE associated with Problem (MF-LQ) consists of two coupled nonlinear differential equations of the following form (for simplicity of notation, we will usually suppress the time variable s below):

Definition 2.8

A solution \((P(\cdot),\Pi (\cdot))\in C([t,T];\mathbb {S}^{n})\times C([t,T];\mathbb {S}^{n})\) of (2.6) is said to be regular if

and

The GRE (2.6) is said to be regularly solvable on [t,T] if it admits a regular solution.

In this section we will deduce necessary conditions for the closed-loop solvability of Problem (MF-LQ). In particular, we shall establish the necessity of the regular solvability of GRE (2.6) by a matrix minimum principle.

Let \(\Theta ^{*}(\cdot),\bar \Theta ^{*}(\cdot)\in L^{2}(t,T;\mathbb {R}^{m\times n})\) and consider the following state equation

and cost functional

where

By Proposition 2.4 (ii), is an optimal closed-loop strategy of Problem (MF-LQ) on [t,T] if and only if for any \(\xi \in L^{2}_{{\mathcal {F}}_{t}}(\Omega ;\mathbb {R}^{n})\), u ^∗(·) is an optimal open-loop control of the problem with the above state equation and cost functional. This leads to the following result.

Proposition 3.1

Let (H1)–(H2) hold. If is an optimal closed-loop strategy of Problem (MF-LQ) on [t,T], then \((\Theta ^{*}(\cdot),\bar \Theta ^{*}(\cdot),0)\) is an optimal closed-loop strategy of Problem (MF-LQ) ⁰ on [t,T].

Proof.

By the preceding discussion and (Theorem 2.3, Sun 2016), we see that \((\Theta ^{*}(\cdot),\bar \Theta ^{*}(\cdot),u^{*}(\cdot))\) is an optimal closed-loop strategy of Problem (MF-LQ) on [t,T] if and only if for any \(\xi \in L^{2}_{{\mathcal {F}}_{t}}(\Omega ;\mathbb {R}^{n})\), the adapted solution (X ^∗(·),Y ^∗(·),Z ^∗(·)) to the following mean-field forward-backward stochastic differential equation (MF-FBSDE, for short):

satisfies

and the following condition holds:

where X(·) is the solution of

Since the MF-FBSDE (3.1) admits a solution for each \(\xi \in L^{2}_{{\mathcal {F}}_{t}}(\Omega ;\mathbb {R}^{n})\) and \((\Theta ^{\ast }(\cdot),\bar \Theta ^{\ast }(\cdot),u^{\ast }(\cdot))\) is independent of ξ, by subtracting solutions corresponding ξ and 0, the latter from the former, we see that for any \(\xi \in L^{2}_{{\mathcal {F}}_{t}}(\Omega ;\mathbb {R}^{n})\), the following MF-FBSDE:

also admits an adapted solution (X(·),Y(·),Z(·)) satisfying

It follows, again from (Theorem 2.3, Sun2016), that \((\Theta ^{\ast }(\cdot),\bar \Theta ^{\ast }(\cdot),0)\) is an optimal closed-loop strategy of Problem (MF-LQ)⁰ on [t,T]. □

Now let us look at Problem (MF-LQ)⁰. If we consider only closed-loop strategies of the form \((\Theta (\cdot),\bar \Theta (\cdot),0)\), then the state equation becomes

and \(\mathbb {E}[X(\cdot)]\) satisfies

By Itô’s formula, the matrices \(\mathbf {X}(s)\triangleq \mathbb {E}\left [X(s)X(s)^{\top }\right ]\) and \(\mathbf {Y}(s)\triangleq \mathbb {E}[X(s)]\mathbb {E}[X(s)]^{\top }\) satisfy the matrix-valued ordinary differential equations (ODEs, for short)

and

respectively. The cost functional \(J^{0}(t,\xi ;\Theta (\cdot)X(\cdot)+\bar \Theta (\cdot)\mathbb {E}[X(\cdot)])\) can be expressed equivalently as

where

Then we may pose the following deterministic optimal control problem.

Problem (O). For any given \((t,\xi)\in [0,T)\times L^{2}_{{\mathcal {F}}_{t}}(\Omega ;\mathbb {R}^{n})\), find \(\Theta ^{\ast }(\cdot),\bar \Theta ^{\ast }(\cdot)\in L^{2}(t,T;\mathbb {R}^{m\times n})\) such that

Rewrite (3.3)–(3.4) as

and denote the integrand in (3.5) by \(L(\mathbf {X}(s),\mathbf {Y}(s),\Theta (s),\bar \Theta (s),s)\). We present the following matrix minimum principle for Problem (O). The interested reader is referred to Athans (1968) for a proof.

Lemma 3.2

Let (H1)–(H2) hold. Suppose that \((\Theta ^{\ast }(\cdot),\bar \Theta ^{\ast }(\cdot))\) is an optimal control of Problem (O) for the initial pair (t,ξ) and let (X ^∗(·),Y ^∗(·)) be the corresponding optimal state process. Then there exist matrix-valued functions P(·) and Λ(·) satisfying the following ODEs (the variable s∈[t,T] is suppressed)

with constraints

Now, we are ready to state and prove the principal result of this section.

Theorem 3.3

Let (H1)–(H2) hold and t∈(0,T). If Problem (MF-LQ) admits an optimal closed-loop strategy on [t,T], then the GRE (2.6) is regularly solvable on [t,T].

Proof.

Suppose that is an optimal closed-loop strategy of Problem (MF-LQ) on [t,T]. Then, by Proposition 3.1, \((\Theta ^{\ast }(\cdot),\bar \Theta ^{\ast }(\cdot),0)\) is an optimal closed-loop strategy of Problem (MF-LQ) ⁰ on [t,T], and it follows from Definition 2.3 (i) that \((\Theta ^{\ast }(\cdot),\bar \Theta ^{\ast }(\cdot))\) is an optimal control of Problem (O) for any \(\xi \in L^{2}_{{\mathcal {F}}_{t}}(\Omega ;\mathbb {R}^{n})\). Thus, by the matrix minimum principle, Lemma 3.2, there exist functions \(P(\cdot),\Lambda (\cdot):[t,T]\to \mathbb {R}^{n}\) such that (3.6)–(3.7) hold. By a straightforward calculation, we see from the first equation in (3.6) that P(·) satisfies

and from the second equation in (3.6), we see that Λ(·) satisfies

Note that P(·)^⊤ and Λ(·)^⊤ also solve (3.8) and (3.9), respectively. Hence, by uniqueness, we have P(·)=P(·)^⊤ and Λ(·)=Λ(·)^⊤. Let

Then, \(\Pi (T)=G+\bar G\) and

Also, from the first equality in (3.7), we have (noting that X ^∗ and Y ^∗ are symmetric)

Likewise, from the second equality in (3.7), we have

Let Φ(·) be the solution to the \(\mathbb {R}^{n\times n}\)-valued ODE

where

Then

Denoting \(\bar \Sigma \equiv R+\bar R+(D+\bar D)^{\top } P(D+\bar D)\), since (3.12) holds for all \(\xi \in L_{{\mathcal {F}}_{t}}^{2}(\Omega ;\mathbb {R}^{n})\) and Φ(s) is invertible for all s∈[t,T], we must have

Now take \(\eta \in L_{{\mathcal {F}}_{t}}^{2}(\Omega ;\mathbb {R})\) with \(\mathbb {E}\eta =0\) and \(\mathbb {E}\eta ^{2}=1\). Then for any \(x\in \mathbb {R}^{n}\), the solution Y ^∗ of (3.4) corresponding to \((\Theta,\bar \Theta)=(\Theta ^{\ast },\bar \Theta ^{\ast })\) and ξ=η x is identically zero, and the corresponding solution X ^∗ of (3.3) satisfies

Let Ψ(·) be the solution to the following SDE for \(\mathbb {R}^{n\times n}\)-valued process:

Then we have

Hence, denoting Σ≡R+D ^⊤ PD, we obtain from (3.11) that

which implies Σ Θ ^∗+B ^⊤ P+D ^⊤ PC+S=0. It follows that \({\mathcal {R}}(B^{\top } P+D^{\top } PC+S)\subseteq {\mathcal {R}}(\Sigma)\). Moreover, since Σ ^† Σ is an orthogonal projection, we have

and

for some \(\theta (\cdot)\in L^{2}(t,T;\mathbb {R}^{m\times n})\). Similarly, from (3.13) we have

and

for some \(\tau (\cdot)\in L^{2}(t,T;\mathbb {R}^{m\times n})\). Substituting (3.14) and (4.8) back into (3.8) and (3.10), respectively, we see that (P(·),Π(·)) satisfies the GRE (2.6). In order to show that (P(·),Π(·)) is regular, it remains to prove that

For this we take any \(u(\cdot)\in {\mathcal {U}}[t,T]\) and let X(·) be the solution to

Applying Itô’s formula to \(s\mapsto \langle P(s)(X(s)-\mathbb {E}[X(s)]),X(s)-\mathbb {E}[X(s)]\rangle \) and \(s\mapsto \langle \Pi (s)\mathbb {E}[X(s)],\mathbb {E}[X(s)]\rangle \), we have

Since \((\Theta ^{\ast }(\cdot),\bar \Theta ^{\ast }(\cdot),0)\) is an optimal closed-loop strategy of Problem (MF-LQ)⁰ on [t,T], we have

Note that for any \(u(\cdot)\in {\mathcal {U}}[t,T]\) of the form

the corresponding solution X(·) of (3.16) satisfies \(\mathbb {E}[X(\cdot)]=0\) and hence \(\mathbb {E}[u(\cdot)]=0\). Then (3.17) yields

which implies that \(\Sigma \geqslant 0\). Likewise, for any \(u(\cdot)\in {\mathcal {U}}[t,T]\) of the form

the corresponding solution X(·) of (3.16) satisfies

Then (3.17) yields

which implies that \(\bar \Sigma \geqslant 0\). The proof is completed. □

The aim of this section is to provide a characterization of the closed-loop solvability of Problem (MF-LQ) in terms of the GRE (2.6), a linear backward stochastic differential equation (BSDE, for short), and a linear terminal value problem of ODE. In the case of Problem (MF-LQ)⁰, it turns out that the regular solvability of the GRE (2.6) is not only necessary but also sufficient for the existence of an optimal closed-loop strategy.

Theorem 4.1

Let (H1)–(H2) hold and t∈(0,T). Then Problem (MF-LQ) is closed-loop solvable on [t,T] if and only if the following hold:

(i) The GRE (2.6) admits a regular solution \((P(\cdot),\Pi (\cdot))\in C([t,T];\mathbb {S}^{n})\times C([t,T];\mathbb {S}^{n})\).

(ii) The adapted solution (η(·),ζ(·)) to the BSDE

satisfies

and the solution \(\bar \eta (\cdot)\) to the ODE

satisfies

where

In the above case, the optimal closed-loop strategy \((\Theta ^{\ast }(\cdot),\bar \Theta ^{\ast }(\cdot),u^{\ast }(\cdot))\) admits the following representation:

where \(\theta (\cdot),\tau (\cdot)\in L^{2}(t,T;\mathbb {R}^{m\times n})\), \(\nu (\cdot)\in L_{\mathbb {F}}^{2}(t,T;\mathbb {R}^{m})\), and \(\bar \nu (\cdot)\in L^{2}(t,T;\mathbb {R}^{m})\). Moreover, the value V(t,ξ) is given by

Proof.

Necessity. Suppose that is an optimal closed-loop strategy of Problem (MF-LQ) on [t,T]. Then it follows from Theorem 3.3 that the GRE (2.6) admits a regular solution (P(·),Π(·)). To determine u ^∗(·), let (X ^∗(·),Y ^∗(·),Z ^∗(·)) be the adapted solution of (3.1). Proceeding as in the proof of Proposition 3.1, we see that (X ^∗(·),Y ^∗(·),Z ^∗(·)) satisfies (3.2). Now, let

and define

By the terminal value of Y ^∗(T) (see (3.1)) and P(T)=G, we obtain

Also, making use of \(\Pi (T)=G+\bar G\), we obtain

Further, it is obvious that

and

We now want to show that

To this end, we recall from the proof of Theorem 3.3 that

and (recall (4.5) and (4.8))

Then we have

Also, we have

Moreover, we have from (3.2):

Now (4.9), (4.12), (4.16) and (4.17) yield

Hence,

Since \(\bar \Sigma ^{\dag }\left [(B+\bar B)^{\top }\widetilde \eta +(D+\bar D)^{\top }(\mathbb {E}[\beta ]+P\mathbb {E}[\sigma ])+\mathbb {E}[\rho ]+\bar \rho \right ] =-\bar \Sigma ^{\dag }\bar \Sigma \mathbb {E}[u^{\ast }]\) and \(\bar \Sigma ^{\dag }\bar \Sigma \) is an orthogonal projection, we have

for some \(\bar \nu (\cdot)\in L^{2}(t,T;\mathbb {R}^{m})\). Consequently,

Likewise, (4.9), (4.13), (4.15) and (4.18) yield

which implies that

and that

for some \(\nu (\cdot)\in L_{\mathbb {F}}^{2}(t,T;\mathbb {R}^{m})\). Consequently,

Since the adapted solution (η(·),ζ(·)) of (4.1) satisfies

by the uniqueness of solutions, we have proved (4.14). Then, (4.19) and (4.20) yield

This proves the necessity, as well as (4.6).

Sufficiency. The proof is much like that of (Theorem 5.2 Sun2016). Let \((\Theta ^{\ast }(\cdot),\bar \Theta ^{\ast }(\cdot),u^{\ast }(\cdot))\) be defined by (4.6). Then we have

For any \(\xi \in L^{2}_{{\mathcal {F}}_{t}}(\Omega ;\mathbb {R}^{n})\) and \(u(\cdot)\in {\mathcal {U}}[t,T]\), let X(·)≡X(· ;t,ξ,u(·)) be the corresponding solution of (1.1). Proceeding similarly to the proof of (Theorem 5.2 Sun2016) and using (4.21), we obtain

Since \(\Sigma,\bar \Sigma \geqslant 0\), (4.22) implies that

Therefore, \((\Theta ^{\ast }(\cdot),\bar \Theta ^{\ast }(\cdot),u^{\ast }(\cdot))\) is an optimal closed-loop strategy of Problem (MF-LQ) on [t,T] and (4.7) holds. The proof is completed. □

In the special case that b(·), σ(·), g, \(\bar g\), q(·), \(\bar q(\cdot)\), ρ(·), and \(\bar \rho (\cdot)\) vanish, if the GRE (2.6) admits a regular solution (P(·),Π(·)), then condition (ii) of Theorem 4.1 holds automatically. Indeed, one can easily check that (η(·),ζ(·))=(0,0) and \(\bar \eta (\cdot)=0\). Thus, we have the following corollary.

Corollary 4.2

Let (H1)–(H2) hold and t∈(0,T). Then Problem (MF-LQ) ⁰ is closed-loop solvable on [t,T] if and only if the GRE (2.6) is regularly solvable.

This is an important yet challenging research topic. Recently there has been increasing interest in studying this type of stochastic control problems as well as their applications. Beside this work, the optimal stochastic control problems under MF-SDEs are underdeveloped in the literature, and therefore many fundamental questions remain open and methodologies need to be significantly improved. To establish new theory and hopefully to shed light on financial investment, we expect the findings of this research program to add to various streams of the literature, such as portfolio selection, optimal control techniques, financial risk management, and relative performance evaluation.

Authors’ contributions

All authors read and approved the final manuscript.

Authors’ information

Xun Li was partially supported by Hong Kong RGC under grants 519913, 15209614 and 15224215. Jingrui Sun was partially supported by the National Natural Science Foundation of China (11401556) and the Fundamental Research Funds for the Central Universities (WK 2040000012). Jiongmin Yong was partially supported by NSF DMS-1406776.

Competing interests

The authors declare that they have no competing interests.

Affiliations

1. Department of Applied Mathematics, The Hong Kong Polytechnic University, Hong Kong, China

   • Xun Li
   •  & Jingrui Sun

2. Department of Mathematics, University of Central Florida, Orlando, 32816, FL, USA

   • Jiongmin Yong

Corresponding author

Correspondence to Jiongmin Yong.

Reprints and Permissions