In this section we will deduce necessary conditions for the closed-loop solvability of Problem (MF-LQ). In particular, we shall establish the necessity of the regular solvability of GRE (2.6) by a matrix minimum principle.
Let \(\Theta ^{*}(\cdot),\bar \Theta ^{*}(\cdot)\in L^{2}(t,T;\mathbb {R}^{m\times n})\) and consider the following state equation
$$\begin{aligned} \left\{\begin{array}{ll} dX(s)=\left\{AX\,+\,\bar{A}\mathbb{E}[\!X]\!+B\left(\Theta^{*}X\,+\,\bar{\Theta}^{*}\mathbb{E}[\!X]\!+u\right) \,+\,\bar{B}\mathbb{E}\left(\Theta^{*}X\,+\,\bar{\Theta}^{*}\mathbb{E}[\!X]\!+u\right)\,+\,b\right\}ds\\ \qquad\quad~~~+\!\left\{\!CX\,+\,\bar{C}\mathbb{E}[\!X]\!+D\left(\Theta^{*} X\,+\,\bar{\Theta}^{*}\mathbb{E}[\!X]+u\right) \,+\,\bar{D}\mathbb{E}\left(\Theta^{*}X\,+\,\bar{\Theta}^{*}\mathbb{E}[\!X]\!+u\right)\,+\,\sigma\!\right\}\!dW(s),\\ X(t)=\xi, \end{array} \right. \end{aligned} $$
and cost functional
$$\begin{aligned} &\widetilde{J}(t,\xi;u(\cdot))\triangleq J(t,\xi;\Theta^{*}(\cdot) X(\cdot)+\bar{\Theta}^{*}(\cdot)\mathbb{E}[X(\cdot)]+u(\cdot))\\ &=\mathbb{E}\left\{\vphantom{+{\int_{t}^{T}} \left[\left\langle\left(\begin{array}{cc}Q&S^{\top}\\S&R\end{array}\right) \left(\begin{array}{c}X\\ \Theta^{*}X+\bar\Theta^{*}\mathbb{E}[X]+u\end{array}\right), \left(\begin{array}{c}X\\ \Theta^{*}X+\bar\Theta^{*}\mathbb{E}[X]+u\end{array}\right)\right\rangle\right.}\langle GX(T),X(T)\rangle+2\langle g,X(T)\rangle +\left\langle\bar G\mathbb{E}[X(T)],\mathbb{E}[X(T)]\right\rangle+2\langle\bar g,\mathbb{E}[X(T)]\rangle\right.\\ &\quad+{\int_{t}^{T}} \left[\left\langle\left(\begin{array}{cc}Q&S^{\top}\\S&R\end{array}\right) \left(\begin{array}{c}X\\ \Theta^{*}X+\bar\Theta^{*}\mathbb{E}[X]+u\end{array}\right), \left(\begin{array}{c}X\\ \Theta^{*}X+\bar\Theta^{*}\mathbb{E}[X]+u\end{array}\right)\right\rangle\right.\\ &\left.\quad+2\left\langle\left(\begin{array}{c}q\\ \rho\end{array}\right), \left(\begin{array}{c}X\\ \Theta^{*}X+\bar\Theta^{*}\mathbb{E}[X]+u\end{array}\right) \right\rangle\right]ds\\ &\quad+{\int_{t}^{T}} \left[\left\langle\left(\begin{array}{ll}\bar Q&\bar S^{\top}\\\bar S&\bar R\end{array}\right) \left(\begin{array}{c}\mathbb{E}[X]\\(\Theta^{*}+\bar\Theta^{*})\mathbb{E}[X]+\mathbb{E}[u]\end{array}\right), \left(\begin{array}{c}\mathbb{E}[X]\\(\Theta^{*}+\bar\Theta^{*})\mathbb{E}[X]+\mathbb{E}[u]\end{array}\right)\right\rangle\right.\\ &\left.\left.\quad+2\left\langle\left(\begin{array}{c}\bar q\\ \bar\rho\end{array}\right), \left(\begin{array}{c}\mathbb{E}[X]\\(\Theta^{*}+\bar\Theta^{*})\mathbb{E}[X]+\mathbb{E}[u]\end{array}\right) \right\rangle\right]ds\right\}\\ &=\mathbb{E}\left\{\vphantom{+{\int_{t}^{T}} \left[\left\langle\left(\begin{array}{cc}Q&S^{\top}\\S&R\end{array}\right) \left(\begin{array}{c}X\\ \Theta^{*}X+\bar\Theta^{*}\mathbb{E}[X]+u\end{array}\right), \left(\begin{array}{c}X\\ \Theta^{*}X+\bar\Theta^{*}\mathbb{E}[X]+u\end{array}\right)\right\rangle\right.}\langle GX(T),X(T)\rangle+2\langle g,X(T)\rangle +\left\langle\bar G\mathbb{E}[X(T)],\mathbb{E}[X(T)]\right\rangle+2\langle\bar g,\mathbb{E}[X(T)]\rangle\right.\\ &\quad+{\int_{t}^{T}} \left[\left\langle\left(\begin{array}{ll}\widetilde Q&\widetilde S^{\top}\\ \widetilde S&R\end{array}\right) \left(\begin{array}{c}X\\ u\end{array}\right), \left(\begin{array}{c}X\\ u\end{array}\right)\right\rangle +2\left\langle\left(\begin{array}{c}\widetilde q\\ \rho\end{array}\right), \left(\begin{array}{c}X\\ u\end{array}\right)\right\rangle\right]ds\\ &\left.\quad+{\int_{t}^{T}} \left[\left\langle\left(\begin{array}{cc}\widehat Q&\widehat S^{\top}\\ \widehat S&\bar R\end{array}\right) \left(\begin{array}{c}\mathbb{E}[X]\\ \mathbb{E}[u]\end{array}\right), \left(\begin{array}{c}\mathbb{E}[X]\\ \mathbb{E}[u]\end{array}\right)\right\rangle +2\left\langle\left(\begin{array}{c}\widehat q\\ \bar\rho\end{array}\right), \left(\begin{array}{c}\mathbb{E}[X]\\\mathbb{E}[u]\end{array}\right) \right\rangle\right]ds\right\}, \end{aligned} $$
where
$${}\begin{aligned} \left\{ \begin{array}{cll} \widetilde Q&=Q+(\Theta^{*})^{\top} S+S^{\top}\Theta^{*}+(\Theta^{*})^{\top} R\Theta^{*}, \quad \widetilde S=S+R\Theta^{*},\quad\widetilde q=q+(\Theta^{*})^{\top}\!\rho,\\ \widehat Q&=\bar Q+(\Theta^{*}+\bar\Theta^{*})^{\top}\bar S+\bar S^{\top}(\Theta^{*}+\bar\Theta^{*}) +(\Theta^{*}+\bar\Theta^{*})^{\top}\bar R(\Theta^{*}+\bar\Theta^{*})\\ &\quad+(\bar\Theta^{*})^{\top} R\bar\Theta^{*}+(\bar\Theta^{*})^{\top} S+S^{\top}\bar\Theta^{*} +(\bar\Theta^{*})^{\top} R\Theta^{*}+(\Theta^{*})^{\top} R\bar\Theta^{*},\\ \widehat S&=\bar S+\bar R(\Theta^{*}+\bar \Theta^{*})+R\bar \Theta^{*}, \quad \widehat q=\bar{q}+(\Theta^{*}+\bar\Theta^{*})^{\top}\bar\rho+(\bar\Theta^{*})^{\top}\mathbb{E}[\rho]. \end{array} \right. \end{aligned} $$
By Proposition 2.4 (ii),
is an optimal closed-loop strategy of Problem (MF-LQ) on [t,T] if and only if for any \(\xi \in L^{2}_{{\mathcal {F}}_{t}}(\Omega ;\mathbb {R}^{n})\), u
∗(·) is an optimal open-loop control of the problem with the above state equation and cost functional. This leads to the following result.
Proposition 3.1
Let (H1)–(H2) hold. If
is an optimal closed-loop strategy of Problem (MF-LQ) on [t,T], then \((\Theta ^{*}(\cdot),\bar \Theta ^{*}(\cdot),0)\) is an optimal closed-loop strategy of Problem (MF-LQ)
0 on [t,T].
Proof.
By the preceding discussion and (Theorem 2.3, Sun 2016), we see that \((\Theta ^{*}(\cdot),\bar \Theta ^{*}(\cdot),u^{*}(\cdot))\) is an optimal closed-loop strategy of Problem (MF-LQ) on [t,T] if and only if for any \(\xi \in L^{2}_{{\mathcal {F}}_{t}}(\Omega ;\mathbb {R}^{n})\), the adapted solution (X
∗(·),Y
∗(·),Z
∗(·)) to the following mean-field forward-backward stochastic differential equation (MF-FBSDE, for short):
$$ \begin{aligned} \left\{\! \begin{array}{ll} dX^{*}(s)&=\left\{(A\,+\,B\Theta^{*})X^{*}\,+\,\left[\bar A\,+\,B\bar\Theta^{*}\,+\,\bar B(\Theta^{*}\,+\,\bar\Theta^{*})\right]\mathbb{E}[\!X^{*}]\! +Bu^{*}\,+\,\bar B\mathbb{E}[u^{*}]\!+b\right\}\!ds\\ &\quad+\!\left\{(C\,+\,D\Theta^{*})X^{*}\,+\,\left[\bar C\,+\,D\bar\Theta^{*}\,+\,\bar D(\Theta^{*}\,+\,\bar\Theta^{*})\right]\mathbb{E}[\!X^{*}] \!+Du^{*}\,+\,\bar D\mathbb{E}[u^{*}]\!+\sigma\!\right\}\!dW(s),\\ dY^{*}(s)&=-\left\{(A\,+\,B\Theta^{*})^{\top} Y^{*}\,+\,\left[\bar A\,+\,B\bar\Theta^{*}\,+\,\bar B(\Theta^{*}\,+\,\bar\Theta^{*})\right]^{\top}\mathbb{E}[Y^{*}]\right.\\ &\quad+(C\,+\,D\Theta^{*})^{\top} Z^{*}\,+\,\left[\bar C\,+\,D\bar\Theta^{*}\,+\,\bar D(\Theta^{*}\,+\,\bar\Theta^{*})\right]^{\top}\mathbb{E}[Z^{*}]\\ &\quad\left.+\widetilde QX^{*}\,+\,\widehat Q\mathbb{E}[\!X^{*}]\!+\widetilde S^{\top} u^{*}\,+\,\widehat S^{\top}\mathbb{E}[\!u^{*}]\!+\widetilde q\,+\,\widehat q\vphantom{(A+B\Theta^{*})^{\top} Y^{*}+\left[\bar A+B\bar\Theta^{*}+\bar B(\Theta^{*}+\bar\Theta^{*})\right]^{\top}\mathbb{E}[Y^{*}]}\right\}\!ds\,+\,Z^{*}dW(s), \qquad s\in[t,T],\\ X^{*}(t)&=\xi,\qquad Y^{*}(T)=GX^{*}(T)\,+\,\bar G\mathbb{E}[X^{*}(T)]+g+\bar g, \end{array} \right. \end{aligned} $$
(3.1)
satisfies
$$Ru^{*}+ B^{\top} Y^{*}+ D^{\top} Z^{*}+\widetilde SX^{*}+\rho +\bar R\mathbb{E}[u^{*}]+\bar B^{\top}\mathbb{E}[Y^{*}]+\bar D^{\top}\mathbb{E}[Z^{*}]+\widehat S\mathbb{E}[X^{*}]+\bar\rho=0, $$
(3.2)
and the following condition holds:
$$\begin{aligned} &\mathbb{E}\left\{\langle GX(T),X(T)\rangle\,+\,\left\langle\bar G\mathbb{E}[\!X(T)],\mathbb{E}[\!X(T)]\right\rangle +\!{\int_{t}^{T}} \!\left[ \langle\widetilde QX,X\rangle\,+\,2\left\langle\widetilde SX,u\right\rangle\,+\,\langle Ru,u\rangle \right] ds\right.\\ &\left.\quad+{\int_{t}^{T}} \left[\left\langle\widehat Q\mathbb{E}[X],\mathbb{E}[X]\right\rangle\,+\,2\left\langle\widehat S\mathbb{E}[\!X],\mathbb{E}[\!u]\right\rangle \,+\,\left\langle\bar R\mathbb{E}[\!u],\mathbb{E}[\!u] \right\rangle\right]\!ds\vphantom{GX(T),X(T)\rangle+\left\langle\bar G\mathbb{E}[X(T)],\mathbb{E}[X(T)]\right\rangle +{\int_{t}^{T}} \left[ \langle\widetilde QX,X\rangle+2\left\langle\widetilde SX,u\right\rangle+\langle Ru,u\rangle \right] ds}\right\} \!\geqslant0,\qquad\forall u(\cdot)\!\in\!{\mathcal{U}}[t,T], \end{aligned} $$
where X(·) is the solution of
$$\begin{aligned} \left\{\!\! \begin{array}{ll} dX(s)=\left\{(A+B\Theta^{\ast})X+\left[\bar A+B\bar\Theta^{\ast}+\bar B(\Theta^{\ast}+\bar\Theta^{\ast})\right]\mathbb{E}[X] +Bu+\bar B\mathbb{E}[u]\right\}ds \\ \qquad\qquad\;\; +\left\{(C+D\Theta^{\ast})X+\left[\bar C+D\bar\Theta^{\ast}+\bar D(\Theta^{\ast}+\bar\Theta^{\ast})\right]\mathbb{E}[X] +Du+\bar D\mathbb{E}[u]\right\}dW(s),\quad s\in[t,T], \\ X(t)=0. \end{array}\right. \end{aligned} $$
Since the MF-FBSDE (3.1) admits a solution for each \(\xi \in L^{2}_{{\mathcal {F}}_{t}}(\Omega ;\mathbb {R}^{n})\) and \((\Theta ^{\ast }(\cdot),\bar \Theta ^{\ast }(\cdot),u^{\ast }(\cdot))\) is independent of ξ, by subtracting solutions corresponding ξ and 0, the latter from the former, we see that for any \(\xi \in L^{2}_{{\mathcal {F}}_{t}}(\Omega ;\mathbb {R}^{n})\), the following MF-FBSDE:
$$\begin{aligned} \left\{\! \begin{array}{ll} dX(s)=\left\{(A\,+\,B\Theta^{\ast})X\,+\,\left[\bar A+B\bar\Theta^{\ast}\,+\,\bar B(\Theta^{\ast}\,+\,\bar\Theta^{\ast})\right]\mathbb{E}[\!X]\right\}\!ds\\ \qquad\quad~~~ +\!\left\{(C\,+\,D\Theta^{\ast})X\,+\,\left[\bar C\,+\,D\bar\Theta^{\ast}\,+\,\bar D(\Theta^{\ast}\,+\,\bar\Theta^{\ast})\right]\mathbb{E}[\!X]\!\right\}\!dW(s),\qquad s\in[t,T],\\ dY(s)=-\left\{(A\,+\,B\Theta^{\ast})^{\top} Y\,+\,\left[\bar A\,+\,B\bar\Theta^{\ast}\,+\,\bar B(\Theta^{\ast}\,+\,\bar\Theta^{\ast})\right]^{\top}\mathbb{E}[\!Y]\!+(C\,+\,D\Theta^{\ast})^{\top} Z\right.\\ \left.\qquad\qquad\quad +\!\left[\bar C\,+\,D\bar\Theta^{\ast}\,+\,\bar D(\Theta^{\ast}\,+\,\bar\Theta^{\ast})\right]^{\top}\!\mathbb{E}[\!Z]\!+\widetilde QX\,+\,\widehat Q\mathbb{E}[\!X]\right\}\!ds\,+\,ZdW(s),\quad s\in[t,T],\\ X(t)=\xi,\qquad Y(T)=GX(T)\,+\,\bar G\mathbb{E}[X(T)], \end{array} \right. \end{aligned} $$
also admits an adapted solution (X(·),Y(·),Z(·)) satisfying
$$B^{\top} Y+ D^{\top} Z+\widetilde SX+\bar B^{\top}\mathbb{E}[Y]+\bar D^{\top}\mathbb{E}[Z]+\widehat S\mathbb{E}[X]=0. $$
It follows, again from (Theorem 2.3, Sun2016), that \((\Theta ^{\ast }(\cdot),\bar \Theta ^{\ast }(\cdot),0)\) is an optimal closed-loop strategy of Problem (MF-LQ)0 on [t,T]. □
Now let us look at Problem (MF-LQ)0. If we consider only closed-loop strategies of the form \((\Theta (\cdot),\bar \Theta (\cdot),0)\), then the state equation becomes
$$\begin{aligned} \left\{ \begin{array}{ll} dX(s)=\left\{(A+B\Theta)X+\left[\bar A+\bar B\Theta+(B+\bar B)\bar\Theta\right]\mathbb{E}[X]\right\}ds\\ \qquad\quad~~~+\left\{(C+D\Theta)X+\left[\bar C+\bar D\Theta+(D+\bar D)\bar\Theta\right]\mathbb{E}[X]\right\}dW(s), \qquad s\in[t,T], \\ X(t)=\xi, \end{array}\right. \end{aligned} $$
and \(\mathbb {E}[X(\cdot)]\) satisfies
$$\left\{ \begin{array}{ll} d\mathbb{E}[X(s)]=\left[A+\bar A+(B+\bar B)(\Theta+\bar\Theta)\right]\mathbb{E}[X]ds, \qquad s\in[t,T], \\ \mathbb{E}[X(t)]=\mathbb{E}[\xi]. \end{array} \right. $$
By Itô’s formula, the matrices \(\mathbf {X}(s)\triangleq \mathbb {E}\left [X(s)X(s)^{\top }\right ]\) and \(\mathbf {Y}(s)\triangleq \mathbb {E}[X(s)]\mathbb {E}[X(s)]^{\top }\) satisfy the matrix-valued ordinary differential equations (ODEs, for short)
$$ \begin{aligned} \left\{ \begin{array}{ll} \dot{\mathbf{X}}=(A+B\Theta)\mathbf{X}+\mathbf{X}(A+B\Theta)^{\top}+(C+D\Theta)\mathbf{X}(C+D\Theta)^{\top}\\ \qquad~+\left[\bar A+\bar B\Theta+(B+\bar B)\bar\Theta\right]\mathbf{Y} +\mathbf{Y}\left[\bar A+\bar B\Theta+(B+\bar B)\bar\Theta\right]^{\top}\\ \qquad~+(C+D\Theta)\mathbf{Y}\left[\bar C+\bar D\Theta+(D+\bar D)\bar\Theta\right]^{\top} +\left[\bar C+\bar D\Theta+(D+\bar D)\bar\Theta\right]\mathbf{Y}(C+D\Theta)^{\top}\\ \qquad~+\left[\bar C+\bar D\Theta+(D+\bar D)\bar\Theta\right]\mathbf{Y} \left[\bar C+\bar D\Theta+(D+\bar D)\bar\Theta\right]^{\top}, \qquad s\in[t,T],\\ \mathbf{X}(t)=\mathbb{E}[\xi\xi^{\top}], \end{array}\right. \end{aligned} $$
(3.3)
and
$$ \begin{aligned}\left\{ \begin{array}{ll} \dot{\mathbf{Y}}=\left[A+\bar A+(B+\bar B)(\Theta+\bar\Theta)\right]\mathbf{Y} +\mathbf{Y}\left[A+\bar A+(B+\bar B)(\Theta+\bar\Theta)\right]^{\top}, \qquad s\in[t,T],\\ \mathbf{Y}(t)=\mathbb{E}[\xi]\mathbb{E}[\xi]^{\top}, \end{array}\right. \end{aligned} $$
(3.4)
respectively. The cost functional \(J^{0}(t,\xi ;\Theta (\cdot)X(\cdot)+\bar \Theta (\cdot)\mathbb {E}[X(\cdot)])\) can be expressed equivalently as
$$ \mathbf{J}(t,\xi;\Theta(\cdot),\bar\Theta(\cdot))=\text{tr}[G\mathbf{X}(T)+\bar G\mathbf{Y}(T)] +{\int_{t}^{T}}\text{tr}[M(s)\mathbf{X}(s)+N(s)\mathbf{Y}(s)]ds, $$
(3.5)
where
$$\left\{\begin{array}{cll} M&=Q+\Theta^{\top} S+S^{\top}\Theta+\Theta^{\top} R\Theta, \\ N&=\bar Q+(\Theta+\bar\Theta)^{\top}\bar S+\bar S^{\top}(\Theta+\bar\Theta)+(\Theta+\bar\Theta)^{\top}\bar R(\Theta+\bar\Theta)\\ &\qquad+\bar\Theta^{\top} R\bar\Theta+\bar\Theta^{\top} S+S^{\top}\bar\Theta+\bar\Theta^{\top} R\Theta+\Theta^{\top} R\bar\Theta. \end{array} \right. $$
Then we may pose the following deterministic optimal control problem.
Problem (O). For any given \((t,\xi)\in [0,T)\times L^{2}_{{\mathcal {F}}_{t}}(\Omega ;\mathbb {R}^{n})\), find \(\Theta ^{\ast }(\cdot),\bar \Theta ^{\ast }(\cdot)\in L^{2}(t,T;\mathbb {R}^{m\times n})\) such that
$$\begin{array}{ll} \mathbf{J}(t,\xi;\Theta^{\ast}(\cdot),\bar\Theta^{\ast}(\cdot))\leqslant\mathbf{J}(t,\xi;\Theta(\cdot),\bar\Theta(\cdot)), \qquad\forall \Theta(\cdot),\bar\Theta(\cdot)\in L^{2}(t,T;\mathbb{R}^{m\times n}). \end{array} $$
Rewrite (3.3)–(3.4) as
$$\left\{ \begin{array}{ll} \left(\begin{array}{c}\dot{\mathbf{X}}(s)\\ \dot{\mathbf{Y}}(s)\end{array}\right) =\left(\begin{array}{c}F_{1}(\mathbf{X}(s),\mathbf{Y}(s),\Theta(s),\bar\Theta(s),s)\\ F_{2}(\mathbf{Y}(s),\Theta(s),\bar\Theta(s),s)\end{array}\right),\qquad s\in[t,T],\\ \mathbf{X}(t)=\mathbb{E}[\xi\xi^{\top}],\quad \mathbf{Y}(t)=\mathbb{E}[\xi]\mathbb{E}[\xi]^{\top}, \end{array}\right. $$
and denote the integrand in (3.5) by \(L(\mathbf {X}(s),\mathbf {Y}(s),\Theta (s),\bar \Theta (s),s)\). We present the following matrix minimum principle for Problem (O). The interested reader is referred to Athans (1968) for a proof.
Lemma 3.2
Let (H1)–(H2) hold. Suppose that \((\Theta ^{\ast }(\cdot),\bar \Theta ^{\ast }(\cdot))\) is an optimal control of Problem (O) for the initial pair (t,ξ) and let (X
∗(·),Y
∗(·)) be the corresponding optimal state process. Then there exist matrix-valued functions P(·) and Λ(·) satisfying the following ODEs (the variable s∈[t,T] is suppressed)
$$ \begin{aligned}\left\{\! \begin{array}{ll} \left(\begin{array}{c}\dot P\\ \dot\Lambda\end{array}\right)\,=\,-\!\left(\!\begin{array}{c} {{\partial}\over{\partial} \mathbf{X}^{\ast}}L(\mathbf{X}^{\ast},\mathbf{Y}^{\ast},\Theta^{\ast},\bar\Theta^{\ast}) \,+\,{{\partial}\over{\partial} \mathbf{X}^{\ast}}\text{tr}[\!F_{1}(\mathbf{X}^{\ast},\mathbf{Y}^{\ast},\Theta^{\ast},\bar\Theta^{\ast})P^{\top}\,+\,F_{2}(\mathbf{Y}^{\ast},\Theta^{\ast},\bar\Theta^{\ast})\Lambda^{\top}]\\ {{\partial}\over{\partial} \mathbf{Y}^{\ast}}L(\mathbf{X}^{\ast},\mathbf{Y}^{\ast},\Theta^{\ast},\bar\Theta^{\ast}) \,+\,{{\partial}\over{\partial} \mathbf{Y}^{\ast}}\text{tr}[\!F_{1}(\mathbf{X}^{\ast},\mathbf{Y}^{\ast},\Theta^{\ast},\bar\Theta^{\ast})P^{\top}\,+\,F_{2}(\mathbf{Y}^{\ast},\Theta^{\ast},\bar\Theta^{\ast})\Lambda^{\top}] \end{array}\right),\\ P(T)=G,\qquad \Lambda(T)=\bar G, \end{array}\right. \end{aligned} $$
(3.6)
with constraints
$$ \begin{aligned} \left\{ \begin{array}{ll} {{\partial}\over{\partial} \Theta^{\ast}}L(\mathbf{X}^{\ast},\mathbf{Y}^{\ast},\Theta^{\ast},\bar\Theta^{\ast}) +{{\partial}\over{\partial} \Theta^{\ast}}\text{tr}[F_{1}(\mathbf{X}^{\ast},\mathbf{Y}^{\ast},\Theta^{\ast},\bar\Theta^{\ast})P^{\top}+F_{2}(\mathbf{Y}^{\ast},\Theta^{\ast},\bar\Theta^{\ast})\Lambda^{\top}]=0,\\ {{\partial}\over{\partial} \bar\Theta^{\ast}}L(\mathbf{X}^{\ast},\mathbf{Y}^{\ast},\Theta^{\ast},\bar\Theta^{\ast}) +{{\partial}\over{\partial} \bar\Theta^{\ast}}\text{tr}[F_{1}(\mathbf{X}^{\ast},\mathbf{Y}^{\ast},\Theta^{\ast},\bar\Theta^{\ast})P^{\top}+F_{2}(\mathbf{Y}^{\ast},\Theta^{\ast},\bar\Theta^{\ast})\Lambda^{\top}]=0. \end{array} \right. \end{aligned} $$
(3.7)
Now, we are ready to state and prove the principal result of this section.
Theorem 3.3
Let (H1)–(H2) hold and t∈(0,T). If Problem (MF-LQ) admits an optimal closed-loop strategy on [t,T], then the GRE (2.6) is regularly solvable on [t,T].
Proof.
Suppose that
is an optimal closed-loop strategy of Problem (MF-LQ) on [t,T]. Then, by Proposition 3.1, \((\Theta ^{\ast }(\cdot),\bar \Theta ^{\ast }(\cdot),0)\) is an optimal closed-loop strategy of Problem (MF-LQ) 0 on [t,T], and it follows from Definition 2.3 (i) that \((\Theta ^{\ast }(\cdot),\bar \Theta ^{\ast }(\cdot))\) is an optimal control of Problem (O) for any \(\xi \in L^{2}_{{\mathcal {F}}_{t}}(\Omega ;\mathbb {R}^{n})\). Thus, by the matrix minimum principle, Lemma 3.2, there exist functions \(P(\cdot),\Lambda (\cdot):[t,T]\to \mathbb {R}^{n}\) such that (3.6)–(3.7) hold. By a straightforward calculation, we see from the first equation in (3.6) that P(·) satisfies
$$ \left\{ \begin{array}{ll} \dot P+(A+B\Theta^{\ast})^{\top} P+P(A+B\Theta^{\ast})+(C+D\Theta^{\ast})^{\top} P(C+D\Theta^{\ast})\\ \quad+\,Q+(\Theta^{\ast})^{\top} S+S^{\top}\Theta^{\ast}+(\Theta^{\ast})^{\top} R\Theta^{\ast}=0,\\ P(T)=G, \end{array} \right. $$
(3.8)
and from the second equation in (3.6), we see that Λ(·) satisfies
$$ \begin{aligned} \left\{ \begin{array}{ll} 0=\dot\Lambda+\bar Q+(\Theta^{\ast}+\bar\Theta^{\ast})^{\top}\bar S+\bar S^{\top}(\Theta^{\ast}+\bar\Theta^{\ast}) +(\Theta^{\ast}+\bar\Theta^{\ast})^{\top}\bar R(\Theta^{\ast}+\bar\Theta^{\ast})\\ \qquad~~~+(\bar\Theta^{\ast})^{\top} R\bar\Theta^{\ast}+(\bar\Theta^{\ast})^{\top} S+S^{\top}\bar\Theta^{\ast} +(\bar\Theta^{\ast})^{\top} R\Theta^{\ast}+(\Theta^{\ast})^{\top} R\bar\Theta^{\ast}\\ \qquad~~~+\left[\bar A+\bar B\Theta^{\ast}+(B+\bar B)\bar\Theta^{\ast}\right]^{\top} P +P\left[\bar A+\bar B\Theta^{\ast}+(B+\bar B)\bar\Theta^{\ast}\right]\\ \qquad~~~+\left[C+D\Theta^{\ast}\right]^{\top} P\left[\bar C+\bar D\Theta^{\ast}+(D+\bar D)\bar\Theta^{\ast}\right]\\ \qquad~~~+\left[\bar C+\bar D\Theta^{\ast}+(D+\bar D)\bar\Theta^{\ast}\right]^{\top} P\left[C+D\Theta^{\ast}\right]\\ \qquad~~~+\left[\bar C+\bar D\Theta^{\ast}+(D+\bar D)\bar\Theta^{\ast}\right]^{\top} P \left[\bar C+\bar D\Theta^{\ast}+(D+\bar D)\bar\Theta^{\ast}\right]\\ \qquad~~~+\left[A+\bar A+(B+\bar B)(\Theta^{\ast}+\bar\Theta^{\ast})\right]^{\top}\Lambda +\Lambda\left[A+\bar A+(B+\bar B)(\Theta^{\ast}+\bar\Theta^{\ast})\right],\\ \Lambda(T)=\bar G. \end{array}\right. \end{aligned} $$
(3.9)
Note that P(·)⊤ and Λ(·)⊤ also solve (3.8) and (3.9), respectively. Hence, by uniqueness, we have P(·)=P(·)⊤ and Λ(·)=Λ(·)⊤. Let
$$\Pi(\cdot)=P(\cdot)+\Lambda(\cdot),\qquad \Delta(\cdot)=\Theta^{\ast}(\cdot)+\bar\Theta^{\ast}(\cdot). $$
Then, \(\Pi (T)=G+\bar G\) and
$$ \begin{array}{lllll} 0=\dot\Pi+Q+\bar Q+\Delta^{\top}(S+\bar S)+(S+\bar S)^{\top}\Delta+\Delta^{\top}(R+\bar R)\Delta\\ \qquad~~~+\left[A+\bar A+(B+\bar B)\Delta\right]^{\top} P+P\left[A+\bar A+(B+\bar B)\Delta\right]\\ \qquad~~~+\left[C+D\Theta^{\ast}\right]^{\top} P\left[C+\bar C+(D+\bar D)\Delta\right]\\ \qquad~~~+\left[\bar C+\bar D\Theta^{\ast}+(D+\bar D)\bar\Theta^{\ast}\right]^{\top} P\left[C+\bar C+(D+\bar D)\Delta\right]\\ \qquad~~~+\left[A+\bar A+(B+\bar B)\Delta\right]^{\top}\Lambda+\Lambda\left[A+\bar A+(B+\bar B)\Delta\right]\\ ~~=\dot\Pi+Q+\bar Q+\Delta^{\top}(S+\bar S)+(S+\bar S)^{\top}\Delta+\Delta^{\top}(R+\bar R)\Delta\\ \qquad~~~+\left[A+\bar A+(B+\bar B)\Delta\right]^{\top}\Pi+\Pi\left[A+\bar A+(B+\bar B)\Delta\right]\\ \qquad~~~+\left[C+\bar C+(D+\bar D)\Delta\right]^{\top} P\left[C+\bar C+(D+\bar D)\Delta\right]\\ ~~=\dot\Pi+\left[A+\bar A+(B+\bar B)\Delta\right]^{\top}\Pi+\Pi\left[A+\bar A+(B+\bar B)\Delta\right]\\ \qquad+Q+\bar Q+(C+\bar C)^{\top} P(C+\bar C)+\Delta^{\top}\left[R+\bar R+(D+\bar D)^{\top} P(D+\bar D)\right]\Delta\\ \qquad+\Delta^{\top}\left[(D+\bar D)^{\top} P(C+\bar C)+S+\bar S\right] +\left[(D+\bar D)^{\top} P(C+\bar C)+S+\bar S\right]^{\top}\Delta. \end{array} $$
(3.10)
Also, from the first equality in (3.7), we have (noting that X
∗ and Y
∗ are symmetric)
$$ \begin{aligned} \begin{array}{lll} 0=2S\mathbf{X}^{\ast}+2R\Theta^{\ast}\mathbf{X}^{\ast}+2\bar S\mathbf{Y}^{\ast}+2\bar R\Theta^{\ast}\mathbf{Y}^{\ast} +2\bar R\bar\Theta^{\ast}\mathbf{Y}^{\ast}+2R\bar\Theta^{\ast}\mathbf{Y}^{\ast}\\ \qquad+\,2B^{\top} P\mathbf{X}^{\ast}+2D^{\top} PC\mathbf{X}^{\ast}+2D^{\top} PD\Theta^{\ast}\mathbf{X}^{\ast}\\ \qquad+\,2\bar B^{\top} P\mathbf{Y}^{\ast}+2\bar D^{\top} PC\mathbf{Y}^{\ast} +2D^{\top} P\left[\bar C+(D+\bar D)\bar\Theta^{\ast}\right]\mathbf{Y}^{\ast}\\ \qquad+\,2D^{\top} P\bar D\Theta^{\ast}\mathbf{Y}^{\ast}+2\bar D^{\top} PD\Theta^{\ast}\mathbf{Y}^{\ast} +2\bar D^{\top} P\left[\bar C+(D+\bar D)\bar\Theta^{\ast}\right]\mathbf{Y}^{\ast}\\ \qquad+\,2\bar D^{\top} P\bar D\Theta^{\ast}\mathbf{Y}^{\ast}+2(B+\bar B)^{\top}\Lambda \mathbf{Y}^{\ast}\\ =2\left[(R+D^{\top} PD)\Theta^{\ast}+B^{\top} P+D^{\top} PC+S\right]\mathbf{X}^{\ast}\\ \qquad+2\left\{(R+\bar R)\bar\Theta^{\ast}+(\bar R+\bar D^{\top} PD)\Theta^{\ast} +\bar B^{\top} P+\bar D^{\top} PC+\bar S\right.\\ \qquad~~~~~~~~~~\left.+(D+\bar D)^{\top} P\left[\bar C+\bar D\Theta^{\ast}+(D+\bar D)\bar\Theta^{\ast}\right] +(B+\bar B)^{\top}\Lambda \right\}\mathbf{Y}^{\ast}\\ =2\left[(R+D^{\top} PD)\Theta^{\ast}+B^{\top} P+D^{\top} PC+S\right]\mathbf{X}^{\ast}\\ \qquad+\,2\left\{-\left[(R+D^{\top} PD)\Theta^{\ast}+B^{\top} P+D^{\top} PC+S\right]\right.\\ \qquad\qquad~~\left.+(R+\bar R)\Delta+(B+\bar B)^{\top}\Pi +(D+\bar D)^{\top} P\left[C+\bar C+(D+\bar D)\Delta\right]+S+\bar S\right\}\mathbf{Y}^{\ast}\\ =2\left[(R+D^{\top} PD)\Theta^{\ast}+B^{\top} P+D^{\top} PC+S\right](\mathbf{X}^{\ast}-\mathbf{Y}^{\ast})\\ \qquad+\,2\left\{\left[R+\bar R+(D+\bar D)^{\top} P(D+\bar D)\right]\Delta+(B+\bar B)^{\top}\Pi\right.\\ \qquad\qquad\left.+(D+\bar D)^{\top} P(C+\bar C)+S+\bar S\right\}\mathbf{Y}^{\ast}. \end{array} \end{aligned} $$
(3.11)
Likewise, from the second equality in (3.7), we have
$$ \begin{array}{ll} 2\left\{\left[R+\bar R+(D+\bar D)^{\top} P(D+\bar D)\right]\Delta+(B+\bar B)^{\top}\Pi\right.\\ \quad~\left.+(D+\bar D)^{\top} P(C+\bar C)+S+\bar S\right\}\mathbf{Y}^{\ast}=0. \end{array} $$
(3.12)
Let Φ(·) be the solution to the \(\mathbb {R}^{n\times n}\)-valued ODE
$$\left\{\begin{array}{ll} \dot \Phi(s)=\widetilde A(s)\Phi(s),\qquad s\in[t,T],\\ \Phi(t)=I, \end{array} \right. $$
where
$$\widetilde A\triangleq A+\bar A+(B+\bar B)(\Theta^{\ast}+\bar\Theta^{\ast})=A+\bar A+(B+\bar B)\Delta. $$
Then
$$\mathbf{Y}^{\ast}(s)=\Phi(s)\mathbb{E}[\xi]\mathbb{E}[\xi]^{\top}\Phi(s)^{\top},\qquad s\in[t,T]. $$
Denoting \(\bar \Sigma \equiv R+\bar R+(D+\bar D)^{\top } P(D+\bar D)\), since (3.12) holds for all \(\xi \in L_{{\mathcal {F}}_{t}}^{2}(\Omega ;\mathbb {R}^{n})\) and Φ(s) is invertible for all s∈[t,T], we must have
$$ \bar\Sigma\Delta+(B+\bar B)^{\top}\Pi+(D+\bar D)^{\top} P(C+\bar C)+S+\bar S=0. $$
(3.13)
Now take \(\eta \in L_{{\mathcal {F}}_{t}}^{2}(\Omega ;\mathbb {R})\) with \(\mathbb {E}\eta =0\) and \(\mathbb {E}\eta ^{2}=1\). Then for any \(x\in \mathbb {R}^{n}\), the solution Y
∗ of (3.4) corresponding to \((\Theta,\bar \Theta)=(\Theta ^{\ast },\bar \Theta ^{\ast })\) and ξ=η
x is identically zero, and the corresponding solution X
∗ of (3.3) satisfies
$$\begin{aligned} \left\{\begin{array}{ll} \dot{\mathbf{X}}^{\ast}=(A+B\Theta^{\ast})\mathbf{X}^{\ast}+\mathbf{X}^{\ast}(A+B\Theta^{\ast})^{\top}+(C+D\Theta^{\ast})\mathbf{X}^{\ast}(C+D\Theta^{\ast})^{\top}, \qquad s\in[t,T],\\ \mathbf{X}^{\ast}(t)=xx^{\top}, \end{array}\right. \end{aligned} $$
Let Ψ(·) be the solution to the following SDE for \(\mathbb {R}^{n\times n}\)-valued process:
$$\begin{aligned} \left\{\begin{array}{ll} d\Psi(s)=\left[A(s)+B(s)\Theta^{\ast}(s)\right]\Psi(s)ds \,+\,\left[C(s)+D(s)\Theta^{\ast}(s)\right]\Psi(s)dW(s),\qquad s\in[t,T],\\ \Psi(t)=I. \end{array}\right. \end{aligned} $$
Then we have
$$\mathbf{X}^{\ast}(s)=\mathbb{E}\left[\Psi(s)xx^{\top}\Psi(s)^{\top}\right],\qquad s\in[t,T]. $$
Hence, denoting Σ≡R+D
⊤
PD, we obtain from (3.11) that
$$\left(\Sigma\Theta^{\ast}+B^{\top} P+D^{\top} PC+S\right)\mathbb{E}\left[\Psi xx^{\top}\Psi^{\top}\right]=0, \qquad\forall x\in\mathbb{R}^{n}, $$
which implies Σ
Θ
∗+B
⊤
P+D
⊤
PC+S=0. It follows that \({\mathcal {R}}(B^{\top } P+D^{\top } PC+S)\subseteq {\mathcal {R}}(\Sigma)\). Moreover, since Σ
†
Σ is an orthogonal projection, we have
$$\Sigma^{\dag}(B^{\top} P+D^{\top} PC+S)\in L^{2}(t,T;\mathbb{R}^{m\times n}), $$
and
$$ \Theta^{\ast}=-\Sigma^{\dag}(B^{\top} P+D^{\top} PC+S)+(I-\Sigma^{\dag}\Sigma)\theta, $$
(3.14)
for some \(\theta (\cdot)\in L^{2}(t,T;\mathbb {R}^{m\times n})\). Similarly, from (3.13) we have
$${\mathcal{R}}\left((B+\bar B)^{\top}\Pi+(D+\bar D)^{\top} P(C+\bar C)+(S+\bar S)\right) \subseteq{\mathcal{R}}(\bar \Sigma), $$
$$\bar\Sigma^{\dag}\left[(B+\bar B)^{\top}\Pi+(D+\bar D)^{\top} P(C+\bar C)+(S+\bar S)\right] \in L^{2}(t,T;\mathbb{R}^{m\times n}), $$
and
$$ \Delta=-\bar\Sigma^{\dag}\left[(B+\bar B)^{\top}\Pi+(D+\bar D)^{\top} P(C+\bar C)+(S+\bar S)\right] +\left(I-\bar\Sigma^{\dag}\bar\Sigma\right)\tau, $$
(3.15)
for some \(\tau (\cdot)\in L^{2}(t,T;\mathbb {R}^{m\times n})\). Substituting (3.14) and (4.8) back into (3.8) and (3.10), respectively, we see that (P(·),Π(·)) satisfies the GRE (2.6). In order to show that (P(·),Π(·)) is regular, it remains to prove that
$$\Sigma\equiv R+D^{\top} PD\geqslant0,\qquad\bar\Sigma\equiv R+\bar R+(D+\bar D)^{\top} P(D+\bar D)\geqslant0. $$
For this we take any \(u(\cdot)\in {\mathcal {U}}[t,T]\) and let X(·) be the solution to
$$ \begin{aligned} \left\{\begin{array}{ll} dX(s)=\left\{A(s)X(s)\,+\,\bar A(s)\mathbb{E}[X(s)]\!+B(s)u(s)\,+\,\bar B(s)\mathbb{E}[u(s)]\right\}ds\\ \qquad\qquad~ +\!\left\{C(s)X(s)\,+\,\bar C(s)\mathbb{E}[X(s)]\!+D(s)u(s)\,+\,\bar D(s)\mathbb{E}[u(s)]\right\}dW(s),\qquad s\in[t,T],\\ X(t)=0. \end{array}\right. \end{aligned} $$
(3.16)
Applying Itô’s formula to \(s\mapsto \langle P(s)(X(s)-\mathbb {E}[X(s)]),X(s)-\mathbb {E}[X(s)]\rangle \) and \(s\mapsto \langle \Pi (s)\mathbb {E}[X(s)],\mathbb {E}[X(s)]\rangle \), we have
$$ \begin{aligned} J^{0}(t,0;u(\cdot))\\ &=\mathbb{E}\left\{\vphantom{+{\int_{t}^{T}}\left\langle\left(\begin{array}{cc}Q+\bar Q&(S+\bar S)^{\top}\\S+\bar S&R+\bar R\end{array}\right) \left(\begin{array}{c}\mathbb{E}[X]\\ \mathbb{E}[u]\end{array}\right), \left(\begin{array}{c}\mathbb{E}[X]\\ \mathbb{E}[u]\end{array}\right)\right\rangle ds}\langle G(X(T)\,-\,\mathbb{E}[\!X(T)]),X(T)\,-\,\mathbb{E}[X(T)]\rangle\,+\,\left\langle(G+\!\bar G)\mathbb{E}[X(T)],\mathbb{E}[X(T)]\right\rangle\right.\\ &\qquad~~+{\int_{t}^{T}}\left\langle\left(\begin{array}{ll}Q&S^{\top} \\ S&R\end{array}\right) \left(\begin{array}{c}X-\mathbb{E}[X]\\ u-\mathbb{E}[u]\end{array}\right), \left(\begin{array}{c}X-\mathbb{E}[X]\\ u-\mathbb{E}[u]\end{array}\right)\right\rangle ds\\ &\qquad~~\left.+{\int_{t}^{T}}\left\langle\left(\begin{array}{cc}Q+\bar Q&(S+\bar S)^{\top}\\S+\bar S&R+\bar R\end{array}\right) \left(\begin{array}{c}\mathbb{E}[X]\\ \mathbb{E}[u]\end{array}\right), \left(\begin{array}{c}\mathbb{E}[X]\\ \mathbb{E}[u]\end{array}\right)\right\rangle ds\right\}\\ &=\! \mathbb{E}\!{\int_{t}^{T}}\!\left\{\!\!\vphantom{+{\int_{t}^{T}}\left\langle\left(\begin{array}{cc}Q+\bar Q&(S+\bar S)^{\top}\\S+\bar S&R+\bar R\end{array}\right) \left(\begin{array}{c}\mathbb{E}[X]\\ \mathbb{E}[u]\end{array}\right), \left(\begin{array}{c}\mathbb{E}[X]\\ \mathbb{E}[u]\end{array}\right)\right\rangle ds}\left\langle\dot P(X\,-\,\mathbb{E}[X]),X\,-\,\mathbb{E}[\!X]\right\rangle +\!\left\langle P\left\{A(X\,-\,\mathbb{E}[\!X])\,+\,B(u\,-\,\mathbb{E}[u])\right\},X\,-\,\mathbb{E}[\!X]\right\rangle\right.\\ &\qquad\qquad~+\left\langle P(X\,-\,\mathbb{E}[X]),A(X\,-\,\mathbb{E}[X])\,+\,B(u\,-\,\mathbb{E}[\!u])\right\rangle\\ &\qquad\qquad~ +\langle P\left\{C(X\,-\,\mathbb{E}[X])\,+\,D(u\,-\,\mathbb{E}[u])\,+\,(C\,+\,\bar C)\mathbb{E}[X]\!+(D\,+\,\bar D)\mathbb{E}[u]\right\},\\ &\left.\qquad\qquad\qquad C(X\,-\,\mathbb{E}[X])\,+\,D(u\,-\,\mathbb{E}[u])\,+\,(C\,+\,\bar C)\mathbb{E}[X]\!+(D\,+\,\bar D)\mathbb{E}[u]\rangle\vphantom{+{\int_{t}^{T}}\left\langle\left(\begin{array}{cc}Q+\bar Q&(S+\bar S)^{\top}\\S+\bar S&R+\bar R\end{array}\right) \left(\begin{array}{c}\mathbb{E}[X]\\ \mathbb{E}[u]\end{array}\right), \left(\begin{array}{c}\mathbb{E}[X]\\ \mathbb{E}[u]\end{array}\right)\right\rangle ds}\right\}ds\\ &\quad +\,{\int_{t}^{T}}\left\{\left\langle\dot\Pi\mathbb{E}[X],\mathbb{E}[X]\right\rangle +\left\langle\Pi\left\{(A+\bar A)\mathbb{E}[X]+(B+\bar B)\mathbb{E}[u]\right\},\mathbb{E}[X]\right\rangle\right.\\ &\qquad\qquad~+\left.\left\langle\Pi\mathbb{E}[X],(A+\bar A)\mathbb{E}[X]+(B+\bar B)\mathbb{E}[u]\right\rangle\right\}ds\\ &\quad +\,\mathbb{E}{\int_{t}^{T}}\left\{\langle Q(X-\mathbb{E}[X]),X-\mathbb{E}[X]\rangle +2\langle S(X-\mathbb{E}[X]),u-\mathbb{E}[u]\rangle \right. \\ &\left.\qquad\qquad~+\langle R(u-\mathbb{E}[u]),u-\mathbb{E}[u]\rangle\right\}ds\\ &\quad+\!{\int_{t}^{T}}\!\!\left\{\left\langle(Q\,+\,\bar Q)\mathbb{E}[\!X],\mathbb{E}[\!X]\right\rangle\,+\,2\left\langle(S\,+\,\bar S)\mathbb{E}[\!X],\mathbb{E}[u]\right\rangle \,+\,\left\langle(R\,+\,\bar R)\mathbb{E}[u],\mathbb{E}[u]\right\rangle\right\}ds\\ &=\mathbb{E}{\int_{t}^{T}}\left\{\left\langle\left(\dot P+PA+A^{\top} P+C^{\top} PC+Q\right)(X-\mathbb{E}[X]),X-\mathbb{E}[X]\right\rangle\right.\\ &\qquad\qquad~ +2\left\langle\left(B^{\top} P+D^{\top} PC+S\right)(X-\mathbb{E}[X]),u-\mathbb{E}[u]\right\rangle \\ &\qquad\qquad~+\left.\left\langle\left(R+D^{\top} PD\right)(u-\mathbb{E}[u]),u-\mathbb{E}[u]\right\rangle\right\}ds\\ &+\,{\int_{t}^{T}}\!\!\left\{\left\langle\left[\dot\Pi\,+\,\Pi(A\,+\,\bar A)\,+\,(A\,+\,\bar A)^{\top}\Pi\,+\,(C\,+\,\bar C)^{\top} P(C\,+\,\bar C) \,+\,Q\,+\,\bar Q\right]\mathbb{E}[\!X],\mathbb{E}[\!X]\right\rangle\right.\\ \qquad\qquad~ &+2\left\langle\left[(B+\bar B)^{\top}\Pi+(D+\bar D)^{\top} P(C+\bar C)+S+\bar S\,\right]\mathbb{E}[X],\mathbb{E}[u]\right\rangle\\ &\left.\qquad\qquad~ +\left\langle\left[R+\bar R+(D+\bar D)^{\top} P(D+\bar D)\right]\mathbb{E}[u],\mathbb{E}[u]\right\rangle\right\}ds\\ &=\mathbb{E}{\int_{t}^{T}}\left\{\left\langle(\Theta^{\ast})^{\top}\Sigma\Theta^{\ast}(X\,-\,\mathbb{E}[X]),X\,-\,\mathbb{E}[X]\right\rangle \,-\,2\langle\Sigma\Theta^{\ast}(X-\mathbb{E}[X]),u-\mathbb{E}[u]\rangle \right. \\ &\left.\qquad\qquad~+\langle\Sigma(u-\mathbb{E}[u]),u-\mathbb{E}[u]\rangle{\vphantom{1^{1^{1}}}}\right\}ds\\ \quad &+\,{\int_{t}^{T}}\left\{\left\langle\Delta^{\top}\bar\Sigma\Delta\mathbb{E}[X],\mathbb{E}[X]\right\rangle-2\left\langle\bar\Sigma\Delta\mathbb{E}[X],\mathbb{E}[u]\right\rangle +\left\langle\bar\Sigma\mathbb{E}[u],\mathbb{E}[u]\right\rangle\right\}ds\\ &=\mathbb{E}{\int_{t}^{T}}\left\langle\Sigma\left\{u-\mathbb{E}[u]-\Theta^{\ast}(X-\mathbb{E}[X])\right\},u-\mathbb{E}[u]-\Theta^{\ast}(X-\mathbb{E}[X])\right\rangle ds\\ &\quad+\,{\int_{t}^{T}}\left\langle\bar\Sigma\left(\mathbb{E}[u]-\Delta\mathbb{E}[X]\right),\mathbb{E}[u]-\Delta\mathbb{E}[X]\right\rangle ds. \end{aligned} $$
Since \((\Theta ^{\ast }(\cdot),\bar \Theta ^{\ast }(\cdot),0)\) is an optimal closed-loop strategy of Problem (MF-LQ)0 on [t,T], we have
$$ \begin{array}{ll} \mathbb{E}{\int_{t}^{T}}\left\langle\Sigma\left\{u-\mathbb{E}[u]-\Theta^{\ast}(X-\mathbb{E}[X])\right\},u-\mathbb{E}[u]-\Theta^{\ast}(X-\mathbb{E}[X])\right\rangle ds\\ \quad~+{\int_{t}^{T}}\left\langle\bar\Sigma\left(\mathbb{E}[u]-\Delta\mathbb{E}[X]\right),\mathbb{E}[u]-\Delta\mathbb{E}[X]\right\rangle ds\\ =J^{0}(t,0;u(\cdot))\geqslant J^{0}(t,0;\Theta^{\ast}(\cdot)X^{\ast}(\cdot)+\bar\Theta^{\ast}(\cdot)\mathbb{E}[X^{\ast}(\cdot)])=0,\!\! \qquad\forall u(\cdot)\in{\mathcal{U}}[t,T]. \end{array} $$
(3.17)
Note that for any \(u(\cdot)\in {\mathcal {U}}[t,T]\) of the form
$$u(s)=\Theta^{\ast}(s)X(s)+v(s)W(s),\qquad v(\cdot)\in L^{2}(t,T;\mathbb{R}^{m}), $$
the corresponding solution X(·) of (3.16) satisfies \(\mathbb {E}[X(\cdot)]=0\) and hence \(\mathbb {E}[u(\cdot)]=0\). Then (3.17) yields
$$\begin{array}{lll} 0&\leqslant\mathbb{E}{\int_{t}^{T}}\left\langle\Sigma(s)[u(s)-\Theta^{\ast}(s)X(s)],u(s)-\Theta^{\ast}(s)X(s)\right\rangle ds\\ &=\mathbb{E}{\int_{t}^{T}}\langle\Sigma(s)v(s)W(s),v(s)W(s)\rangle ds\\ &\leqslant T{\int_{t}^{T}}\langle\Sigma(s)v(s),v(s)\rangle ds, \qquad\forall v(\cdot)\in L^{2}(t,T;\mathbb{R}^{m}), \end{array} $$
which implies that \(\Sigma \geqslant 0\). Likewise, for any \(u(\cdot)\in {\mathcal {U}}[t,T]\) of the form
$$u(s)=\Theta^{\ast}(s)\left\{X(s)-\mathbb{E}[X(s)]\right\}+\Delta(s)\mathbb{E}[X(s)]+v(s),\qquad v(\cdot)\in L^{2}(t,T;\mathbb{R}^{m}), $$
the corresponding solution X(·) of (3.16) satisfies
$$u(s)-\mathbb{E}[u(s)]=\Theta^{\ast}(s)\left\{X(s)-\mathbb{E}[X(s)]\right\},\qquad \mathbb{E}[u(s)]-\Delta(s)\mathbb{E}[X(s)]=v(s). $$
Then (3.17) yields
$$\begin{array}{lll} 0&\leqslant&{\int_{t}^{T}}\left\langle\bar\Sigma(s)\left\{\mathbb{E}[u(s)]-\Delta(s)\mathbb{E}[X(s)]\right\},\mathbb{E}[u(s)]-\Delta(s)\mathbb{E}[X(s)]\right\rangle ds\\ &=&{\int_{t}^{T}}\left\langle\bar\Sigma(s)v(s),v(s)\right\rangle ds, \qquad \forall v(\cdot)\in L^{2}(t,T;\mathbb{R}^{m}), \end{array} $$
which implies that \(\bar \Sigma \geqslant 0\). The proof is completed. □