Skip to main content


Stochastic global maximum principle for optimization with recursive utilities


In this paper, we study the recursive stochastic optimal control problems. The control domain does not need to be convex, and the generator of the backward stochastic differential equation can contain z. We obtain the variational equations for backward stochastic differential equations, and then obtain the maximum principle which solves completely Peng’s open problem.


Let \((\Omega,\mathcal {F},P)\) be a complete probability space and let W be a d-dimensional Brownian motion. The filtration \(\{\mathcal {F}_{t}:t\geq 0\}\) is generated by W, i.e.,

$$\mathcal{F}_{t}:=\sigma\{W(s):s\leq t\}\vee\mathcal{N}, $$

where \(\mathcal {N}\) is all P-null sets. Let U be a set in \(\mathbb {R}^{k}\) and T>0 be a given terminal time. Set

$$\begin{aligned} \mathcal{U}[0,T]:&=\{(u(s))_{s\in\lbrack0,T]}:u \,\text{is progressively measurable,}\ u(s)\in U, \text{and } \\ &\quad E\left[\int_{0}^{T} | u(s)|^{\beta}ds\right]<\infty \text{for all }\beta>0\}, \end{aligned} $$

where U is called the control domain and \(\mathcal {U}[0,T]\) is called the set of all admissible controls. In fact, we only need \(E\left [\int _{0} ^{T}|u(s)|^{\beta _{0}}ds\right ]<\infty \) for some β 0>0. For simplicity, we do not explicitly give this β 0 in this paper. We consider the following state equation:

$$ \left\{ \begin{aligned} dx(t)&=b(t,x(t),u(t))dt+\sigma(t,x(t),u(t))dW(t),\\ x(0)&=x_{0}\in\mathbb{R}^{n}, \end{aligned} \right. $$

where \(b:[0,T]\times \mathbb {R}^{n}\times \mathbb {R}^{k}\rightarrow \mathbb {R}^{n}\), \(\sigma :[0,T]\times \mathbb {R}^{n}\times \mathbb {R} ^{k}\rightarrow \mathbb {R}^{n\times d}\). The cost functional is defined by

$$ J(u(\cdot))=E\left[\phi(x(T))+\int_{0}^{T}f(t,x(t),u(t))dt\right], $$

where \(\phi :\mathbb {R}^{n}\rightarrow \mathbb {R}\), \(g:[0,T]\times \mathbb {R} ^{n}\times \mathbb {R}^{k}\rightarrow \mathbb {R}\). The classical stochastic optimal control problem is to minimize J(u(·)) over \(\mathcal {U}[0,T]\). If there exists a \(\bar {u}\in \mathcal {U}[0,T]\) such that

$$J(\bar{u}(\cdot))=\inf_{u\in\mathcal{U}[0,T]}J(u(\cdot)), $$

The process \(\bar {u}\) is called an optimal control. The process \(\bar {x} (\cdot)\), which is the solution of state Eq. (1) corresponding to \(\bar {u}\), is called an optimal trajectory. The maximum principle is to find a necessary condition for the optimality of the control \(\bar {u}\).

A method for deriving the maximum principle is the variational principle. When U is not convex, we use the spike variation method. More precisely, let ε>0 and E ε [0,T] be a Borel set with Borel measure |E ε |=ε, define

$$u^{\varepsilon}(t)=\bar{u}(t)I_{E_{\varepsilon}^{c}}(t)+u(t)I_{E_{\varepsilon }}(t), $$

where \(u\in \mathcal {U}[0,T]\). This u ε is called a spike variation of the optimal control \(\bar {u}\). For deriving the maximum principle, we only need to use E ε =[s,s+ε] for s[0,Tε] and ε>0. The difficulty in the classical stochastic optimal control problem is the variational equation for x(·), which is completely different from that in the deterministic optimal control problem. Peng (1990) was the first to consider the second-order term in the Taylor expansion of the variation and to obtain the maximum principle for the classical stochastic optimal control problem.

Consider the following backward stochastic differential equation (BSDE for short):

$$ y(t)=\phi(x(T))+\int_{t}^{T}f(s,x(s),y(s),z(s),u(s))ds-\int_{t}^{T}z(s)dW(s), $$

where \(\phi :\mathbb {R}^{n}\rightarrow \mathbb {R}\), \(f:[0,T]\times \mathbb {R} ^{n}\times \mathbb {R}\times \mathbb {R}^{d}\times \mathbb {R}^{k}\rightarrow \mathbb {R}\). Pardoux and Peng (1990) were the first to obtain that the BSDE (3) has a unique solution (y(·),z(·)) if f is measurable of linear growth and satisfies a Lipschitz condition in (y,z). Duffie and Epstein (1992) introduced the notion of recursive utilities in continuous time, which is a type of BSDE where f is independent of z. In (El Karoui et al. 1997; 2001), the authors extended the recursive utility to the case where f contains z. The term z can be interpreted as an ambiguity aversion term in the market (see (Chen and Epstein 2002)).

When f is independent of (y,z), it is easy to check that \(y(0)=E[\phi (x(T))+\int _{0}^{T}f(t,x(t),u(t))dt]\). So it is natural to extend the classical stochastic optimal control problem to the recursive case. we consider the control system which formed by Eqs. (1) and (3), and we define the cost functional

$$ J(u(\cdot))=y(0). $$

The recursive stochastic optimal control problem is to minimize J(u(·)) in (4) over \(\mathcal {U}[0,T]\). When the control domain U is convex, the local maximum principle for this problem was studied in (Dokuchaev and Zhou 1999, Ji and Zhou 2006, Peng 1993, Shi and Wu 2006, Wu 1998, Xu 1995) see also the references therein. In this paper, the control domain U is not necessarily convex, and we shall obtain our global maximum principle using the spike variation method.

A direct method for treating this problem would be to consider the second-order terms in the Taylor expansion of the variation for the BSDE (3) as in (Peng 1990). When f depends nonlinearly on z, there are two major difficulties (see (Yong 2010)) one meets: (i) What is the second-order variational equation for the BSDE (3)? it is not similar to the one in (Peng 1990). (ii) How to obtain the second-order adjoint equation which seems to be unexpectedly complicate due to the quadratic form with respect to the variation of z?

Facing on these difficulties, Peng (1998) proposed the following open problem on page 269:

“The corresponding ‘global maximum principle’ for the case where f depends nonlinearly on z is open.”

Recently, a new method for treating this problem is to see z(·) as a control process and the terminal condition y(T)=ϕ(x(T)) as a constraint, and then use the Ekeland variational principle to obtain the maximum principle. This idea was used in (Kohlmann and Zhou 2000; Lim and Zhou 2001) for studying the backward linear-quadratic optimal control problem, and then in (Wu 2013; Yong 2010) for studying the recursive stochastic optimal control problem. But the maximum principle obtained by these method contains unknown parameters.

In this paper, we overcome these both major difficulties one meets in the above direct method by introducing two new adjoint equations in Peng’s open problem. The second-order variational equation for the BSDE (3) and the maximum principle are obtained. The main difference of our variational equations with those in (Peng 1990) consists in the term \(\langle p(t),\delta \sigma (t)\rangle I_{E_{\varepsilon }}(t)\) (see equation (15) in Variational equation for BSDEs and maximum principle for the definition of p(t)) in the variation of z, which is O(ε) for any order expansion of f. So it is not helpful to use the second-order Taylor expansion for treating this term. Moreover, we also obtain the structure of the variation for (y,z) and the variation for x. Based on this, we can get the second-order adjoint equation. Due to the term \(\langle p(t),\delta \sigma (t)\rangle I_{E_{\varepsilon }}(t)\) in the variation of z, our global maximum principle is novel and different from that in (Wu 2013; Yong 2010), which solves completely Peng’s open problem. Furthermore, our maximum principle is stronger than the one in (Wu 2013; Yong 2010) (see Example 1).

The paper is organized as follows. In Preliminaries, we give some basic results and the idea for the variation of BSDEs. The variational equations for BSDEs and the maximum principle are deduced in Variational equation for BSDEs and maximum principle. In Problem with state constraint, we obtain the maximum principle for the control system with state constraint.


The results of this section can be found in (Peng 1990; Yong and Zhou 1999). For simplicity of presentation, we suppose d=1. We need the following assumption:

  1. (A1)

    The functions b=b(t,x,u), σ=σ(t,x,u) are twice continuously differentiable with respect to x; b, b x , b xx , σ, σ x , σ xx are continous in (x,u); b x , b xx , σ x , σ xx are bounded; b, σ are bounded by C(1+|x|+|u|).

Let \(\bar {u}(\cdot)\) be the optimal control for the cost function defined in (2) and let \(\bar {x}(\cdot)\) be the corresponding solution of equation (1). Similarly, we define (x ε(·),u ε(·)). Set

$$ \begin{array}{ll} b(\cdot)=(b^{1}(\cdot),\ldots,b^{n}(\cdot))^{T}, & \sigma(\cdot)=(\sigma^{1}(\cdot),\ldots,\sigma^{n}(\cdot))^{T},\\ b(t)=b(t,\bar{x}(t),\bar{u}(t)), & \delta b(t)=b(t,\bar{x}(t),u(t))-b(t), \end{array} $$

and define similarly b x (t), \(b_{xx}^{i}(t)\), δ b x (t), \(\delta b_{xx}^{i}(t)\), σ(t), σ x (t), \(\sigma _{xx}^{i}(t)\), δ σ(t), δ σ x (t) and \(\delta \sigma _{xx}^{i}(t)\), in, where \(b_{x}=(b_{x_{j}}^{i})_{i,j}\), \(\sigma _{x}=(\sigma _{x_{j} }^{i})_{i,j}\). Let x i (·), i=1, 2, be the solution of the following stochastic differential equation (SDE for short):

$$ \left\{ \begin{array}{l} dx_{1}(t)=b_{x}(t)x_{1}(t)dt+\{\sigma_{x}(t)x_{1}(t)+\delta\sigma (t)I_{E_{\varepsilon}}(t)\}dW(t),\\ x_{1}(0)=0, \end{array} \right. $$


$$ \left\{ \begin{array}{rl} dx_{2}(t)= & \left\{b_{x}(t)x_{2}(t)+\delta b(t)I_{E_{\varepsilon}}(t)+\frac{1} {2}b_{xx}(t)x_{1}(t)x_{1}(t)\right\}dt\\ & +\left\{\sigma_{x}(t)x_{2}(t)+\delta\sigma_{x}(t)x_{1}(t)I_{E_{\varepsilon} }(t)+\frac{1}{2}\sigma_{xx}(t)x_{1}(t)x_{1}(t)\right\}dW(t),\\ x_{2}(0)= & 0, \end{array} \right. $$

respectively, where \(b_{xx}(t)x_{1}(t)x_{1}(t)=(\text {tr}[b_{xx}^{1} (t)x_{1}(t)x_{1}(t)^{T}],\ldots,\text {tr}[b_{xx}^{n}(t)x_{1}(t)x_{1} (t)^{T}])^{T}\) and similarly for σ xx (t)x 1(t)x 1(t).

Lemma 1

Suppose (A1) holds. Then, for any β≥1,

$$ E\left[\sup_{t\in\lbrack0,T]}|x^{\varepsilon}(t)-\bar{x}(t)|^{2\beta} \right]=O\left(\varepsilon^{\beta}\right), $$
$$ E\left[\sup_{t\in\lbrack0,T]}|x_{1}(t)|^{2\beta}\right]=O\left(\varepsilon^{\beta}\right), $$
$$ E\left[\sup_{t\in\lbrack0,T]}|x_{2}(t)|^{2\beta}\right]=O\left(\varepsilon^{2\beta}\right), $$
$$ E\left[\sup_{t\in\lbrack0,T]}|x^{\varepsilon}(t)-\bar{x}(t)-x_{1}(t)|^{2\beta }\right]=O\left(\varepsilon^{2\beta}\right), $$
$$ E\left[\sup_{t\in\lbrack0,T]}|x^{\varepsilon}(t)-\bar{x}(t)-x_{1}(t)-x_{2} (t)|^{2\beta}\right]=o\left(\varepsilon^{2\beta}\right). $$

Moreover, for all \(\phi \in C^{2}(\mathbb {R}^{n})\) such that ϕ xx is bounded, we have the following expansion:

$$ \begin{array}{rl} E[\phi(x^{\varepsilon}(T))]-E[\phi(\bar{x}(T))]= & E[\langle\phi_{x}(\bar {x}(T)),x_{1}(T)+x_{2}(T)\rangle]\\ & +E\left[\frac{1}{2}\langle\phi_{xx}(\bar{x}(T))x_{1}(T),x_{1}(T)\rangle \right]+o(\varepsilon). \end{array} $$

From this result, we can simply write \(x_{1}(t)=O(\sqrt {\varepsilon })\), x 2(t)=O(ε), and

$$ x^{\varepsilon}(t)=\bar{x}(t)+x_{1}(t)+x_{2}(t)+o(\varepsilon). $$

In the following we recall standard estimates of BSDEs (see (Briand et al. 2003) and the references therein).

Lemma 2

Let (Y i ,Z i ), i=1,2, be the solution of the following BSDE

$$Y_{i}(t)=\xi_{i}+\int_{t}^{T}f_{i}(s,Y_{i}(s),Z_{i}(s))ds-\int_{t}^{T} Z_{i}(s)dW(s), $$

where E[|ξ i |β]<, \(f_{i}=f_{i}(s,\omega,y,z):[0,T]\times \Omega \times \mathbb {R}\times \mathbb {R}^{d}\rightarrow \mathbb {R}\) is progressively measurable for each fixed (y,z), Lipschitz in (y,z), and \(E[(\int _{0}^{T}|f_{i}(s,0,0)|ds)^{\beta }]<\infty \) for some β>1. Then there exists a constant C β >0 depending on β, T and the Lipschitz constant such that

$$\begin{aligned} & E\left[\sup_{t\in[0,T]}|Y_{1}(t)-Y_{2}(t)|^{\beta}+\left(\int_{0}^{T} |Z_{1}(s)-Z_{2}(s)|^{2}ds\right)^{\beta/2}\right]\\ & \leq C_{\beta}E\left[|\xi_{1}-\xi_{2}|^{\beta}+\left(\int_{0}^{T}|f_{1} (s,Y_{1}(s),Z_{1}(s))-f_{2}(s,Y_{1}(s),Z_{1}(s))|ds\right)^{\beta}\right]. \end{aligned} $$

In particular, taking ξ 1=0 and f 1=0, we have

$$\begin{aligned} E\left[\sup_{t\in[0,T]}|Y_{2}(t)|^{\beta}+\left(\int_{0}^{T}|Z_{2}(s)|^{2} ds\right)^{\beta/2}\right]\leq C_{\beta}E\left[|\xi_{2}|^{\beta}+\left(\int_{0}^{T}|f_{2} (s,0,0)|ds\right)^{\beta}\right]. \end{aligned} $$

Variational equation for BSDEs and maximum principle

Peng’s open problem

Suppose n=1 and d=1 for simplicity of presentation. The results for the multi-dimensional case will be given in the next subsection.

We consider the control system composed of SDE (1) and BSDE (3). The cost function J(u(·)) is defined in (4). The control problem is to minimize J(u(·)) over \(\mathcal {U}[0,T]\).

We need the following assumption:

  1. (A2)

    The functions f=f(t,x,y,z), ϕ=ϕ(x) are twice continuously differentiable with respect to (x,y,z); f, Df, D 2 f are continuous in (x,y,z,u); Df, D 2 f, ϕ xx are bounded; f is bounded by C(1+|x|+|y|+|z|+|u|).

Here Df is the gradient of f with respect to (x,y,z), D 2 f is the Hessian matrix of f with respect to (x,y,z).

Let \(\bar {u}(\cdot)\) be the optimal control and let \((\bar {x}(\cdot),\bar {y}(\cdot),\bar {z}(\cdot))\) be the corresponding solution of the equations (1) and (3). Similarly, we define (x ε(·),y ε(·),z ε(·),u ε(·)).

In order to obtain the variational equation for BSDE (3), we consider the following two adjoint equations:

$$ \left\{ \begin{array}{l} -dp(t)=F(t)dt-q(t)dW(t),\\ p(T)=\phi_{x}(\bar{x}(T)), \end{array} \right. $$
$$ \left\{ \begin{array}{rl} -dP(t)= & G(t)dt-Q(t)dW(t),\\ P(T)= & \phi_{xx}(\bar{x}(T)), \end{array} \right. $$

where F(t) and G(t) are adapted processes with suitable properties, the will be chosen later.

Applying Itô’s formula to \(p(t)(x_{1}(t)+x_{2}(t))+\frac {1}{2} P(t)(x_{1}(t))^{2}\), we get

$${} \begin{array}{l} \displaystyle p(T)(x_{1}(T)+x_{2}(T))+\frac{1}{2}P(T)(x_{1}(T))^{2} =p(t)(x_{1}(t)+x_{2}(t))+\frac{1}{2}P(t)(x_{1}(t))^{2}\\ \displaystyle +\int_{t}^{T}\left\{A_{1}(s)I_{E_{\varepsilon}}(s)+A_{2} (s)(x_{1}(s)+x_{2}(s))+\frac{1}{2}A_{3}(s)(x_{1}(s))^{2}+A_{4}(s)x_{1} (s)I_{E_{\varepsilon}}(s)\right\}ds\\ \displaystyle +\int_{t}^{T}\left\{p(s)\delta\sigma(s)I_{E_{\varepsilon}} (s)+A_{5}(s)(x_{1}(s)+x_{2}(s))+\frac{1}{2}A_{6}(s)(x_{1}(s))^{2} +A_{7}(s)x_{1}(s)I_{E_{\varepsilon}}(s)\!\right\}dW(s), \end{array} $$


$$ \begin{array}{l} A_{1}(s)=p(s)\delta b(s)+q(s)\delta\sigma(s)+\frac{1}{2}P(s)(\delta \sigma(s))^{2},\\ A_{2}(s)=b_{x}(s)p(s)+\sigma_{x}(s)q(s)-F(s),\\ A_{3}(s)=b_{xx}(s)p(s)+\sigma_{xx}(s)q(s)+2P(s)b_{x}(s)+2Q(s)\sigma_{x}(s)+P(s)(\sigma_{x}(s))^{2}-G(s),\\ A_{4}(s)=q(s)\delta\sigma_{x}(s)+Q(s)\delta\sigma(s)+P(s)\sigma_{x} (s)\delta\sigma(s),\\ A_{5}(s)=\sigma_{x}(s)p(s)+q(s),\\ A_{6}(s)=\sigma_{xx}(s)p(s)+2P(s)\sigma_{x}(s)+Q(s),\\ A_{7}(s)=p(s)\delta\sigma_{x}(s)+P(s)\delta\sigma(s). \end{array} $$

Remark 1

From the above computation, we can see that F(s) and G(s) do not appear in the d W(s)-term.

By Lemma 1, for any β≥2, we have

$$\begin{aligned} E\left[\left|\phi\left(x^{\varepsilon}(T)\right)-\phi\left(\bar{x}(T)\right)-p(T)\left(x_{1}(T)+x_{2}(T)\right)-\frac {1}{2}P(T)\left(x_{1}(T)\right)^{2}\right|^{\beta}\right]=o\left(\varepsilon^{\beta}\right) \end{aligned} $$


$$E\left[\left(\int_{0}^{T}|A_{4}(s)x_{1}(s)I_{E_{\varepsilon}}(s)|ds\right)^{\beta }\right]=o\left(\varepsilon^{\beta}\right). $$

For simplicity of presentation, we write

$$\begin{aligned} \phi(x^{\varepsilon}(T))&=\phi(\bar{x}(T))+p(T)(x_{1}(T)+x_{2}(T))+\frac{1} {2}P(T)(x_{1}(T))^{2} +o(\varepsilon),\\ &\quad \text{}A_{4}(s)x_{1}(s)I_{E_{\varepsilon }}(s)=o(\varepsilon). \end{aligned} $$

In the following, the meaning for o(ε) is in the above sense, similarly for O(ε). If we replace ϕ(x ε(T)) by \(\phi (\bar {x}(T))+p(T)(x_{1}(T)+x_{2}(T))+\frac {1}{2}P(T)(x_{1}(T))^{2} +o(\varepsilon)\) and note (17), then BSDE (3) corresponding to (y ε(·),z ε(·),u ε(·)) can be rewritten as

$$\begin{array}{@{}rcl@{}} \bar{y}^{\varepsilon}(t)&=& \phi(\bar{x} (T))+o(\varepsilon)+\int_{t}^{T}\left\{\vphantom{\frac{1}{2}}f(s,x^{\varepsilon}(s),y^{\varepsilon }(s),z^{\varepsilon}(s),u^{\varepsilon}(s))+A_{1}(s)I_{E_{\varepsilon} }(s)\right.\\&&+\left.A_{2}(s)(x_{1}(s)+x_{2}(s)) +\frac{1}{2}A_{3}(s)(x_{1}(s))^{2}\right\}ds-\int_{t}^{T}\bar {z}^{\varepsilon}(s)dW(s),\quad\qquad \end{array} $$


$$ \begin{aligned} \bar{y}^{\varepsilon}(t)&= y^{\varepsilon}(t)-\left[p(t)(x_{1}(t)+x_{2} (t))+\frac{1}{2}P(t)(x_{1}(t))^{2}\right],\\ \bar{z}^{\varepsilon}(t)&= z^{\varepsilon}(t)-\left\{\vphantom{\frac{1}{2}}p(t)\delta\sigma (t)I_{E_{\varepsilon}}(t)+A_{5}(t)(x_{1}(t)+x_{2}(t))\right.\\ &\quad\left.+\frac{1}{2} A_{6}(t)(x_{1}(t))^{2}+A_{7}(t)x_{1}(t)I_{E_{\varepsilon}}(t)\right\}. \end{aligned} $$

Remark 2

By Lemma 1, for any β≥2, we have \(E\left [|\phi (x^{\varepsilon }(T))-\phi (\bar {x}(T))|^{\beta }\right ]=O(\varepsilon ^{\beta /2})\). So the purpose of this transformation is to simplify the complex terminal condition ϕ(x ε(T)).


$$ \hat{y}^{\varepsilon}(t)=\bar{y}^{\varepsilon}(t)-\bar{y}(t),\text{}\hat {z}^{\varepsilon}(t)=\bar{z}^{\varepsilon}(t)-\bar{z}(t), $$

then we obtain

$${} \begin{aligned} \hat{y}^{\varepsilon}(t)&= o(\varepsilon)+\int_{t}^{T}\left\{\vphantom{\frac{1}{2}}f(s,x^{\varepsilon}(s),y^{\varepsilon}(s),z^{\varepsilon }(s),u^{\varepsilon}(s))-f(s,\bar{x}(s),\bar{y}(s),\bar{z}(s),\bar {u}(s))\right.\\&\quad\left.+A_{1}(s)I_{E_{\varepsilon}}(s) +A_{2}(s)(x_{1}(s)+x_{2}(s))+\frac{1}{2}A_{3}(s)(x_{1} (s))^{2}\right\}ds-\int_{t}^{T}\hat{z}^{\varepsilon}(s)dW(s). \end{aligned} $$

Note that

$$ \begin{array}{l} f(s,x^{\varepsilon}(s),y^{\varepsilon}(s),z^{\varepsilon}(s),u^{\varepsilon }(s))-f(s,\bar{x}(s),\bar{y}(s),\bar{z}(s),\bar{u}(s))\\ =\left[f(s,\bar{x}(s),\bar{y}(s),\bar{z}(s)+p(s)\delta\sigma(s),u(s))-f(s,\bar {x}(s),\bar{y}(s),\bar{z}(s),\bar{u}(s))\right]I_{E_{\varepsilon}}(s)\\ \ \ \ +f(s,x^{\varepsilon}(s),y^{\varepsilon}(s),z^{\varepsilon} (s),u^{\varepsilon}(s))-f(s,\bar{x}(s),\bar{y}(s),\bar{z}(s)+p(s)\delta \sigma(s)I_{E_{\varepsilon}}(s),u^{\varepsilon}(s))\\ =[f(s,\bar{x}(s),\bar{y}(s),\bar{z}(s)+p(s)\delta\sigma(s),u(s))-f(s,\bar {x}(s),\bar{y}(s),\bar{z}(s),\bar{u}(s))]I_{E_{\varepsilon}}(s)\\ \ \ \ +f(s,\bar{x}(s)+x_{1}(s)+x_{2}(s),\bar{y}(s)+\hat{y}^{\varepsilon }(s)+A_{8}(s),\bar{z}(s)+\hat{z}^{\varepsilon}(s)+A_{9}(s),\bar{u}(s))\\ \ \ \ -f(s,\bar{x}(s),\bar{y}(s),\bar{z}(s),\bar{u}(s))+o(\varepsilon), \end{array} $$


$$ \begin{array}{l} A_{8}(s)=p(s)(x_{1}(s)+x_{2}(s))+\frac{1}{2}P(s)(x_{1}(s))^{2},\\ A_{9}(s)=A_{5}(s)(x_{1}(s)+x_{2}(s))+\frac{1}{2}A_{6}(s)(x_{1}(s))^{2} +A_{7}(s)x_{1}(s)I_{E_{\varepsilon}}(s), \end{array} $$

then \((\hat {y}^{\varepsilon }(\cdot),\hat {z}^{\varepsilon }(\cdot))\) can almost be seen as a linear BSDE if F(t) and G(t) are chosen to satisfy the following conditions:

  1. (1)

    F(t) and G(t) are determined by \((\bar {x}(\cdot),\bar {y} (\cdot),\bar {z}(\cdot),\bar {u}(\cdot))\) ;

  2. (2)

    \(f(s,\bar {x}(s)+x_{1}(s)+x_{2}(s),\bar {y}(s)+A_{8}(s),\bar {z}(s)+A_{9}(s),\bar {u}(s))-f(s,\bar {x}(s),\bar {y}(s),\bar {z}(s),\bar {u}(s))+A_{2}(s)(x_{1}(s)+x_{2}(s))+\frac {1}{2}A_{3}(s)(x_{1}(s))^{2} =O(\varepsilon)\) and the O(ε) part does not contain x 1(s) and x 2(s).

For this reason, applying Taylor’s expansion to \(f(s,\bar {x}(s)+x_{1} (s)+x_{2}(s),\bar {y}(s)+A_{8}(s),\bar {z}(s)+A_{9}(s),\bar {u}(s))-f(s,\bar {x}(s),\bar {y}(s),\bar {z}(s),\bar {u}(s))\), F(t) and G(t) must have been chosen as follows:

$$\begin{array}{@{}rcl@{}} F(t)&=& \left[f_{y}(t)+f_{z}(t)\sigma_{x}(t)+b_{x}(t)\right]p(t)+\left[f_{z}(t)+\sigma_{x}(t)\right]q(t)+f_{x}(t),\\ G(t)&= & \left[f_{y}(t)+2f_{z}(t)\sigma_{x}(t)+2b_{x}(t)+(\sigma_{x}(t))^{2} \right]P(t)+\left[f_{z}(t)+2\sigma_{x}(t)\right]Q(t)\\ &&+b_{xx}(t)p(t) +\sigma_{xx}(t)\left[f_{z}(t)p(t)+q(t)\right]+[1,p(t),\sigma_{x}(t)p(t)\\ &&+q(t)]D^{2}f(t)\left[1,p(t),\sigma_{x}(t)p(t)+q(t)\right]^{T}, \end{array} $$

where \(f_{x}(t)=f_{x}(t,\bar {x}(t),\bar {y}(t),\bar {z}(t),\bar {u}(t))\) and similarly for f y (t), f z (t) and D 2 f(t).

Remark 3

If f is independent of (y,z), then the adjoint Eqs. (15) and (16) are the same as those in (Peng 1990).

By the assumptions (A1) and (A2), we get that the adjoint equations (15) and (16) have unique solutions (p(·),q(·)) and (P(·),Q(·)), respectively, and for any β≥2,

$$ \begin{aligned} E\left[\sup_{t\in[0,T]}\left(|p(t)|^{\beta}+|P(t)|^{\beta}\right)+\left(\int_{0} ^{T}\left(|q(s)|^{2}+|Q(s)|^{2}\right)ds\right)^{\beta/2}\right]<\infty. \end{aligned} $$

Consider the following BSDE:

$$\begin{array}{@{}rcl@{}} \hat{y}(t)&= & \int_{t}^{T}\left\{f_{y}(s)\hat{y} (s)+f_{z}(s)\hat{z}(s)+\left[p(s)\delta b(s)+q(s)\delta\sigma(s)+\frac{1} {2}P(s)(\delta\sigma(s))^{2}\right.\right.\\ &&+\left.\left.f(s,\bar{x}(s),\bar{y}(s),\bar{z}(s)+p(s)\delta\sigma (s),u(s))-f(s,\bar{x}(s),\bar{y}(s),\bar{z}(s),\bar{u}(s))\vphantom{\frac{1}{2}}\right]I_{E_{\varepsilon} }(s)\right\}ds\\ && -\int_{t}^{T}\hat{z}(s)dW(s). \end{array} $$

In the following theorem, we will prove that \(\hat {y}^{\varepsilon }(t)-\hat {y}(t)=o(\varepsilon)\).

Theorem 1

Suppose (A1) and (A2) hold. Then, for any β≥2,

$$ \begin{aligned} E\left[\sup_{t\in[0,T]}|\hat{y}^{\varepsilon}(t)|^{2}+\int_{0}^{T}|\hat {z}^{\varepsilon}(t)|^{2}dt\right]=O(\varepsilon^{2}), \end{aligned} $$
$$ \begin{aligned} E\left[\sup_{t\in[0,T]}|\hat{y}^{\varepsilon}(t)|^{\beta}+\left(\int_{0}^{T} |\hat{y}^{\varepsilon}(t)|^{2}dt\right)^{\beta/2}\right]=o(\varepsilon^{\beta/2}), \end{aligned} $$
$$ \begin{aligned} E\left[\sup_{t\in[0,T]}|\hat{y}(t)|^{2}+\int_{0}^{T}|\hat{z}(t)|^{2} dt\right]=O(\varepsilon^{2}), \end{aligned} $$
$$ \begin{aligned} E\left[\sup_{t\in[0,T]}|\hat{y}^{\varepsilon}(t)-\hat{y}(t)|^{2}+\int_{0} ^{T}|\hat{z}^{\varepsilon}(t)-\hat{z}(t)|^{2}dt\right]=o(\varepsilon^{2}). \end{aligned} $$


We first prove (28) and (29). Set

$$ \begin{array}{l} B_{1}(s)=f(s,x^{\varepsilon}(s),y^{\varepsilon}(s),z^{\varepsilon }(s),u^{\varepsilon}(s))-f(s,\bar{x}(s)+x_{1}(s)+x_{2}(s),\bar{y} ^{\varepsilon}(s)\\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ +A_{8}(s),\bar{z}^{\varepsilon}(s)+A_{9}(s),\bar{u}(s)),\\ B_{2}(s)=f(s,\bar{x}(s)+x_{1}(s)+x_{2}(s),\bar{y}^{\varepsilon}(s)+A_{8} (s),\bar{z}^{\varepsilon}(s)+A_{9}(s),\bar{u}(s))\\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ -f(s,\bar{x}(s)+x_{1}(s)+x_{2}(s),\bar{y}(s)+A_{8} (s),\bar{z}(s)+A_{9}(s),\bar{u}(s)),\\ B_{3}(s)=f(s,\bar{x}(s)+x_{1}(s)+x_{2}(s),\bar{y}(s)+A_{8}(s),\bar{z} (s)+A_{9}(s),\bar{u}(s))\\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ -f(s,\bar{x}(s),\bar{y}(s),\bar{z}(s),\bar{u}(s))\\ B_{4}(s)=f_{z}(s)[p(s)\delta\sigma_{x}(s)+P(s)\delta\sigma(s)],\\ B_{5}(s)=[1,p(s),\sigma_{x}(s)p(s)+q(s)], \end{array} $$


$$f(s,x^{\varepsilon}(s),y^{\varepsilon}(s),z^{\varepsilon}(s),u^{\varepsilon }(s))-f(s,\bar{x}(s),\bar{y}(s),\bar{z}(s),\bar{u}(s))=B_{1}(s\!)+B_{2} (s\!)+B_{3}(s\!), $$
$$B_{2}(s)=\tilde{f}_{y}(s)\hat{y}^{\varepsilon}(s)+\tilde{f}_{z}(s)\hat {z}^{\varepsilon}(s), $$


$$\begin{aligned} B_{3}(s) & =f_{x}(s)(x_{1}(s)+x_{2}(s))+f_{y}(s)A_{8}(s)+f_{z}(s)A_{9}(s)\\ & \ \ \ +\frac{1}{2}\left[x_{1}(s)+x_{2}(s),A_{8}(s),A_{9}(s)\right]\tilde{D} ^{2}f(s)[x_{1}(s)+x_{2}(s),A_{8}(s),A_{9}(s)]^{T}, \end{aligned} $$


$$\begin{array}{rl} \displaystyle\tilde{f}_{y}(s)= & \displaystyle\int_{0}^{1}f_{y}(s,\bar {x}(s)+x_{1}(s)+x_{2}(s),\bar{y}(s)+A_{8}(s)+\mu\hat{y}^{\varepsilon}(s),\\ & \displaystyle\bar{z}(s)+A_{9}(s)+\mu\hat{z}^{\varepsilon}(s),\bar{u} (s))d\mu,\\ \displaystyle\tilde{f}_{z}(s)= & \displaystyle\int_{0}^{1}f_{z}(s,\bar {x}(s)+x_{1}(s)+x_{2}(s),\bar{y}(s)+A_{8}(s)+\mu\hat{y}^{\varepsilon}(s),\\ & \displaystyle\bar{z}(s)+A_{9}(s)+\mu\hat{z}^{\varepsilon}(s),\bar{u} (s))d\mu,\\ \displaystyle\tilde{D}^{2}f(s)= & \displaystyle2\int_{0}^{1}\int_{0} ^{1}\lambda D^{2}f(s,\bar{x}(s)+\lambda\mu(x_{1}(s)+x_{2}(s)),\bar {y}(s)+\lambda\mu A_{8}(s),\\ & \displaystyle\bar{z}(s)+\lambda\mu A_{9}(s),\bar{u}(s))d\lambda d\mu. \end{array} $$


$$ \begin{array}{cl} \displaystyle\hat{y}^{\varepsilon}(t)= & \displaystyle o(\varepsilon)+\int_{t}^{T}\{A_{1}(s)I_{E_{\varepsilon}}(s)+B_{1}(s)+\tilde{f}_{y}(s)\hat {y}^{\varepsilon}(s)+\tilde{f}_{z}(s)\hat{z}^{\varepsilon}(s)\\ & \displaystyle+\frac{1}{2}[x_{1}(s)+x_{2}(s),A_{8}(s),A_{9}(s)]\tilde{D} ^{2}f(s)[x_{1}(s)+x_{2}(s),A_{8}(s),A_{9}(s)]^{T}\\ & \displaystyle+B_{4}(s)x_{1}(s)I_{E_{\varepsilon}}(s)-\frac{1}{2} B_{5}(s)D^{2}f(s)B_{5}^{T}(s)(x_{1}(s))^{2}\}ds\\ & \displaystyle-\int_{t}^{T}\hat{z}^{\varepsilon}(s)dW(s), \end{array} $$

By the Lemmas 1 and 2, and the Eqs. (26) and (33), we can easily obtain (28) and (29). By Lemma 2, Eq. (30) is obvious. We now prove (31). Set

$$\tilde{x}^{\varepsilon}(t)=x^{\varepsilon}(t)-\bar{x}(t)-x_{1}(t)-x_{2} (t),\text{}\tilde{y}^{\varepsilon}(t)=\hat{y}^{\varepsilon}(t)-\hat {y}(t),\text{}\tilde{z}^{\varepsilon}(t)=\hat{z}^{\varepsilon}(t)-\hat {z}(t). $$

By Eqs. (33) and (27), we get

$${\kern-1.5pt} \begin{array}{cl} \displaystyle\tilde{y}^{\varepsilon}(t)= & \displaystyle o(\varepsilon)+\int_{t}^{T}\{\tilde{f}_{y}(s)\tilde{y}^{\varepsilon}(s)+\tilde{f} _{z}(s)\tilde{z}^{\varepsilon}(s)+(\tilde{f}_{y}(s)-f_{y}(s))\hat {y}(s)+(\tilde{f}_{z}(s)-f_{z}(s))\hat{z}(s)\\ & \displaystyle+B_{1}(s)-[f(s,\bar{x}(s),\bar{y}(s),\bar{z}(s)+p(s)\delta \sigma(s),u(s))-f(s)]I_{E_{\varepsilon}}(s)\\ & \displaystyle+\frac{1}{2}[x_{1}(s)+x_{2}(s),A_{8}(s),A_{9}(s)]\tilde{D} ^{2}f(s)[x_{1}(s)+x_{2}(s),A_{8}(s),A_{9}(s)]^{T}\\ & \displaystyle+B_{4}(s)x_{1}(s)I_{E_{\varepsilon}}(s)-\frac{1}{2} B_{5}(s)D^{2}f(s)B_{5}^{T}(s)(x_{1}(s))^{2}\}ds\\ & \displaystyle-\int_{t}^{T}\tilde{z}^{\varepsilon}(s)dW(s). \end{array} $$

By the Lemmas 1 and 2, one can easily check that we only need to show that

$$\begin{array}{l} \displaystyle E\left[\left(\int_{0}^{T}|\left(\tilde{f}_{y}(s)-f_{y}(s)\right)\hat{y}(s)+\left(\tilde {f}_{z}(s)-f_{z}(s)\right)\hat{z}(s)|ds\right)^{2}\right]=o(\varepsilon^{2}),\\ \displaystyle E\left[\left(\int_{0}^{T}|B_{5}(s)(\tilde{D}^{2}f(s)-D^{2}f(s))B_{5} ^{T}(s)(x_{1}(s))^{2}|ds\right)^{2}\right]=o(\varepsilon^{2}),\\ \displaystyle E\left[\left(\int_{0}^{T}|B_{1}(s)-\left[f\left(s,\bar{x}(s),\bar{y}(s),\bar {z}(s)+p(s)\delta\sigma(s),u(s)\right)-f(s)\right]I_{E_{\varepsilon}}(s)|ds\right)^{2} \right]=o(\varepsilon^{2}). \end{array} $$

Note that

$$|\!\tilde{f}_{y}\!(s)-f_{y}\!(s)|+|\tilde{f}_{z}\!(s)-f_{z}\!(s)|\leq C\!(|x_{1} \!(s)+x_{2}\!(s)|+|A_{8}\!(s)|+|A_{9}\!(s)|+|\hat{y}^{\varepsilon}\!(s)|+|\hat {z}^{\varepsilon}\!(s)|), $$


$$ \begin{aligned} E\left[\left(\int_{0}^{T}|q(s)x_{1}(s)\hat{z}(s)|ds\right)^{2}\right] & \leq E\left[\sup\nolimits_{s\in \lbrack0,T]}|x_{1}(s)|^{2}\left(\int_{0}^{T}|q(s)|^{2}ds\right)\left(\int_{0}^{T}|\hat {z}(s)|^{2}ds\right)\right]\\ & \leq\left(E\left[\left(\int_{0}^{T}|\hat{z}(s)|^{2}ds\right)^{2}\right]\right)^{1/2}\left(E\left[\sup\nolimits_{s\in\lbrack 0,T]}|x_{1}(s)|^{8}\right]\right)^{1/4}\\ &\quad\times\left(E\left[\left(\int_{0}^{T}|q(s)|^{2}ds\right)^{4}\right]\right)^{1/4}\\ & =o(\varepsilon^{2}), \end{aligned} $$

then we can easily obtain \(E[(\int _{0}^{T}|(\tilde {f}_{y}(s)-f_{y}(s))\hat {y}(s)+(\tilde {f}_{z}(s)-f_{z}(s))\hat {z}(s)|ds)^{2}]=o(\varepsilon ^{2})\). Since D 2 f is bounded, we get that for each β≥2,

$$E\left[\left(\int_{0}^{T}|\left(\tilde{D}^{2}f(s)-D^{2}f(s)\right)||q(s)|^{2}ds\right)^{\beta }\right]\rightarrow0\text{as }\varepsilon\rightarrow0\text{.} $$

Thus we easily deduce \(E[(\int _{0}^{T}|B_{5}(s)(\tilde {D}^{2}f(s)-D^{2} f(s))B_{5}^{T}(s)(x_{1}(s))^{2}|ds)^{2}]=o(\varepsilon ^{2})\). It is easy to verify that

$$\begin{array}{l} |B_{1}(s)-\left[f\left(s,\bar{x}(s),\bar{y}(s),\bar{z}(s)+p(s)\delta\sigma (s),u(s)\right)-f(s)\right]I_{E_{\varepsilon}}(s)|\\ \leq C\left\{|\tilde{x}^{\varepsilon}(s)|+\left[|x_{1}(s)+x_{2}(s)|+|\hat{y} ^{\varepsilon}(s)|+|\hat{z}^{\varepsilon}(s)|+|A_{8}(s)|+|A_{9} (s)|\right]I_{E_{\varepsilon}}(s)\right\}. \end{array} $$


$$\begin{aligned} E\left[\left(\int_{0}^{T}|q(s)x_{1}(s)|I_{E_{\varepsilon}}(s)ds\right)^{2}\right] & \leq E\left[\sup_{s\in[0,T]}|x_{1}(s)|^{2}\int_{E_{\varepsilon}}|q(s)|^{2} ds\right]\varepsilon\\ & \leq\left(E\left[\sup_{s\in[0,T]}|x_{1}(s)|^{4}\right]\right)^{1/2}\left(E\left[\left(\int_{E_{\varepsilon }}|q(s)|^{2}ds\right)^{2}\right]\right)^{1/2}\varepsilon\\ & =o(\varepsilon^{2}), \end{aligned} $$

we can easily obtain \(E[(\int _{0}^{T}|B_{1}(s)-[f(s,\bar {x}(s),\bar {y} (s),\bar {z}(s)+p(s)\delta \sigma (s),u(s))-f(s)]I_{E_{\varepsilon }} (s)|ds)^{2}]=o(\varepsilon ^{2})\). The proof is complete. □

Thus, from (18), (20), (21) and the above theorem, we obtain the following variational equation for BSDE (3):

$$ \begin{array}{rl} y^{\varepsilon}(t)= & \bar{y}(t)+p(t)(x_{1}(t)+x_{2}(t))+\frac{1}{2} P(t)(x_{1}(t))^{2}+\hat{y}(t)+o(\varepsilon),\\ z^{\varepsilon}(t)= & \bar{z}(t)+p(t)\delta\sigma(t)I_{E_{\varepsilon} }(t)+(\sigma_{x}(t)p(t)+q(t))(x_{1}(t)+x_{2}(t))\\ & +\frac{1}{2}(\sigma_{xx}(t)p(t)+2P(t)\sigma_{x}(t)+Q(t))(x_{1}(t))^{2}\\ & +(p(t)\delta\sigma_{x}(t)+P(t)\delta\sigma(t))x_{1}(t)I_{E_{\varepsilon} }(t)+\hat{z}(t)+o(\varepsilon). \end{array} $$

Remark 4

We also give the variational equations for BSDE (3) as in (Peng 1990). Set

$$ y_{1}(t)=p(t)x_{1}(t),\text{}z_{1}(t)=p(t)\delta\sigma(t)I_{E_{\varepsilon} }(t)+[\sigma_{x}(t)p(t)+q(t)]x_{1}(t), $$

it is easy to check that (y 1,z 1) satisfies the following BSDE:

$$ \left\{ \begin{array}{rl} -dy_{1}(t)= & \{f_{x}(t)x_{1}(t)+f_{y}(t)y_{1}(t)+f_{z}(t)z_{1}(t)\\ & -[f_{z}(t)p(t)\delta\sigma(t)+q(t)\delta\sigma(t)]I_{E_{\varepsilon} }(t)\}dt-z_{1}(t)dW(t),\\ y_{1}(T)= & \phi_{x}(\bar{x}(T))x_{1}(T). \end{array} \right. $$


$$ \begin{array}{rl} y_{2}(t)= & p(t)x_{2}(t)+\frac{1}{2}P(t)(x_{1}(t))^{2}+\hat{y}(t),\\ z_{2}(t)= & [\sigma_{x}(t)p(t)+q(t)]x_{2}(t)+[\delta\sigma_{x} (t)p(t)+P(t)\delta\sigma(t)]x_{1}(t)I_{E_{\varepsilon}}(t)\\ & +\frac{1}{2}[\sigma_{xx}(t)p(t)+2P(t)\sigma_{x}(t)+Q(t)](x_{1}(t))^{2} +\hat{z}(t), \end{array} $$

it is easy to verify that

$$ \left\{ \begin{array}{rl} -dy_{2}(t)= & \{f_{x}(t)x_{2}(t)+f_{y}(t)y_{2}(t)+f_{z}(t)z_{2}(t)\\ & +\frac{1}{2}[x_{1}(t),y_{1}(t),z_{1}(t)]D^{2}f(t)[x_{1}(t),y_{1} (t),z_{1}(t)]^{T}\\ & +[q(t)\delta\sigma(t)-\frac{1}{2}f_{zz}(t)(p(t)\delta\sigma(t))^{2} +f(t,\bar{x}(t),\bar{y}(t),\bar{z}(t)+p(t)\delta\sigma(t),u(t))\\ & -f(t,\bar{x}(t),\bar{y}(t),\bar{z}(t),\bar{u}(t))]I_{E_{\varepsilon} }(t)+L(t)x_{1}(t)I_{E_{\varepsilon}}(t)\}dt-z_{2}(t)dW(t),\\ y_{2}(T)= & \phi_{x}(\bar{x}(T))x_{2}(T)+\frac{1}{2}\phi_{xx}(\bar {x}(T))(x_{1}(T))^{2}, \end{array} \right. $$

where \(\phantom {\dot {i}\!}L(t)x_{1}(t)I_{E_{\varepsilon }}(t)=o(\varepsilon)\), so we do not give the explicit formula for L(t). Note the Eqs. (37) and (39), then the adjoint equations for (z 1(t))2 and other terms are essential for x 1(t), x 2(t) and x 1(t)(x 1(t))T, which is solved in (Peng 1990). In order to further explain the difference between the expansions for SDEs and BSDEs, we consider the following equations:

$$ \left\{ \begin{array}{rl} -d\tilde{y}_{1}(t)= & \{f_{x}(t)x_{1}(t)+f_{y}(t)\tilde{y}_{1}(t)+f_{z} (t)\tilde{z}_{1}(t)\}dt-\tilde{z}_{1}(t)dW(t),\\ \tilde{y}_{1}(T)= & \phi_{x}(\bar{x}(T))x_{1}(T), \end{array} \right. $$
$${} \left\{ \begin{array}{rl} -d\tilde{y}_{2}(t)= & \{f_{x}(t)x_{2}(t)+f_{y}(t)\tilde{y}_{2}(t)+f_{z} (t)\tilde{z}_{2}(t)\\ & +\frac{1}{2}[x_{1}(t),\tilde{y}_{1}(t),\tilde{z}_{1}(t)]D^{2}f(t)[x_{1} (t),\tilde{y}_{1}(t),\tilde{z}_{1}(t)]^{T}\\ & +[f(t,\bar{x}(t),\bar{y}(t),\bar{z}(t)+p(t)\delta\sigma(t),u(t\!))-f(t,\bar {x}(t),\bar{y}(t),\bar{z}(t),\bar{u}(t))\\ & -f_{z}(t)p(t)\delta\sigma(t)-\frac{1}{2}f_{zz}(t)(p(t)\delta\sigma (t))^{2}]I_{E_{\varepsilon}}(t)\}dt-\tilde{z}_{2}(t)dW(t),\\ \tilde{y}_{2}(T)= & \phi_{x}(\bar{x}(T))x_{2}(T)+\frac{1}{2}\phi_{xx}(\bar {x}(T))(x_{1}(T))^{2}. \end{array} \right. $$

By Lemma 2, it is easy to show that

$$\begin{aligned} E\left[\sup_{t\in\lbrack0,T]}|y_{1}(t)+y_{2}(t)-\tilde{y}_{1}(t)-\tilde{y} _{2}(t\!)|^{2}+\int_{0}^{T}\!|z_{1}(t)+z_{2}(t)-\tilde{z}_{1}(t)-\tilde{z} _{2}(t)|^{2}dt\right]=o(\varepsilon^{2}). \end{aligned} $$

Thus, by Eq. (36), we get

$$\begin{array}{l} y^{\varepsilon}(t)=\bar{y}(t)+\tilde{y}_{1}(t)+\tilde{y}_{2}(t)+o(\varepsilon),\\ z^{\varepsilon}(t)=\bar{z}(t)+\tilde{z}_{1}(t)+\tilde{z}_{2}(t)+o(\varepsilon). \end{array} $$

The main difference with the variation equation for SDEs is equation (42) which is due to the term \(p(t)\delta \sigma (t)I_{E_{\varepsilon }}(t)\) in the variation of z. If f is independent of z, the variational equations for (y,z) are the same as in (Peng 1990), which is pointed out in (Peng 1998).

Now we consider the maximum principle. From Eq. (36), we get

$$J(u^{\varepsilon}(\cdot))-J(\bar{u}(\cdot))=y^{\varepsilon}(0)-\bar{y} (0)=\hat{y}(0)+o(\varepsilon). $$

Define the following adjoint equation for BSDE (27):

$$\left\{ \begin{array}{l} d\gamma(t)=f_{y}(t)\gamma(t)dt+f_{z}(t)\gamma(t)dW(t),\\ \gamma(0)=1. \end{array} \right. $$

Applying Itô’s formula to \(\gamma (t)\hat {y}(t)\), we can obtain

$$ \begin{array}{rl} \hat{y}(0)= & E[\int_{0}^{T}\gamma(s)[p(s)\delta b(s)+q(s)\delta \sigma(s)+\frac{1}{2}P(s)(\delta\sigma(s))^{2}\\ & +f(s,\bar{x}(s),\bar{y}(s),\bar{z}(s)+p(s)\delta\sigma(s),u(s))-f(s,\bar {x}(s),\bar{y}(s),\bar{z}(s),\bar{u}(s))]I_{E_{\varepsilon}}(s)ds]. \end{array} $$

Note that γ(s)>0, we then define the following function:

$$ \begin{array}{rl} \mathcal{H}(t,x,y,z,u,p,q,P\!)= & pb(t,x,u)+q\sigma(t,x,u)+\frac{1}{2} P(\sigma(t,x,u)-\sigma(t,\bar{x},\bar{u}))^{2}\\ & +f(t,x,y,z+p(\sigma(t,x,u)-\sigma(t,\bar{x},\bar{u})),u), \end{array} $$

where (p,q,P) is defined by the Eqs. (15) and (16). Thus we obtain the following maximum principle.

Theorem 2

Suppose (A1) and (A2) hold. Let \(\bar {u}(\cdot)\) be an optimal control and \((\bar {x}(\cdot),\bar {y}(\cdot),\bar {z}(\cdot))\) be the corresponding solution. Then

$$ \begin{aligned} &\mathcal{H}(t,\bar{x}(t),\bar{y}(t),\bar{z}(t),u,p(t),q(t),P(t))\geq \mathcal{H}(t,\bar{x}(t),\bar{y}(t),\bar{z}(t),\bar{u} (t),p(t),q(t),P(t)),\\ &\forall u\in U,\text{a.e., a.s.,} \end{aligned} $$

where \(\mathcal {H}(\cdot)\) is defined in (44).

Remark 5

If f is independent of (y,z), the above theorem is called Peng’s maximum principle, which was first obtained by Peng in (1990).

If the control domain U is convex, we can get the following corollary which was obtained by Peng in (1993).

Corollary 1

Let the assumptions as in Theorem 2 hold. If U is convex and b, σ, f are continuously differentiable with respect to u, then

$$\langle b_{u}^{T}(t)p(t)+\sigma_{u}^{T}(t)q(t)+f_{z}(t)\sigma_{u} ^{T}(t)p(t)+f_{u}(t),u-\bar{u}(t)\rangle\geq0,\text{}\forall u\in U,a.e.,a.s.. $$

Now we give an example to compare our result with the result in (Wu 2013; Yong 2010).

Example 1

Suppose n=d=k=1. U is a given subset in \(\mathbb {R}\). Consider the following control system:

$$dx(t)=u(t)dW(t),\text{}x(0)=0, $$
$$y(t)=x(T)+\int_{t}^{T}f(z(s))ds-\int_{t}^{T}z(s)dW(s). $$

In this case, our maximum principle is

$$ f(\bar{z}(t)+u-\bar{u}(t))-f(\bar{z}(t))\geq0,\text{}\forall u\in U,a.e.,a.s.. $$

Note that

$$y(t)-\int_{0}^{t}u(s)dW(s)=\int_{t}^{T}f(z(s)-u(s)+u(s))ds-\int_{t} ^{T}(z(s)-u(s))dW(s), $$

then by the comparison theorem of BSDEs, it is easy to check that inequality (46) is a sufficient condition. For the case U={0,1}, f(0)=0, f (0)<0, f(1)>0, f(−1)<0, it is easy to verify that \((\bar {x},\bar {y},\bar {z},\bar {u})=(0,0,0,0)\) satisfies (46), thus \(\bar {u}=0\) is an optimal control. But \(f_{z}(\bar {z}(t))(1-\bar {u}(t))<0\), which implies that \(\bar {u}=0\) is not an optimal control for the case U=[0,1]. The maximum principle in (Yong 2010) is \(f_{z}(\bar {z}(t))(u-\bar {u}(t))\geq 0\), uU, a.e., a.s., which only covers the case when U is convex. The maximum principle in (Wu 2013) contains two unknown parameters.

Remark 6

In (Wu 2013; Yong 2010), the authors consider the control system which consists of SDE (1) and the following state equation:

$$ y(t)=y_{0}-\int_{0}^{t}f(s,x(s),y(s),v(s),u(s))ds+\int_{0}^{t}v(s)dW(s), $$

where the set of all admissible controls

$$\mathcal{\tilde{U}}[0,T]=\{(u,y_{0},v)\in\mathcal{U}[0,T]\times\mathbb{R} \times M^{2}(0,T):y(T)=\phi(x(T))\}. $$

The optimal control problem is to minimize J(u(·),y 0,v(·))=y 0 over \(\mathcal {\tilde {U}}[0,T]\). Obviously, this problem is equivalent to Peng’s problem, i.e., the control system composed of SDE (1) and BSDE (3), the control problem is to minimize J(u(·)) over \(\mathcal {U}[0,T]\). Thus our maximum principle also solves completely this control problem.

Multi-dimensional case

In this subsection, we extend Peng’s problem to the multi-dimensional case, i.e., the functions in BSDE (3) are m-dimensional, \(\phi :\mathbb {R}^{n}\rightarrow \mathbb {R}^{m}\), \(f:[0,T]\times \mathbb {R}^{n} \times \mathbb {R}^{m}\times \mathbb {R}^{m\times d}\times \mathbb {R} ^{k}\rightarrow \mathbb {R}^{m}\). The cost functional is defined by

$$ J(u(\cdot))=h(y(0)), $$

where \(h:\mathbb {R}^{m}\rightarrow \mathbb {R}\). For deriving the variational equation for BSDE (3), we use the following notations:

$$ \begin{array}{l} W(t)=(W^{1}(t),\ldots,W^{d}(t))^{T},\text{}\phi(x)=(\phi^{1}(x),\ldots,\phi^{m}(x))^{T},\\ \sigma(t,x,u)=(\sigma^{ij}(t,x,u))_{1\leq i\leq n,1\leq j\leq d},\text{}\\ \sigma^{j}(t,x,u)=(\sigma^{1j}(t,x,u),\ldots,\sigma^{nj}(t,x,u))^{T},j=1,\ldots,d,\\ f(t,x,y,z,u)=(f^{1}(t,x,y,z,u),\ldots,f^{m}(t,x,y,z,u))^{T},\\ y(t)=(y^{1}(t),\ldots,y^{m}(t))^{T},z(t)=(z^{ij}(t)),i\leq m,j\leq d,\\ z^{j}(t)=(z^{1j}(t),\ldots,z^{mj}(t))^{T},j=1,\ldots,d. \end{array} $$

We keep Assumptions (A1) and (A2) extended in obvious way to the multi-dimensional case n≥1, m≥1, d≥1, and introduce the following adjoint equations: for i=1,…,m,

$$ \left\{ \begin{array}{rl} -dp_{i}(t)= & F_{i}(t)dt-\sum_{j=1}^{d}q_{i}^{j}(t)dW^{j}(t),\\ p_{i}(T)= & \phi_{x}^{i}(\bar{x}(T)), \end{array} \right. $$
$$ \left\{ \begin{array}{rl} -dP_{i}(t)= & G_{i}(t)dt-\sum_{j=1}^{d}Q_{i}^{j}(t)dW^{j}(t),\\ P_{i}(T)= & \phi_{xx}^{i}(\bar{x}(T)), \end{array} \right. $$

where, using the notations

$$ \begin{array}{l} p(t)=\left[p_{1}(t),\ldots,p_{m}(t)\right]_{n\times m}\text{,} q^{j}(t)=\left[q_{1} ^{j}(t),\ldots,q_{m}^{j}(t)\right]_{n\times m},\\ p_{l}(t)=\left(p_{l}^{1}(t),\ldots,p_{l}^{n}(t)\right)^{T},\text{}q_{l}^{j} (t)=\left(q_{l}^{1j}(t),\ldots,q_{l}^{nj}(t)\right)^{T},\text{}\\ b_{xx}^{T}(t)p_{l}(t)=\sum_{i=1}^{n}p_{l}^{i}(t)(b_{xx}^{i}(t))^{T},\text{ }\left(\sigma_{xx}^{j}(t))^{T}p_{l}(t)=\sum_{i=1}^{n}p_{l}^{i}(t)(\sigma_{xx} ^{ij}(t)\right)^{T},\\ (\sigma_{xx}^{j}(t))^{T}q_{l}^{j}(t)=\sum_{i=1}^{n}q_{l}^{ij}(t)(\sigma_{xx}^{ij}(t))^{T},l=1,\ldots,m,\text{}j=1,\ldots,d, \end{array} $$

F i (t) and G i (t) are defined as follows:

$$ \begin{array}{rl} F_{i}(t)= & b_{x}^{T}(t)p_{i}(t)+f_{x}^{i}(t)+\sum_{l=1}^{m}f_{y^{l}} ^{i}(t)p_{l}(t)+\sum_{j=1}^{d}(\sigma_{x}^{j}(t))^{T}q_{i}^{j}(t)\\ & +\sum_{j=1}^{d}\sum_{l=1}^{m}f_{z^{lj}}^{i}(t)[(\sigma_{x}^{j}(t))^{T} p_{l}(t)+q_{l}^{j}(t)],\\ G_{i}(t)= & P_{i}(t)b_{x}(t)+(b_{x}(t))^{T}P_{i}(t)+\sum_{l=1}^{m}f_{y^{l} }^{i}(t)P_{l}(t)+\sum_{j=1}^{d}[Q_{i}^{j}(t)\sigma_{x}^{j}(t)\\ &+(\sigma_{x} ^{j}(t))^{T}Q_{i}^{j}(t) +(\sigma_{xx}^{j}(t))^{T}q_{i}^{j}(t)+(\sigma_{x}^{j}(t))^{T}P_{i} (t)\sigma_{x}^{j}(t)]\\ &+\sum_{j=1}^{d}\sum_{l=1}^{m}[f_{z^{lj}}^{i} (t)P_{l}(t)\sigma_{x}^{j}(t)+f_{z^{lj}}^{i}(t)(\sigma_{x}^{j}(t))^{T} P_{l}(t) +f_{z^{lj}}^{i}(t)Q_{l}^{j}(t)\\ &+f_{z^{lj}}^{i}(t)(\sigma_{xx}^{j} (t))^{T}p_{l}(t)]+b_{xx}^{T}(t)p_{i}(t)+[I_{n\times n},p(t),(\sigma_{x} ^{1}(t))^{T}p(t)\\ &+q^{1}(t),\ldots, (\sigma_{x}^{d}(t))^{T}p(t)+q^{d}(t)]D^{2}f^{i}(t)[I_{n\times n},p(t),(\sigma_{x}^{1}(t))^{T}p(t)\\ &+q^{1}(t),\ldots,(\sigma_{x}^{d} (t))^{T}p(t)+q^{d}(t)]^{T}, \end{array} $$

where D 2 f i is the Hessian matrix of f i with respect to (x,y,z 1,…,z d). Let \(\hat {y}(t)=(\hat {y}^{1}(t),\ldots,\hat {y} ^{m}(t))^{T}\), \(\hat {z}(t)=(\hat {z}^{ij}(t))\) be the solution of the following BSDE:

$$ \begin{array}{rl} \hat{y}(t)= & \int_{t}^{T}[f_{y}(s)\hat{y}(s)+\sum_{j=1}^{d}f_{z^{j}}\! (s)\hat{z}^{j}(s)+\{p^{T}(s)\delta b(s)+\sum_{j=1}^{d}[(q^{j}(s))^{T} \delta\sigma^{j}(s)\\ &+\frac{1}{2}P^{T}(s)\delta\sigma^{j}(s)\delta\sigma^{j}(s)] +f(s,\bar{x}(s),\bar{y}(s),\bar{z}(s)+p^{T}(s)\delta\sigma(s),u)\\ &-f(s,\bar {x}(s),\bar{y}(s),\bar{z}(s),\bar{u}(s))\}I_{E_{\varepsilon}}(s)]ds -\sum_{j=1}^{d}\int_{t}^{T}\hat{z}^{j}(s)dW^{j}(s), \end{array} $$


$$\begin{aligned} P(t)&=[P_{1}(t),\ldots,P_{m}(t)],\text{}P^{T}(s)\delta\sigma^{j} (s)\delta\sigma^{j}(s)\\&=(\langle P_{1}(s)\delta\sigma^{j}(s),\delta\sigma^{j}(s)\rangle,\ldots,\langle P_{m}(s)\delta\sigma^{j}(s),\delta\sigma^{j}(s)\rangle)^{T}. \end{aligned} $$

Similar to the analysis in Theorem 1, we get the following variational principle:

$$ \begin{array}{rl} y^{i;\varepsilon}(t)= & \bar{y}^{i}(t)+\langle p_{i}(t),x_{1}(t)+x_{2} (t)\rangle+\frac{1}{2}\langle P_{i}(t)x_{1}(t),x_{1}(t)\rangle+\hat{y} ^{i}(t)+o(\varepsilon),\\ z^{ij;\varepsilon}(t)= & \bar{z}^{ij}(t)+\langle p_{i}(t),\delta\sigma^{j}(t)\rangle I_{E_{\varepsilon}}(t)+\langle(\sigma_{x}^{j}(t))^{T} p_{i}(t)+q_{i}^{j}(t),x_{1}(t)+x_{2}(t)\rangle\\ & +\langle(\delta\sigma_{x}^{j}(t))^{T}p_{i}(t)+\frac{1}{2}P_{i} (t)\delta\sigma^{j}(t)+\frac{1}{2}P_{i}^{T}(t)\delta\sigma^{j}(t),x_{1} (t)\rangle I_{E_{\varepsilon}}(t)\\ & +\frac{1}{2}\langle\lbrack(\sigma_{xx}^{j}(t))^{T}p_{i}(t)+P_{i} (t)\sigma_{x}^{j}(t)+(\sigma_{x}^{j}(t))^{T}P_{i}(t)+Q_{i}^{j}(t)]x_{1} (t),x_{1}(t)\rangle\\ & +\hat{z}^{ij}(t)+o(\varepsilon),\text{}i=1,\ldots,m,\text{}j=1,\ldots,d. \end{array} $$

Let \(h\in C^{1}(\mathbb {R}^{m})\). Then we obtain

$$J(u^{\varepsilon}(\cdot))-J(\bar{u}(\cdot))=\langle h_{y}(\bar{y}(0)),\hat {y}(0)\rangle+o(\varepsilon). $$

We introduce the following adjoint equation for BSDE (54).

$$ \left\{ \begin{array}{l} d\gamma(t)=f_{y}^{T}(t)\gamma(t)dt+\sum_{j=1}^{d}f_{z^{j}}^{T}(t)\gamma (t)dW^{j}(t),\\ \gamma(0)=h_{y}(\bar{y}(0)). \end{array} \right. $$

Applying Itô’s formula to \(\langle \gamma (t),\hat {y}(t)\rangle \), we obtain the following maximum principle.

Theorem 3

Suppose (A1) and (A2) hold. Let \(\bar {u}(\cdot)\) be an optimal control and \((\bar {x}(\cdot),\bar {y}(\cdot),\bar {z}(\cdot))\) be the corresponding solution. The cost functional is defined in (48) and \(h\in C^{1}(\mathbb {R}^{m})\). Then

$$ \begin{array}{l} \langle\gamma(t),p^{T}(t)\delta b(t)+\sum_{j=1}^{d}[(q^{j}(t))^{T}\delta \sigma^{j}(t)+\frac{1}{2}P^{T}(t)\delta\sigma^{j}(t)\delta\sigma^{j}(t)]\\ +f(t,\bar{x}(t),\bar{y}(t),\bar{z}(t)+p^{T}(t)\delta\sigma(t),u)-f(s,\bar {x}(s),\bar{y}(s),\bar{z}(s),\bar{u}(s))\rangle\\ \geq0,\text{\ }\forall u\in U,\text{a.e., a.s.,} \end{array} $$

where p, q j, P and γ are given in Eqs. (50), (51), (53) and (56).

Problem with state constraint

For simplicity of presentation, suppose d=m=1, the multi-dimensional case can be treated with the same method.

We consider the control system: SDE (1) and BSDE (3). The cost function J(u(·)) is defined in (4). In addition, we consider the following state constraint:

$$ E[\varphi(x(T),y(0))]=0, $$

where \(\varphi :\mathbb {R}^{n}\times \mathbb {R}\rightarrow \mathbb {R}\). We need the following assumption: (A3) The function φ=φ(x,y) is twice continuously differentiable with respect to (x,y); D 2 φ is bounded.

Define all admissible controls as follows:

$$\mathcal{U}_{ad}[0,T]=\{u(\cdot)\in\mathcal{U}[0,T]:E[\varphi (x(T),y(0))]=0\}. $$

The control problem is to minimize J(u(·)) over \(\mathcal {U}_{ad}[0,T]\).

Let \(\bar {u}(\cdot)\in \mathcal {U}_{ad}[0,T]\) be an optimal control and \((\bar {x}(\cdot),\bar {y}(\cdot),\bar {z}(\cdot))\) be the corresponding solution of Eqs. (1) and (3). Similarly, we define (x(·),y(·),z(·),u(·)) for any \(u(\cdot)\in \mathcal {U}[0,T]\). For any ρ>0, define the following cost functional on \(\mathcal {U}[0,T]\):

$$ J_{\rho}(u(\cdot))=\{[(y(0)-\bar{y}(0))+\rho]^{2}+|E[\varphi(x(T),y(0))]|^{2} \}^{1/2}. $$

It is easy to check that

$$\left\{ \begin{array}{l} J_{\rho}(u(\cdot))>0,\text{}\forall u(\cdot)\in\mathcal{U}[0,T],\\ J_{\rho}(\bar{u}(\cdot))=\rho\leq\inf_{u\in\mathcal{U}[0,T]}J_{\rho} (u(\cdot))+\rho. \end{array} \right. $$

In order to use the well-known Ekeland variational principle, we define the following metric on \(\mathcal {U}[0,T]\):

$$d\left(u(\cdot),v(\cdot)\right)=E\left[\int_{0}^{T}I_{\{u\not =v\}}(t,\omega)dt\right]. $$

Suppose that \((\mathcal {U}[0,T],d)\) is a complete space and J ρ (·) is continuous, otherwise we can use the technique in (Tang and Li 1994; Wu 2013) and obtain the same result. Thus, by Ekeland’s variational principle, there exists a \(u_{\rho }(\cdot)\in \mathcal {U}[0,T]\) such that

$$ \begin{array}{l} J_{\rho}(u_{\rho}(\cdot))\leq\rho,\text{}d(u_{\rho}(\cdot),\bar{u} (\cdot))\leq\sqrt{\rho},\\ J_{\rho}(u(\cdot))-J_{\rho}(u_{\rho}(\cdot))+\sqrt{\rho}d(u_{\rho} (\cdot),u(\cdot))\geq0,\text{}\forall u(\cdot)\in\mathcal{U}[0,T]. \end{array} $$

For any ε>0, let E ε [0,T] be a Borel subset with Borel measure |E ε |=ε, and define

$$u_{\rho}^{\varepsilon}(t)=u_{\rho}(t)I_{E_{\varepsilon}^{c}} (t)+u(t)I_{E_{\varepsilon}}(t),\text{for }u(\cdot)\in\mathcal{U}[0,T]. $$

It is easy to check that \(d(u_{\rho }(\cdot),u_{\rho }^{\varepsilon }(\cdot))\leq \varepsilon \). Let (x ρ (·),y ρ (·),z ρ (·)) be the solution corresponding to u ρ (·). Similarly, \((x_{\rho }^{\varepsilon }(\cdot),y_{\rho }^{\varepsilon }(\cdot),z_{\rho }^{\varepsilon }(\cdot),u_{\rho }^{\varepsilon }(\cdot))\) is associated with \(u_{\rho }^{\varepsilon }(\cdot)\). Thus, with (60), we get

$$ \begin{array}{cl} 0 & \leq J_{\rho}(u_{\rho}^{\varepsilon}(\cdot))-J_{\rho}(u_{\rho} (\cdot))+\sqrt{\rho}\varepsilon\\ & \leq\lambda_{\rho}\left[y_{\rho}^{\varepsilon}(0)-y_{\rho}(0)\right]+\mu_{\rho }\left\{E\left[\varphi(x_{\rho}^{\varepsilon}(T),y_{\rho}^{\varepsilon}(0))\right]\right.\\&\left.\quad-E[\varphi (x_{\rho}(T),y_{\rho}(0))]\vphantom{\left[\varphi(x_{\rho}^{\varepsilon}(T),y_{\rho}^{\varepsilon}(0))\right]}\right\}+\sqrt{\rho}\varepsilon+o(\varepsilon), \end{array} $$


$$\lambda_{\rho}=J_{\rho}(u_{\rho}(\cdot))^{-1}[(y_{\rho}(0)-\bar{y} (0))+\rho],\text{}\mu_{\rho}=J_{\rho}(u_{\rho}(\cdot))^{-1}E[\varphi(x_{\rho }(T),y_{\rho}(0))]. $$

Let (p ρ(·),q ρ(·)) and (P ρ(·),Q ρ(·)) be, respectively, the solution of Eq. (15) and (16) with \((\bar {x}(\cdot),\bar {y}(\cdot),\bar {z}(\cdot),\bar {u}(\cdot))\) replaced by (x ρ (·),y ρ (·),z ρ (·),u ρ (·)), and all the coefficients endowed with the superscript ρ. Then, the same computation as for Theorem 1 leads to:

$$ y_{\rho}^{\varepsilon}(0)-y_{\rho}(0)=\hat{y}_{\rho}(0)+o(\varepsilon), $$


$${} \begin{array}{cl} \displaystyle\hat{y}_{\rho}(t)= & \displaystyle\int_{t}^{T}\{f_{y}^{\rho }(s)\hat{y}_{\rho}(s)+f_{z}^{\rho}(s)\hat{z}_{\rho}(s)+[\langle p^{\rho }(s),\delta b^{\rho}(s)\rangle+\langle q^{\rho}(s),\delta\sigma^{\rho }(s)\rangle\\ &\!\!+\frac{1}{2}\langle P^{\rho}(s)\delta\sigma^{\rho}(s),\delta \sigma^{\rho}(s)\rangle +f(s,x_{\rho}(s),y_{\rho}(s),z_{\rho}(s)+\langle p^{\rho }(s),\delta\sigma^{\rho}(s)\rangle,u)\\ &\!\!-f(s,x_{\rho}(s),y_{\rho}(s),z_{\rho }(s),u_{\rho}(s))]I_{E_{\varepsilon}}(s)\}ds \displaystyle-\int_{t}^{T}\hat{z}_{\rho}(s)dW(s). \end{array} $$

Similar to Variational equation for BSDEs and maximum principle, we consider the BSDEs:

$$ \left\{ \begin{array}{rl} -dp_{0}^{\rho}(t)= & [(b_{x}^{\rho}(t))^{T}p_{0}^{\rho}(t)+(\sigma_{x}^{\rho }(t))^{T}q_{0}^{\rho}(t)]dt-q_{0}^{\rho}(t)dW(t),\\ p_{0}(T)= & \mu_{\rho}\varphi_{x}(x_{\rho}(T),y_{\rho}(0)), \end{array} \right. $$
$$ \left\{ \begin{array}{rl} -dP_{0}^{\rho}(t)= & [(b_{x}^{\rho}(t))^{T}P_{0}^{\rho}(t)+P_{0}^{\rho }(t)b_{x}^{\rho}(t)+(\sigma_{x}^{\rho}(t))^{T}P_{0}^{\rho}(t)\sigma_{x}^{\rho }(t)+(\sigma_{x}^{\rho}(t))^{T}Q_{0}^{\rho}(t)\\&+Q_{0}^{\rho}(t)\sigma_{x} ^{\rho}(t) +(b_{xx}^{\rho}(t))^{T}p_{0}^{\rho}(t)+(\sigma_{xx}^{\rho}(t))^{T} q_{0}^{\rho}(t)]dt-Q_{0}^{\rho}(t)dW(t),\\ P_{0}(T)= & \mu_{\rho}\varphi_{xx}(x_{\rho}(T),y_{\rho}(0)). \end{array} \right. $$

Then, using (62)-(65), we can deduce

$$ \begin{array}{l} \mu_{\rho}\{E[\varphi(x_{\rho}^{\varepsilon}(T),y_{\rho}^{\varepsilon }(0))]-E[\varphi(x_{\rho}(T),y_{\rho}(0))]\}\\ =E[\int_{0}^{T}\{\langle p_{0}^{\rho}(s),\delta b^{\rho}(s)\rangle+\langle q_{0}^{\rho}(s),\delta\sigma^{\rho}(s)\rangle+\frac{1}{2}\langle P_{0}^{\rho }(s)\delta\sigma^{\rho}(s),\delta\sigma^{\rho}(s)\rangle\}I_{E_{\varepsilon} }(s)ds]\\ \text{\ }+\mu_{\rho}E[\varphi_{y}(x_{\rho}(T),y_{\rho}(0))]\hat{y}_{\rho }(0)+o(\varepsilon). \end{array} $$

Let us now introduce the following adjoint equation for BSDE (63):

$$\left\{ \begin{array}{l} d\gamma^{\rho}(t)=f_{y}^{\rho}(t)\gamma^{\rho}(t)dt+f_{z}^{\rho} (t)\gamma^{\rho}(t)dW(t),\\ \gamma^{\rho}(0)=\lambda_{\rho}+\mu_{\rho}E[\varphi_{y}(x_{\rho}(T),y_{\rho }(0))]. \end{array} \right. $$

Then we get

$$ \begin{array}{l} \{\lambda_{\rho}+\mu_{\rho}E[\varphi_{y}(x_{\rho}(T),y_{\rho}(0))]\}\hat {y}_{\rho}(0)\\ =E[\int_{0}^{T}\gamma^{\rho}(s)\{\langle p^{\rho}(s),\delta b^{\rho} (s)\rangle+\langle q^{\rho}(s),\delta\sigma^{\rho}(s)\rangle+\frac{1} {2}\langle P^{\rho}(s)\delta\sigma^{\rho}(s),\delta\sigma^{\rho}(s)\rangle\\ \text{\ }+f(s,x_{\rho}(s),y_{\rho}(s),z_{\rho}(s)+\langle p^{\rho} (s),\delta\sigma^{\rho}(s)\rangle,u)-f(s,x_{\rho}(s),y_{\rho}(s),z_{\rho }(s),u_{\rho}(s))\}I_{E_{\varepsilon}}(s)ds], \end{array} $$

and for the Hamiltonian

$$ \begin{array}{l} \mathcal{H}(t,x,y,z,u,x^{\prime},u^{\prime},p_{0},q_{0},P_{0},p,q,P,\gamma)\\ =\langle p_{0}+\gamma p,b(t,x,u)\rangle+\langle q_{0}+\gamma q,\sigma (t,x,u)\rangle\\ \text{\ }+\frac{1}{2}\langle(P_{0}+\gamma P)(\sigma(t,x,u)-\sigma (t,x^{\prime},u^{\prime})),\sigma(t,x,u)-\sigma(t,x^{\prime},u^{\prime })\rangle\\ \text{\ }+\gamma(t)f(t,x,y,z+\langle p,\sigma(t,x,u)-\sigma(t,x^{\prime },u^{\prime})\rangle,u), \end{array} $$

it follows from (61), (62), (66) and (67) that

$$\begin{aligned} 0 \leq E[\int_{0}^{T}\{&\mathcal{H}(t,x_{\rho}(t),y_{\rho}(t),z_{\rho }(t),u,x_{\rho}(t),u_{\rho}(t),p_{0}^{\rho}(t),q_{0}^{\rho}(t),P_{0}^{\rho }(t),p^{\rho}(t),q^{\rho}(t),\\& P^{\rho}(t),\gamma^{\rho}(t))-\mathcal{H}(t,x_{\rho}(t),y_{\rho}(t),z_{\rho}(t),u_{\rho }(t),x_{\rho}(t),u_{\rho}(t),p_{0}^{\rho}(t),q_{0}^{\rho}(t),\\ &P_{0}^{\rho }(t),p^{\rho}(t),q^{\rho}(t),P^{\rho}(t),\gamma^{\rho}(t))\}I_{E_{\varepsilon }}(t)dt] +\sqrt{\rho}\varepsilon+o(\varepsilon). \end{aligned} $$

Thus, from the arbitrariness of ε>0 and E ε [0,T] with |E ε |=ε, we obtain

$$\begin{array}{l} \mathcal{H}(t,x_{\rho}(t),y_{\rho}(t),z_{\rho}(t),u,x_{\rho}(t),u_{\rho }(t),p_{0}^{\rho}(t),q_{0}^{\rho}(t),P_{0}^{\rho}(t),p^{\rho}(t),q^{\rho }(t),P^{\rho}(t),\gamma^{\rho}(t))\\ \geq\mathcal{H}(t,x_{\rho}(t),y_{\rho}(t),z_{\rho}(t),u_{\rho}(t),x_{\rho }(t),u_{\rho}(t),p_{0}^{\rho}(t),q_{0}^{\rho}(t),P_{0}^{\rho}(t),p^{\rho }(t),q^{\rho}(t),P^{\rho}(t),\gamma^{\rho}(t))\\ \text{\ }-\sqrt{\rho},\text{for all }u\in U,\text{a.e., a.s..} \end{array} $$

Recall (59) and the definition of λ ρ and μ ρ , we have |λ ρ |2+|μ ρ |2=1. Thus there exists a subsequence of (λ ρ ,μ ρ ) which converges to (λ,μ) with |λ|2+|μ|2=1, as ρ→0. As \(d(u_{\rho }(\cdot),\bar {u}(\cdot))\leq \sqrt {\rho }\), we can choose a sub-subsequence satisfying

$$\begin{array}{l} (x_{\rho}(\cdot),y_{\rho}(\cdot),z_{\rho}(\cdot),u_{\rho}(\cdot),p_{0}^{\rho }(\cdot),q_{0}^{\rho}(\cdot),P_{0}^{\rho}(\cdot),p^{\rho}(\cdot),q^{\rho }(\cdot),P^{\rho}(\cdot),\gamma^{\rho}(\cdot))\rightarrow\\ (\bar{x}(\cdot),\bar{y}(\cdot),\bar{z}(\cdot),\bar{u}(\cdot),p_{0} (\cdot),q_{0}(\cdot),P_{0}(\cdot),p(\cdot),q(\cdot),P(\cdot),\gamma (\cdot)),\text{a.e., a.s.} \end{array} $$

Here (p 0(·),q 0(·)) and (P 0(·),Q 0(·)) is, respectively, the solution of Eqs. (15) and (16),

$$ \left\{ \begin{array}{rl} -dp_{0}(t)= & [(b_{x}(t))^{T}p_{0}(t)+(\sigma_{x}(t))^{T}q_{0}(t)]dt-q_{0} (t)dW(t),\\ p_{0}(T)= & \mu\varphi_{x}(\bar{x}(T),\bar{y}(0)), \end{array} \right. $$
$$ \left\{ \begin{array}{rl} -dP_{0}(t)= & [(b_{x}(t))^{T}P_{0}(t)+P_{0}(t)b_{x}(t)+(\sigma_{x} (t))^{T}P_{0}(t)\sigma_{x}(t)+(\sigma_{x}(t))^{T}Q_{0}(t)\\ &+Q_{0}(t)\sigma_{x}(t) +(b_{xx}(t))^{T}p_{0}(t)+(\sigma_{xx}(t))^{T}q_{0}(t)]dt-Q_{0}(t)dW(t),\\ P_{0}(T)= & \mu\varphi_{xx}(\bar{x}(T),\bar{y}(0)), \end{array} \right. $$


$$ \left\{ \begin{array}{l} d\gamma(t)=f_{y}(t)\gamma(t)dt+f_{z}(t)\gamma(t)dW(t),\\ \gamma(0)=\lambda+\mu E[\varphi_{y}(\bar{x}(T),\bar{y}(0))]. \end{array} \right. $$

Thus we obtain the following theorem.

Theorem 4

Suppose (A1), (A2) and (A3) hold. Let \(\bar {u}(\cdot)\) be an optimal control with state constraint (58) and \((\bar {x}(\cdot),\bar {y} (\cdot),\bar {z}(\cdot))\) be the corresponding solution. Then there exist two contants λ and μ with |λ|2+|μ|2=1 such that

$$\begin{array}{l} \mathcal{H}(t,\bar{x}(t),\bar{y}(t),\bar{z}(t),u,\bar{x}(t),\bar{u} (t),p_{0}(t),q_{0}(t),P_{0}(t),p(t),q(t),P(t),\gamma(t))\\ \geq\mathcal{H}(t,\bar{x}(t),\bar{y}(t),\bar{z}(t),\bar{u},\bar{x}(t),\bar {u}(t),p_{0}(t),q_{0}(t),P_{0}(t),p(t),q(t),P(t),\gamma(t)),\\ \text{ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ for all}u\in U,\text{a.e., a.s.,} \end{array} $$

where \(\mathcal {H}(\cdot)\), (p(·),q(·)), (P(·),Q(·)), (p 0(·),q 0(·)), (P 0(·),Q 0(·)) and γ(·) are defined in (68), (15), (16), (69), (70) and (71).


  1. Briand, PH, Delyon, B, Hu, Y, Pardoux, E, Stoica, L: L p solutions of backward stochastic differential equations. Stochastic Process. Appl. 108, 109–129 (2003).

  2. Chen, Z, Epstein, L: Ambiguity, risk, and asset returns in continuous time. Econometrica. 70, 1403–1443 (2002).

  3. Dokuchaev, M, Zhou, XY: Stochastic controls with terminal contingent conditions. J. Math. Anal. Appl. 238, 143–165 (1999).

  4. Duffie, D, Epstein, L: Stochastic differential utility. Econometrica. 60, 353–394 (1992).

  5. El Karoui, N, Peng, S, Quenez, MC: Backward stochastic differential equations in finance. Math. Finance. 7, 1–71 (1997).

  6. El Karoui, N, Peng, S, Quenez, MC: A dynamic maximum priciple for the optimization of recursive utilities under constraints. Ann. Appl. Probab. 11, 664–693 (2001).

  7. Ji, S, Zhou, XY: A maximum principle for stochastic optimal control with terminal state constrains, and its applications. Comm. Inf. Syst. 6, 321–337 (2006).

  8. Kohlmann, M, Zhou, XY: Relationship between backward stochastic differential equations and stochastic controls: a linear-quadratic approach. SIAM J. Control Optim. 38, 1392–1407 (2000).

  9. Lim, A, Zhou, XY: Linear-quadratic control of backward stochastic differential equations. SIAM J. Control Optim. 40, 450–474 (2001).

  10. Pardoux, E, Peng, S: Adapted Solutions of Backward Stochastic Equations. Systerm Control Lett. 14, 55–61 (1990).

  11. Peng, S: A general stochastic maximum principle for optimal control problems. SIAM J. Control Optim. 28, 966–979 (1990).

  12. Peng, S: Backward stochastic differential equations and applications to optimal control. Appl. Math. Optim. 27, 125–144 (1993).

  13. Peng, S: Open problems on backward stochastic differential equations. In: Chen, S, Li, X, Yong, J, Zhou, XY (eds.)Control of distributed parameter and stocastic systems, pp. 265–273, Boston: Kluwer Acad. Pub. (1998).

  14. Shi, J, Wu, Z: The maximum principle for fully coupled forward-backward stochastic control system. Acta Automat. Sinica. 32, 161–169 (2006).

  15. Tang, S, Li, X: Necessary conditions for optimal control of stochastic systems with random jumps. SIAM J. Control Optim. 32, 1447–1475 (1994).

  16. Wu, Z: Maximum principle for optimal control problem of fully coupled forward-backward stochastic systems. Syst. Sci. Math. Sci. 11, 249–259 (1998).

  17. Wu, Z: A general maximum principle for optimal control of forward-backward stochastic systems. Automatica. 49, 1473–1480 (2013).

  18. Xu, W: Stochastic maximum principle for optimal control problem of forward and backward system. J. Austral. Math. Soc. Ser. B. 37, 172–185 (1995).

  19. Yong, J: Optimality variational principle for controlled forward-backward stochastic differential equations with mixed initial-terminal conditions. SIAM J. Control Optim. 48, 4119–4156 (2010).

  20. Yong, J, Zhou, XY: Stochastic controls: Hamiltonian systems and HJB equations. Springer-Verlag, New York (1999).

Download references


I would like to thank Professor S. Peng and S. Ji for many helpful discussions and valuable comments. I also would like to thank the referees for careful reading and valuable comments.

Research supported by NSF (No. 11671231, 11201262 and 10921101), Shandong Province (No.BS2013SF020 and ZR2014AP005), Young Scholars Program of Shandong University and the 111 Project (No. B12023).

Authors’ contributions

All authors read and approved the final manuscript.

Competing interests

The author declares that he has no competing interests.

Author information

Correspondence to Mingshang Hu.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (, which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Hu, M. Stochastic global maximum principle for optimization with recursive utilities. Probab Uncertain Quant Risk 2, 1 (2017).

Download citation


  • Backward stochastic differential equations
  • Recursive stochastic optimal control
  • Maximum principle
  • Variational equation

AMS subject classifications

  • 93E20
  • 60H10
  • 49K45