 Research
 Open Access
 Published:
Stochastic global maximum principle for optimization with recursive utilities
Probability, Uncertainty and Quantitative Riskvolume 2, Article number: 1 (2017)
Abstract
In this paper, we study the recursive stochastic optimal control problems. The control domain does not need to be convex, and the generator of the backward stochastic differential equation can contain z. We obtain the variational equations for backward stochastic differential equations, and then obtain the maximum principle which solves completely Peng’s open problem.
Introduction
Let \((\Omega,\mathcal {F},P)\) be a complete probability space and let W be a ddimensional Brownian motion. The filtration \(\{\mathcal {F}_{t}:t\geq 0\}\) is generated by W, i.e.,
where \(\mathcal {N}\) is all Pnull sets. Let U be a set in \(\mathbb {R}^{k}\) and T>0 be a given terminal time. Set
where U is called the control domain and \(\mathcal {U}[0,T]\) is called the set of all admissible controls. In fact, we only need \(E\left [\int _{0} ^{T}u(s)^{\beta _{0}}ds\right ]<\infty \) for some β _{0}>0. For simplicity, we do not explicitly give this β _{0} in this paper. We consider the following state equation:
where \(b:[0,T]\times \mathbb {R}^{n}\times \mathbb {R}^{k}\rightarrow \mathbb {R}^{n}\), \(\sigma :[0,T]\times \mathbb {R}^{n}\times \mathbb {R} ^{k}\rightarrow \mathbb {R}^{n\times d}\). The cost functional is defined by
where \(\phi :\mathbb {R}^{n}\rightarrow \mathbb {R}\), \(g:[0,T]\times \mathbb {R} ^{n}\times \mathbb {R}^{k}\rightarrow \mathbb {R}\). The classical stochastic optimal control problem is to minimize J(u(·)) over \(\mathcal {U}[0,T]\). If there exists a \(\bar {u}\in \mathcal {U}[0,T]\) such that
The process \(\bar {u}\) is called an optimal control. The process \(\bar {x} (\cdot)\), which is the solution of state Eq. (1) corresponding to \(\bar {u}\), is called an optimal trajectory. The maximum principle is to find a necessary condition for the optimality of the control \(\bar {u}\).
A method for deriving the maximum principle is the variational principle. When U is not convex, we use the spike variation method. More precisely, let ε>0 and E _{ ε }⊂[0,T] be a Borel set with Borel measure E _{ ε }=ε, define
where \(u\in \mathcal {U}[0,T]\). This u ^{ε} is called a spike variation of the optimal control \(\bar {u}\). For deriving the maximum principle, we only need to use E _{ ε }=[s,s+ε] for s∈[0,T−ε] and ε>0. The difficulty in the classical stochastic optimal control problem is the variational equation for x(·), which is completely different from that in the deterministic optimal control problem. Peng (1990) was the first to consider the secondorder term in the Taylor expansion of the variation and to obtain the maximum principle for the classical stochastic optimal control problem.
Consider the following backward stochastic differential equation (BSDE for short):
where \(\phi :\mathbb {R}^{n}\rightarrow \mathbb {R}\), \(f:[0,T]\times \mathbb {R} ^{n}\times \mathbb {R}\times \mathbb {R}^{d}\times \mathbb {R}^{k}\rightarrow \mathbb {R}\). Pardoux and Peng (1990) were the first to obtain that the BSDE (3) has a unique solution (y(·),z(·)) if f is measurable of linear growth and satisfies a Lipschitz condition in (y,z). Duffie and Epstein (1992) introduced the notion of recursive utilities in continuous time, which is a type of BSDE where f is independent of z. In (El Karoui et al. 1997; 2001), the authors extended the recursive utility to the case where f contains z. The term z can be interpreted as an ambiguity aversion term in the market (see (Chen and Epstein 2002)).
When f is independent of (y,z), it is easy to check that \(y(0)=E[\phi (x(T))+\int _{0}^{T}f(t,x(t),u(t))dt]\). So it is natural to extend the classical stochastic optimal control problem to the recursive case. we consider the control system which formed by Eqs. (1) and (3), and we define the cost functional
The recursive stochastic optimal control problem is to minimize J(u(·)) in (4) over \(\mathcal {U}[0,T]\). When the control domain U is convex, the local maximum principle for this problem was studied in (Dokuchaev and Zhou 1999, Ji and Zhou 2006, Peng 1993, Shi and Wu 2006, Wu 1998, Xu 1995) see also the references therein. In this paper, the control domain U is not necessarily convex, and we shall obtain our global maximum principle using the spike variation method.
A direct method for treating this problem would be to consider the secondorder terms in the Taylor expansion of the variation for the BSDE (3) as in (Peng 1990). When f depends nonlinearly on z, there are two major difficulties (see (Yong 2010)) one meets: (i) What is the secondorder variational equation for the BSDE (3)? it is not similar to the one in (Peng 1990). (ii) How to obtain the secondorder adjoint equation which seems to be unexpectedly complicate due to the quadratic form with respect to the variation of z?
Facing on these difficulties, Peng (1998) proposed the following open problem on page 269:
“The corresponding ‘global maximum principle’ for the case where f depends nonlinearly on z is open.”
Recently, a new method for treating this problem is to see z(·) as a control process and the terminal condition y(T)=ϕ(x(T)) as a constraint, and then use the Ekeland variational principle to obtain the maximum principle. This idea was used in (Kohlmann and Zhou 2000; Lim and Zhou 2001) for studying the backward linearquadratic optimal control problem, and then in (Wu 2013; Yong 2010) for studying the recursive stochastic optimal control problem. But the maximum principle obtained by these method contains unknown parameters.
In this paper, we overcome these both major difficulties one meets in the above direct method by introducing two new adjoint equations in Peng’s open problem. The secondorder variational equation for the BSDE (3) and the maximum principle are obtained. The main difference of our variational equations with those in (Peng 1990) consists in the term \(\langle p(t),\delta \sigma (t)\rangle I_{E_{\varepsilon }}(t)\) (see equation (15) in Variational equation for BSDEs and maximum principle for the definition of p(t)) in the variation of z, which is O(ε) for any order expansion of f. So it is not helpful to use the secondorder Taylor expansion for treating this term. Moreover, we also obtain the structure of the variation for (y,z) and the variation for x. Based on this, we can get the secondorder adjoint equation. Due to the term \(\langle p(t),\delta \sigma (t)\rangle I_{E_{\varepsilon }}(t)\) in the variation of z, our global maximum principle is novel and different from that in (Wu 2013; Yong 2010), which solves completely Peng’s open problem. Furthermore, our maximum principle is stronger than the one in (Wu 2013; Yong 2010) (see Example 1).
The paper is organized as follows. In Preliminaries, we give some basic results and the idea for the variation of BSDEs. The variational equations for BSDEs and the maximum principle are deduced in Variational equation for BSDEs and maximum principle. In Problem with state constraint, we obtain the maximum principle for the control system with state constraint.
Preliminaries
The results of this section can be found in (Peng 1990; Yong and Zhou 1999). For simplicity of presentation, we suppose d=1. We need the following assumption:

(A1)
The functions b=b(t,x,u), σ=σ(t,x,u) are twice continuously differentiable with respect to x; b, b _{ x }, b _{ xx }, σ, σ _{ x }, σ _{ xx } are continous in (x,u); b _{ x }, b _{ xx }, σ _{ x }, σ _{ xx } are bounded; b, σ are bounded by C(1+x+u).
Let \(\bar {u}(\cdot)\) be the optimal control for the cost function defined in (2) and let \(\bar {x}(\cdot)\) be the corresponding solution of equation (1). Similarly, we define (x ^{ε}(·),u ^{ε}(·)). Set
and define similarly b _{ x }(t), \(b_{xx}^{i}(t)\), δ b _{ x }(t), \(\delta b_{xx}^{i}(t)\), σ(t), σ _{ x }(t), \(\sigma _{xx}^{i}(t)\), δ σ(t), δ σ _{ x }(t) and \(\delta \sigma _{xx}^{i}(t)\), i≤n, where \(b_{x}=(b_{x_{j}}^{i})_{i,j}\), \(\sigma _{x}=(\sigma _{x_{j} }^{i})_{i,j}\). Let x _{ i }(·), i=1, 2, be the solution of the following stochastic differential equation (SDE for short):
and
respectively, where \(b_{xx}(t)x_{1}(t)x_{1}(t)=(\text {tr}[b_{xx}^{1} (t)x_{1}(t)x_{1}(t)^{T}],\ldots,\text {tr}[b_{xx}^{n}(t)x_{1}(t)x_{1} (t)^{T}])^{T}\) and similarly for σ _{ xx }(t)x _{1}(t)x _{1}(t).
Lemma 1
Suppose (A1) holds. Then, for any β≥1,
Moreover, for all \(\phi \in C^{2}(\mathbb {R}^{n})\) such that ϕ _{ xx } is bounded, we have the following expansion:
From this result, we can simply write \(x_{1}(t)=O(\sqrt {\varepsilon })\), x _{2}(t)=O(ε), and
In the following we recall standard estimates of BSDEs (see (Briand et al. 2003) and the references therein).
Lemma 2
Let (Y _{ i },Z _{ i }), i=1,2, be the solution of the following BSDE
where E[ξ _{ i }^{β}]<∞, \(f_{i}=f_{i}(s,\omega,y,z):[0,T]\times \Omega \times \mathbb {R}\times \mathbb {R}^{d}\rightarrow \mathbb {R}\) is progressively measurable for each fixed (y,z), Lipschitz in (y,z), and \(E[(\int _{0}^{T}f_{i}(s,0,0)ds)^{\beta }]<\infty \) for some β>1. Then there exists a constant C _{ β }>0 depending on β, T and the Lipschitz constant such that
In particular, taking ξ _{1}=0 and f _{1}=0, we have
Variational equation for BSDEs and maximum principle
Peng’s open problem
Suppose n=1 and d=1 for simplicity of presentation. The results for the multidimensional case will be given in the next subsection.
We consider the control system composed of SDE (1) and BSDE (3). The cost function J(u(·)) is defined in (4). The control problem is to minimize J(u(·)) over \(\mathcal {U}[0,T]\).
We need the following assumption:

(A2)
The functions f=f(t,x,y,z), ϕ=ϕ(x) are twice continuously differentiable with respect to (x,y,z); f, Df, D ^{2} f are continuous in (x,y,z,u); Df, D ^{2} f, ϕ _{ xx } are bounded; f is bounded by C(1+x+y+z+u).
Here Df is the gradient of f with respect to (x,y,z), D ^{2} f is the Hessian matrix of f with respect to (x,y,z).
Let \(\bar {u}(\cdot)\) be the optimal control and let \((\bar {x}(\cdot),\bar {y}(\cdot),\bar {z}(\cdot))\) be the corresponding solution of the equations (1) and (3). Similarly, we define (x ^{ε}(·),y ^{ε}(·),z ^{ε}(·),u ^{ε}(·)).
In order to obtain the variational equation for BSDE (3), we consider the following two adjoint equations:
where F(t) and G(t) are adapted processes with suitable properties, the will be chosen later.
Applying Itô’s formula to \(p(t)(x_{1}(t)+x_{2}(t))+\frac {1}{2} P(t)(x_{1}(t))^{2}\), we get
where
Remark 1
From the above computation, we can see that F(s) and G(s) do not appear in the d W(s)term.
By Lemma 1, for any β≥2, we have
and
For simplicity of presentation, we write
In the following, the meaning for o(ε) is in the above sense, similarly for O(ε). If we replace ϕ(x ^{ε}(T)) by \(\phi (\bar {x}(T))+p(T)(x_{1}(T)+x_{2}(T))+\frac {1}{2}P(T)(x_{1}(T))^{2} +o(\varepsilon)\) and note (17), then BSDE (3) corresponding to (y ^{ε}(·),z ^{ε}(·),u ^{ε}(·)) can be rewritten as
where
Remark 2
By Lemma 1, for any β≥2, we have \(E\left [\phi (x^{\varepsilon }(T))\phi (\bar {x}(T))^{\beta }\right ]=O(\varepsilon ^{\beta /2})\). So the purpose of this transformation is to simplify the complex terminal condition ϕ(x ^{ε}(T)).
Set
then we obtain
Note that
where
then \((\hat {y}^{\varepsilon }(\cdot),\hat {z}^{\varepsilon }(\cdot))\) can almost be seen as a linear BSDE if F(t) and G(t) are chosen to satisfy the following conditions:

(1)
F(t) and G(t) are determined by \((\bar {x}(\cdot),\bar {y} (\cdot),\bar {z}(\cdot),\bar {u}(\cdot))\) ;

(2)
\(f(s,\bar {x}(s)+x_{1}(s)+x_{2}(s),\bar {y}(s)+A_{8}(s),\bar {z}(s)+A_{9}(s),\bar {u}(s))f(s,\bar {x}(s),\bar {y}(s),\bar {z}(s),\bar {u}(s))+A_{2}(s)(x_{1}(s)+x_{2}(s))+\frac {1}{2}A_{3}(s)(x_{1}(s))^{2} =O(\varepsilon)\) and the O(ε) part does not contain x _{1}(s) and x _{2}(s).
For this reason, applying Taylor’s expansion to \(f(s,\bar {x}(s)+x_{1} (s)+x_{2}(s),\bar {y}(s)+A_{8}(s),\bar {z}(s)+A_{9}(s),\bar {u}(s))f(s,\bar {x}(s),\bar {y}(s),\bar {z}(s),\bar {u}(s))\), F(t) and G(t) must have been chosen as follows:
where \(f_{x}(t)=f_{x}(t,\bar {x}(t),\bar {y}(t),\bar {z}(t),\bar {u}(t))\) and similarly for f _{ y }(t), f _{ z }(t) and D ^{2} f(t).
Remark 3
If f is independent of (y,z), then the adjoint Eqs. (15) and (16) are the same as those in (Peng 1990).
By the assumptions (A1) and (A2), we get that the adjoint equations (15) and (16) have unique solutions (p(·),q(·)) and (P(·),Q(·)), respectively, and for any β≥2,
Consider the following BSDE:
In the following theorem, we will prove that \(\hat {y}^{\varepsilon }(t)\hat {y}(t)=o(\varepsilon)\).
Theorem 1
Suppose (A1) and (A2) hold. Then, for any β≥2,
Proof
We first prove (28) and (29). Set
then
and
where
Thus
By the Lemmas 1 and 2, and the Eqs. (26) and (33), we can easily obtain (28) and (29). By Lemma 2, Eq. (30) is obvious. We now prove (31). Set
By the Lemmas 1 and 2, one can easily check that we only need to show that
Note that
and
then we can easily obtain \(E[(\int _{0}^{T}(\tilde {f}_{y}(s)f_{y}(s))\hat {y}(s)+(\tilde {f}_{z}(s)f_{z}(s))\hat {z}(s)ds)^{2}]=o(\varepsilon ^{2})\). Since D ^{2} f is bounded, we get that for each β≥2,
Thus we easily deduce \(E[(\int _{0}^{T}B_{5}(s)(\tilde {D}^{2}f(s)D^{2} f(s))B_{5}^{T}(s)(x_{1}(s))^{2}ds)^{2}]=o(\varepsilon ^{2})\). It is easy to verify that
Since
we can easily obtain \(E[(\int _{0}^{T}B_{1}(s)[f(s,\bar {x}(s),\bar {y} (s),\bar {z}(s)+p(s)\delta \sigma (s),u(s))f(s)]I_{E_{\varepsilon }} (s)ds)^{2}]=o(\varepsilon ^{2})\). The proof is complete. □
Thus, from (18), (20), (21) and the above theorem, we obtain the following variational equation for BSDE (3):
Remark 4
We also give the variational equations for BSDE (3) as in (Peng 1990). Set
it is easy to check that (y _{1},z _{1}) satisfies the following BSDE:
Set
it is easy to verify that
where \(\phantom {\dot {i}\!}L(t)x_{1}(t)I_{E_{\varepsilon }}(t)=o(\varepsilon)\), so we do not give the explicit formula for L(t). Note the Eqs. (37) and (39), then the adjoint equations for (z _{1}(t))^{2} and other terms are essential for x _{1}(t), x _{2}(t) and x _{1}(t)(x _{1}(t))^{T}, which is solved in (Peng 1990). In order to further explain the difference between the expansions for SDEs and BSDEs, we consider the following equations:
By Lemma 2, it is easy to show that
Thus, by Eq. (36), we get
The main difference with the variation equation for SDEs is equation (42) which is due to the term \(p(t)\delta \sigma (t)I_{E_{\varepsilon }}(t)\) in the variation of z. If f is independent of z, the variational equations for (y,z) are the same as in (Peng 1990), which is pointed out in (Peng 1998).
Now we consider the maximum principle. From Eq. (36), we get
Define the following adjoint equation for BSDE (27):
Applying Itô’s formula to \(\gamma (t)\hat {y}(t)\), we can obtain
Note that γ(s)>0, we then define the following function:
where (p,q,P) is defined by the Eqs. (15) and (16). Thus we obtain the following maximum principle.
Theorem 2
Suppose (A1) and (A2) hold. Let \(\bar {u}(\cdot)\) be an optimal control and \((\bar {x}(\cdot),\bar {y}(\cdot),\bar {z}(\cdot))\) be the corresponding solution. Then
where \(\mathcal {H}(\cdot)\) is defined in (44).
Remark 5
If f is independent of (y,z), the above theorem is called Peng’s maximum principle, which was first obtained by Peng in (1990).
If the control domain U is convex, we can get the following corollary which was obtained by Peng in (1993).
Corollary 1
Let the assumptions as in Theorem 2 hold. If U is convex and b, σ, f are continuously differentiable with respect to u, then
Now we give an example to compare our result with the result in (Wu 2013; Yong 2010).
Example 1
Suppose n=d=k=1. U is a given subset in \(\mathbb {R}\). Consider the following control system:
In this case, our maximum principle is
Note that
then by the comparison theorem of BSDEs, it is easy to check that inequality (46) is a sufficient condition. For the case U={0,1}, f(0)=0, f ^{′}(0)<0, f(1)>0, f(−1)<0, it is easy to verify that \((\bar {x},\bar {y},\bar {z},\bar {u})=(0,0,0,0)\) satisfies (46), thus \(\bar {u}=0\) is an optimal control. But \(f_{z}(\bar {z}(t))(1\bar {u}(t))<0\), which implies that \(\bar {u}=0\) is not an optimal control for the case U=[0,1]. The maximum principle in (Yong 2010) is \(f_{z}(\bar {z}(t))(u\bar {u}(t))\geq 0\), ∀u∈U, a.e., a.s., which only covers the case when U is convex. The maximum principle in (Wu 2013) contains two unknown parameters.
Remark 6
In (Wu 2013; Yong 2010), the authors consider the control system which consists of SDE (1) and the following state equation:
where the set of all admissible controls
The optimal control problem is to minimize J(u(·),y _{0},v(·))=y _{0} over \(\mathcal {\tilde {U}}[0,T]\). Obviously, this problem is equivalent to Peng’s problem, i.e., the control system composed of SDE (1) and BSDE (3), the control problem is to minimize J(u(·)) over \(\mathcal {U}[0,T]\). Thus our maximum principle also solves completely this control problem.
Multidimensional case
In this subsection, we extend Peng’s problem to the multidimensional case, i.e., the functions in BSDE (3) are mdimensional, \(\phi :\mathbb {R}^{n}\rightarrow \mathbb {R}^{m}\), \(f:[0,T]\times \mathbb {R}^{n} \times \mathbb {R}^{m}\times \mathbb {R}^{m\times d}\times \mathbb {R} ^{k}\rightarrow \mathbb {R}^{m}\). The cost functional is defined by
where \(h:\mathbb {R}^{m}\rightarrow \mathbb {R}\). For deriving the variational equation for BSDE (3), we use the following notations:
We keep Assumptions (A1) and (A2) extended in obvious way to the multidimensional case n≥1, m≥1, d≥1, and introduce the following adjoint equations: for i=1,…,m,
where, using the notations
F _{ i }(t) and G _{ i }(t) are defined as follows:
where D ^{2} f ^{i} is the Hessian matrix of f ^{i} with respect to (x,y,z ^{1},…,z ^{d}). Let \(\hat {y}(t)=(\hat {y}^{1}(t),\ldots,\hat {y} ^{m}(t))^{T}\), \(\hat {z}(t)=(\hat {z}^{ij}(t))\) be the solution of the following BSDE:
where
Similar to the analysis in Theorem 1, we get the following variational principle:
Let \(h\in C^{1}(\mathbb {R}^{m})\). Then we obtain
We introduce the following adjoint equation for BSDE (54).
Applying Itô’s formula to \(\langle \gamma (t),\hat {y}(t)\rangle \), we obtain the following maximum principle.
Theorem 3
Suppose (A1) and (A2) hold. Let \(\bar {u}(\cdot)\) be an optimal control and \((\bar {x}(\cdot),\bar {y}(\cdot),\bar {z}(\cdot))\) be the corresponding solution. The cost functional is defined in (48) and \(h\in C^{1}(\mathbb {R}^{m})\). Then
where p, q ^{j}, P and γ are given in Eqs. (50), (51), (53) and (56).
Problem with state constraint
For simplicity of presentation, suppose d=m=1, the multidimensional case can be treated with the same method.
We consider the control system: SDE (1) and BSDE (3). The cost function J(u(·)) is defined in (4). In addition, we consider the following state constraint:
where \(\varphi :\mathbb {R}^{n}\times \mathbb {R}\rightarrow \mathbb {R}\). We need the following assumption: (A3) The function φ=φ(x,y) is twice continuously differentiable with respect to (x,y); D ^{2} φ is bounded.
Define all admissible controls as follows:
The control problem is to minimize J(u(·)) over \(\mathcal {U}_{ad}[0,T]\).
Let \(\bar {u}(\cdot)\in \mathcal {U}_{ad}[0,T]\) be an optimal control and \((\bar {x}(\cdot),\bar {y}(\cdot),\bar {z}(\cdot))\) be the corresponding solution of Eqs. (1) and (3). Similarly, we define (x(·),y(·),z(·),u(·)) for any \(u(\cdot)\in \mathcal {U}[0,T]\). For any ρ>0, define the following cost functional on \(\mathcal {U}[0,T]\):
It is easy to check that
In order to use the wellknown Ekeland variational principle, we define the following metric on \(\mathcal {U}[0,T]\):
Suppose that \((\mathcal {U}[0,T],d)\) is a complete space and J _{ ρ }(·) is continuous, otherwise we can use the technique in (Tang and Li 1994; Wu 2013) and obtain the same result. Thus, by Ekeland’s variational principle, there exists a \(u_{\rho }(\cdot)\in \mathcal {U}[0,T]\) such that
For any ε>0, let E _{ ε }⊂[0,T] be a Borel subset with Borel measure E _{ ε }=ε, and define
It is easy to check that \(d(u_{\rho }(\cdot),u_{\rho }^{\varepsilon }(\cdot))\leq \varepsilon \). Let (x _{ ρ }(·),y _{ ρ }(·),z _{ ρ }(·)) be the solution corresponding to u _{ ρ }(·). Similarly, \((x_{\rho }^{\varepsilon }(\cdot),y_{\rho }^{\varepsilon }(\cdot),z_{\rho }^{\varepsilon }(\cdot),u_{\rho }^{\varepsilon }(\cdot))\) is associated with \(u_{\rho }^{\varepsilon }(\cdot)\). Thus, with (60), we get
where
Let (p ^{ρ}(·),q ^{ρ}(·)) and (P ^{ρ}(·),Q ^{ρ}(·)) be, respectively, the solution of Eq. (15) and (16) with \((\bar {x}(\cdot),\bar {y}(\cdot),\bar {z}(\cdot),\bar {u}(\cdot))\) replaced by (x _{ ρ }(·),y _{ ρ }(·),z _{ ρ }(·),u _{ ρ }(·)), and all the coefficients endowed with the superscript ρ. Then, the same computation as for Theorem 1 leads to:
where
Similar to Variational equation for BSDEs and maximum principle, we consider the BSDEs:
Then, using (62)(65), we can deduce
Let us now introduce the following adjoint equation for BSDE (63):
Then we get
and for the Hamiltonian
it follows from (61), (62), (66) and (67) that
Thus, from the arbitrariness of ε>0 and E _{ ε }⊂[0,T] with E _{ ε }=ε, we obtain
Recall (59) and the definition of λ _{ ρ } and μ _{ ρ }, we have λ _{ ρ }^{2}+μ _{ ρ }^{2}=1. Thus there exists a subsequence of (λ _{ ρ },μ _{ ρ }) which converges to (λ,μ) with λ^{2}+μ^{2}=1, as ρ→0. As \(d(u_{\rho }(\cdot),\bar {u}(\cdot))\leq \sqrt {\rho }\), we can choose a subsubsequence satisfying
Here (p _{0}(·),q _{0}(·)) and (P _{0}(·),Q _{0}(·)) is, respectively, the solution of Eqs. (15) and (16),
and
Thus we obtain the following theorem.
Theorem 4
Suppose (A1), (A2) and (A3) hold. Let \(\bar {u}(\cdot)\) be an optimal control with state constraint (58) and \((\bar {x}(\cdot),\bar {y} (\cdot),\bar {z}(\cdot))\) be the corresponding solution. Then there exist two contants λ and μ with λ^{2}+μ^{2}=1 such that
where \(\mathcal {H}(\cdot)\), (p(·),q(·)), (P(·),Q(·)), (p _{0}(·),q _{0}(·)), (P _{0}(·),Q _{0}(·)) and γ(·) are defined in (68), (15), (16), (69), (70) and (71).
References
Briand, PH, Delyon, B, Hu, Y, Pardoux, E, Stoica, L: L ^{p} solutions of backward stochastic differential equations. Stochastic Process. Appl. 108, 109–129 (2003).
Chen, Z, Epstein, L: Ambiguity, risk, and asset returns in continuous time. Econometrica. 70, 1403–1443 (2002).
Dokuchaev, M, Zhou, XY: Stochastic controls with terminal contingent conditions. J. Math. Anal. Appl. 238, 143–165 (1999).
Duffie, D, Epstein, L: Stochastic differential utility. Econometrica. 60, 353–394 (1992).
El Karoui, N, Peng, S, Quenez, MC: Backward stochastic differential equations in finance. Math. Finance. 7, 1–71 (1997).
El Karoui, N, Peng, S, Quenez, MC: A dynamic maximum priciple for the optimization of recursive utilities under constraints. Ann. Appl. Probab. 11, 664–693 (2001).
Ji, S, Zhou, XY: A maximum principle for stochastic optimal control with terminal state constrains, and its applications. Comm. Inf. Syst. 6, 321–337 (2006).
Kohlmann, M, Zhou, XY: Relationship between backward stochastic differential equations and stochastic controls: a linearquadratic approach. SIAM J. Control Optim. 38, 1392–1407 (2000).
Lim, A, Zhou, XY: Linearquadratic control of backward stochastic differential equations. SIAM J. Control Optim. 40, 450–474 (2001).
Pardoux, E, Peng, S: Adapted Solutions of Backward Stochastic Equations. Systerm Control Lett. 14, 55–61 (1990).
Peng, S: A general stochastic maximum principle for optimal control problems. SIAM J. Control Optim. 28, 966–979 (1990).
Peng, S: Backward stochastic differential equations and applications to optimal control. Appl. Math. Optim. 27, 125–144 (1993).
Peng, S: Open problems on backward stochastic differential equations. In: Chen, S, Li, X, Yong, J, Zhou, XY (eds.)Control of distributed parameter and stocastic systems, pp. 265–273, Boston: Kluwer Acad. Pub. (1998).
Shi, J, Wu, Z: The maximum principle for fully coupled forwardbackward stochastic control system. Acta Automat. Sinica. 32, 161–169 (2006).
Tang, S, Li, X: Necessary conditions for optimal control of stochastic systems with random jumps. SIAM J. Control Optim. 32, 1447–1475 (1994).
Wu, Z: Maximum principle for optimal control problem of fully coupled forwardbackward stochastic systems. Syst. Sci. Math. Sci. 11, 249–259 (1998).
Wu, Z: A general maximum principle for optimal control of forwardbackward stochastic systems. Automatica. 49, 1473–1480 (2013).
Xu, W: Stochastic maximum principle for optimal control problem of forward and backward system. J. Austral. Math. Soc. Ser. B. 37, 172–185 (1995).
Yong, J: Optimality variational principle for controlled forwardbackward stochastic differential equations with mixed initialterminal conditions. SIAM J. Control Optim. 48, 4119–4156 (2010).
Yong, J, Zhou, XY: Stochastic controls: Hamiltonian systems and HJB equations. SpringerVerlag, New York (1999).
Acknowledgments
I would like to thank Professor S. Peng and S. Ji for many helpful discussions and valuable comments. I also would like to thank the referees for careful reading and valuable comments.
Research supported by NSF (No. 11671231, 11201262 and 10921101), Shandong Province (No.BS2013SF020 and ZR2014AP005), Young Scholars Program of Shandong University and the 111 Project (No. B12023).
Authors’ contributions
All authors read and approved the final manuscript.
Competing interests
The author declares that he has no competing interests.
Author information
Rights and permissions
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.
About this article
Received
Accepted
Published
DOI
Keywords
 Backward stochastic differential equations
 Recursive stochastic optimal control
 Maximum principle
 Variational equation
AMS subject classifications
 93E20
 60H10
 49K45