Cours Controle Marrakech

7/27/2019 Cours Controle Marrakech

1/77

Lectures on

Stochastic control and

applications in finance

Huyen PHAM

University Paris Diderot, LPMA

Institut Universitaire de France and CREST-ENSAE

[email protected]

http://www.proba.jussieu.fr/pageperso/pham/pham.html

Autumn school on Stochastic control problems for FBSDEs and Applications

Marrakech, December 1-11, 2010


2/77

Stochastic Control and applications in finance

Abstract. The aim of these lectures is to present an introduction to stochastic

control, a classsical topic in applied mathematics, which has known important devel-

opments over the last years inspired especially by problems in mathematical finance.We give an overview of the main methods and results in this area.

We first present the standard approach by dynamic programming equation and

verification, and point out the limits of this method. We then move on to the vis-

cosity solutions approach: it requires more theory and technique, but provides the

general mathematical tool for dealing with stochastic control in a Markovian context.

The last lecture is devoted to an introduction to the theory of Backward stochasticdifferential equations (BSDEs), which has emerged as a major research topic with

significant contributions in relation with stochastic control beyond the Markovian

framework. The various methods presented in these lectures will be illustrated by

several applications arising in economics and finance.

Lecture 1 : Classical approach to stochastic control problemLecture 2 : Viscosity solutions and stochastic control

Lecture 3 : BSDEs and stochastic control


3/77

References for these lectures:

H. Pham (2009): Continuous-time stochastic control and optimization with fi-

nancial applications, Series SMAP, Springer.

I. Kharroubi, J. Ma, H. Pham and J. Zhang (2010): Backward stochastic dif-

ferential equations with constrained jumps and quasi-variational inequalities,

Annals of Probability, Vol. 38, 794-840.


4/77

Lecture 1 : Classical approach to stochastic control problem

Introduction

Controlled diffusion processes

Dynamic Programming Principle (DPP)

Hamilton-Jacobi-Bellman (HJB) equation

Verification theorem

Applications : Merton portfolio selection (CRRA utility functions and general

utility functions by duality approach), Merton portfolio/ consumption choice

Some other classes of stochastic control


5/77

I. Introduction

Basic structure of stochastic control problem

Dynamic system in an uncertain environment:

- filtered probability space (, F,F = (Ft),P): uncertainty and information

- state variables X = (Xt): F-adapted stochastic process representing the evolu-

tion of the quantitative variables describing the system

Control: a process = (t) whose value is decided at time t in function of

the available information Ft, and which can influence the dynamics of the state

process X.

Performance/criterion: optimize over controls a functional J(X, ), e.g.

J(X, ) = E

T0

f(Xt, t)dt + g(XT)

on a finite horizon

or

J(X, ) = E

0

etf(Xt, t)dt on an infinite horizon Various and numerous applications in economics and finance

In parallel, problems in mathematical finance new developments in the theory

of stochastic control


6/77

Solving a stochastic control problem

Basic goal: find the optimal control(which achieves the optimum of the objective

functional) if it exists and the value function (the optimal objective functional)

Tractable characterization of the value function and optimal control

- if possible, explicit solutions

- otherwise: qualitative description and quantitative results via numerical solu-

tions

Mathematical tools

Dynamic programming principle and stochastic calculus

- PDE characterization in a Markovian context

- BSDE in general

Stochastic control is a topic at the interface between probability, stochastic analysis

and PDE.


7/77

II. Controlled diffusion processes

Dynamics of the state variables in Rn:

dXs = b(Xs, s)ds + (Xs, s)dWs, (1)

W d-dimensional Brownian motion on (, F,F = (Ft),P).

- The control = (t) is an F-adapted process, valued in A subset of Rm, and

satisfying some integrability conditions and/or state constraints A set ofadmissible

controls.

- Given A, (t, x) [0, T] Rn, we denote by Xt,x = Xt,x, the solution to (1)

starting from x at t.

Performance criterion (on finite horizon)

Given a function f from Rn A into R, and a function g from Rn into R, we define

the objective functional:

J(t,x,) = E Tt f(Xt,xs , s)ds + g(Xt,xT ) , (t, x) [0, T] Rn, A,and the value function:

v(t, x) = supA

J(t,x,).

A is an optimal control if: v(t, x) = J(t,x, ).

A process in the form s = a(s, Xt,xs ) for some measurable function a from

[0, T] Rn into A is called Markovian or feedback control.


8/77

III. Dynamic programming principle

Bellmans principle of optimality

An optimal policy has the property that whatever the initial state and initial decision

are, the remaining decisions must constitute an optimal policy with regard to the

state resulting from the first decision

(See Bellman, 1957, Ch. III.3)

Mathematical formulation of the Bellmans principle or Dynamic Pro-

gramming Principle (DPP)

The usual version of the DPP is written as

v(t, x) = s u pA

E

t

f(Xt,xs , s)ds + v(, Xt,x )

, (2)

for any stopping time Tt,T (set of stopping times valued in [t, T]).


9/77

Stronger version of the DPP

In a stronger and useful version of the DPP, may actually depend on in (2). This

means:

v(t, x) = s u pA supTt,TE

t f(Xt,xs , s)ds + v(, X

t,x

)= sup

AinfTt,T

E

t


.

(i) For all A and Tt,T:

v(t, x) E

t


.

(ii) For all > 0, there exists A such that for all Tt,T

v(t, x) E

t


.


10/77

Proof of the DPP. (First part)

1. Given A, we have by pathwise uniqueness of the flow of the SDE for X, the

Markovian structure

Xt,xs = X

,Xt,x

s , s ,

for any Tt,T. By the law of iterated conditional expectation, we then get

J(t,x,) = E

t

f(s, Xt,xs , s)ds + J(, Xt,x , )

,

Since J(. , . ,) v, and is arbitrary in Tt,T, this implies

J(t,x,) infTt,T

E

t

f(s, Xt,xs , s)ds + v(, Xt,x )

sup

AinfTt,T

E

t


.

=

v(t, x) supA

infTt,T

E

t


. (3)


11/77

Proof of the DPP. (Second part)

2. Fix some arbitrary control A and Tt,T. By definition of the value functions,

for any > 0 and , there exists , A, which is an -optimal control for

v((), Xt,x()()), i.e.

v((), Xt,x()

()) J((), Xt,x()

(), ,). (4)

Let us now define the process

s() =

s(), s [0, ()]

,s (), s [(), T].

Delicate measurability questions! By measurable selection results, one can showthat is F-adapted, and so A.

By using again the law of iterated conditional expectation, and from (4):

v(t, x) J(t,x, ) = E

t

f(s, Xt,xs , s)ds + J(, Xt,x ,

)

E

tf(s, Xt,xs , s)ds + v(, Xt,x ) .

Since , and are arbitrary, this implies

v(t, x) supA

supTt,T

E

t


.


12/77

IV. Hamilton-Jacobi-Bellman (HJB) equation

The HJB equation is the infinitesimal version of the dynamic programming principle:

it describes the local behavior of the value function when we send the stopping time

in the DPP (2) to t. The HJB equation is also called dynamic programming equation.

Formal derivation of HJB

We assume that the value function is smooth enough to apply It os formula, and we

postpone integrability questions.

For any A, and a controlled process Xt,x, apply Itos formula to v(s, Xt,xs

)

between s = t and s = t + h:

v(t + h, Xt,xt+h) = v(t, x) +

t+ht

v

t+ Lsv

(s, Xt,xs )ds + (local) martingale,

where for a A, La is the second-order operator associated to the diffusion X with

constant control a:

Lav = b(x, a).Dxv +1

2tr (x, a)(x, a)D2xv .


13/77


14/77

V. Verification approach

Theorem

Let w be a function in C1,2([0, T] Rn), solution to the HJB equation:

wt

(t, x) + supaA

Law(t, x) + f(x, a) = 0, (t, x) [0, T) Rn,w(T, x) = g(x), x Rn.

(and satisfying eventually additional growth conditions related to f and g). Suppose

there exists a measurable function a(t, x), (t, x) [0, T) Rn, valued in A, attaining

the supremum in HJB i.e.

supaA

Law(t, x) + f(x, a) = La(t,x)w(t, x) + f(x, a(t, x)),such that the SDE

dXs = b(Xs, a(s, Xs))ds + (Xs, a(s, Xs))dWs

admits a unique solution, denoted by Xt,xs , given an initial condition Xt = x, and the

process = {a(s,

Xt,x

s ) t s T} lies in A. Then,

w = v,

and is an optimal feedback control.


15/77

Proof of the verification theorem. (First part)

1. Suppose that w is a smooth supersolution to the HJB equation:

w

t(t, x) sup

aA

Law(t, x) + f(x, a)

0, (t, x) [0, T) Rn, (6)

w(T, x) g(x), x Rn. (7)

For any A, and a controlled process Xt,x, apply Itos formula to w(s, Xt,xs )

between s = t and s = T n, and take expectation:

E

w(T n, X

t,xTn

)

= w(t, x) + E

Tnt

w

t+ Lsw

(s, Xt,xs )ds

where (n) is a localizing sequence of stopping times for the local martingale appearingin Itos formula.

Since w is a supersolution to HJB (6), this implies:

E

w(T n, Xt,xTn

)

+ E

Tnt

f(Xt,xs , s)ds

w(t, x).

By sending n to infinity, and under suitable integrability conditions, we get:

E

w(T, Xt,xT )

+ ET

t

f(Xt,xs , s)ds

w(t, x).

Since w(T, .) g, and is arbitrary, we obtain

v(t, x) = s upA

E

Tt

f(Xt,xs , s)ds + g(Xt,xT )

w(t, x).


16/77

Proof of the verification theorem. (Second part)

2. Suppose that the supremum in HJB equation is attained:

w

t(t, x) La(t,x)w(t, x) + f(x, a(t, x) ) = 0, (t, x) [0, T) Rn, (8)

w(T, x) = g(x), x Rn. (9)

Apply Itos formula to w(s, Xt,xs ) for the feedback control . By same arguments

as in the first part, we have now the equality (after an eventual localization):

w(t, x) = E

Tt

w

t+ Lsw

(s, Xt,xs )ds + w(T, X

t,xT )

= E T

tf(Xt,xs , s)ds + g(Xt,xT ) ( v(t, x)).

Together with the first part, this proves that w = v and is an optimal feedback

control.


17/77

Probabilistic formulation of the verification approach

The analytic statement of the verification theorem has a probabilistic formulation:

Suppose that the measurable function w on [0, T] Rn satisfies the two properties:

for any control A with associated controlled process X, the process

w(t, Xt) +

t0

f(Xs, s)ds is a supermartingale (10)

there exists a control A with associated controlled process X, such that the

process

w(t, Xt) + t

0

f(Xs, s)ds is a martingale. (11)

Then, w = v, and is an optimal control.

Remark. Notice that in the probabilistic verification approach, we do not need

smoothness of w, but we require a supermartingale property. In the analytic verifi-

cation approach, the smoothness of w is used for applying Itos formula to w(t, Xt).

This allows us to derive the supermartingale property as in (10), which is in fact the

key feature for proving that w v, and then w = v with the martingale property

(11).


18/77

VI. Applications

1. Merton portfolio selection in finite horizon

An agent invests at any time t a proportion t of his wealth X in a stock of price S

and 1 t in a bond of price S0

with interest rate r. The investor faces the portfolioconstraint that at any time t, t is valued in A closed convex subset ofR.

Assuming a Black-Scholes model for S(with constant rate of return and volatility

> 0), the dynamics of the controlled wealth process is:

dXt =Xtt

StdSt +

Xt(1 t)

S0tdS0t

= Xt (r + t( r)) dt + XttdWt.

The preferences of the agent is described by a utility function U: increasing and con-

cave function. The performance of a portfolio strategy is measured by the expected

utility from terminal wealth Utility maximization problem at a finite horizon T:

v(t, x) = s u pA

E[U(Xt,xT )], (t, x) [0, T] (0, ).

Standard stochastic control problem


19/77

HJB equation for Mertons problem

vt + rxvx + supaA

a( r)xvx +1

2x2a22vxx

= 0, (t, x) [0, T) (0, )

v(T, x) = U(x), x > 0.

The case of CRRA utility functions:

U(x) =xp

p, p < 1, p = 0

Relative Risk Aversion: xU(x)/U(x) = 1 p.

We look for a candidate solution to HJB in the form

w(t, x) = (t)U(x).

Plugging into HJB, we see that should satisfy the ODE:

(t) + (t) = 0, (T) = 1,

where

= rp +p supaA

a( r)

1

2a2(1 p)2

,

(t) = e(Tt).


20/77

The value function is equal to

v(t, x) = e(Tt)U(x),

and the optimal control is constant (in proportion of wealth invested)

a = argmaxaA a( r) 12a2(1 p)2.When A = R (no portfolio constraint), the values of and a are explicitly given by

=( r)2

22p

1 p+ rp.

and

a = r2(1 p)

,


21/77

General utility functions:

U is C1, strictly increasing and concave on (0, ), and satisfies the Inada conditions:

U(0) = , U() = 0.

Convex conjugate of U:

U(y) := supx>0

[U(x) xy] = U(I(y)) yI(y), y > 0,

where I := (U)1 = U.

Assume that A = R (no portfolio constraint and complete market) and for simplicity

r = 0 so that HJB is also written as

vt 1

2

2

2v2xvxx

= 0,

with a candidate for the optimal feedback control:

a(t, x) =

2vx

x2vxx.

Recall the terminal condition:

v(T, x) = U(x).

Fully nonlinear second order PDE

But remarkably, it can be solved explicitly by convex duality!


22/77

Introduce the convex conjugate of v, also called dual value function:

v(t, y) = s u px>0

[v(t, x) xy], y > 0.

change of variables: y = vx and x = vy.

v satisfies the linear parabolic Cauchy problem:

vt +1

2

2

2y2vyy = 0

v(T, y) = U(y).

From Feynman-Kac formula, v is represented as

v(t, y) = E

U(yYtT)

,

where Yt is the solution to

dYts = Yts

dWs, Y

tt = 1.

Remark. YtT = EdQ/dPFt is the density of the risk-neutral probability measureQ, under which S is a martingale:

dSt = StdWQt ,


23/77

The primal value function is obtained by duality relation:

v(t, x) = inf y>0

v(t, y) + xy

, x > 0.

From the representation of v, we get:

v(t, x) = inf y>0

E

U(yYtT)

+ xy

(12)

Recalling that U = (U)1 =: I, the infimum in (12) is attained at y = y(t, x) s.t.

E

YtTI(yYtT)

= x, (saturation budget constraint) (13)

and we have

v(t, x) = E

U(yYtT) + yYtTI(yY

tT)

.

Recalling that the supremum in U is attained at x = I(y), i.e. U(y) = U(I(y))yI(y),

we obtain:

v(t, x) = EU(Xt,xT ), with Xt,xT = I(yYtT). (14)


24/77

Consider now the strictly positive Q-martingale process:

Xt,xs := EQ

I(yYtT)Fs, t s T.

- From the saturation budget constraint (13), we have Xt,xt = x.

- From the martingale representation theorem (or since the market is complete), there

exists A s.t.

dXt,xs = Xt,xs sdW

Qs =

Xt,xs sSs

dSs,

which means that Xt,x is a wealth process controlled by the proportion , and starting

from initial capital x at time t.

From the representation (14) of the value function, this proves that Xt,x is the

optimal wealth process:

v(t, x) = E

U(Xt,xT )

.


25/77

2. Merton portfolio/consumption choice on infinite horizon

In addition to the investment in the stock, the agent can also consume from his

wealth:

(ct)t0 consumption per unit of wealth

The wealth process, controlled by (, c) is governed by:

dXt = Xt (r + t( r) ct) dt + XttdWt.

The preferences of the agent is described by a utility U from consumption, and the

goal is to maximize over portfolio/consumption the expected utility from intertem-

poral consumption up to a random time horizon:

v(x) = sup(,c)

E

0

etU(ctXxt )dt

, x > 0.

We assume that is independent of F (market information), and follows E().

Infinite horizon stochastic control problem:

v(x) = s u p(,c)

E

0

e(+)tU(ctXxt )dt

, x > 0.


26/77

HJB equation

(+ )v rxv supaA

[a( r)v +1

2a2x22v] sup

c0[U(cx) cxv] = 0, x > 0.

Explicit solution for CRRA utility function: U(x) = xp/p.

Under the condition that + > , we have

v(x) = K U(x), with K =

1 p

+

1p.

The optimal portfolio/consumption strategies are:

a = arg maxaA

[a( r) 1

2a2(1 p)2]

c =1

x(v(x))

1p1 = K

1p1 .


27/77

VII. Some other classes of stochastic control problems

Ergodic and risk-sensitive control problems

- Risk-sensitive control problem:

lim supT

1

T lnE exp T

0 f(Xt, t)dt Applications in finance: Bielecki, Pliska, Fleming, Sheu, Nagai, Davis, etc ...

- Large deviations control problem:

lim supT

1

TP

XTT

x

Dual of risk-sensitive control problem

Applications in finance: Pham, Sekine, Nagai, Hata, Sheu


28/77

Optimal stopping problems:

The control decision is a stopping time where we decide to stop the process

Value function of optimal stopping problem (over a finite horizon):

v(t, x) = suptT

E

t

f(Xt,xs )ds + g(Xt,x )

.

The HJB equation is a free boundary or variational inequality:

min

v

t Lv f , v g

= 0,

where L is the infinitesimal generator of the Markov process X.

Typical applications in finance in American option pricing


29/77

Impulse and optimal switching problems:

The control is a sequence of increasing stopping times (n)n associated to a sequence

of actions (n)n: n represents the time decision when we decide to intervene on the

state system X by using an action n Fn-measurable: Xn (Xn , n)

Value function:

v(t, x) = sup(nn)

E

Tt

f(Xt,xs )ds + g(Xt,xT ) +

n

c(Xn , n)

.

The HJB equation is a quasi-variational inequality:

min

v

t Lv f , v Hv

= 0,

where L is the infinitesimal generator of the Markov process X, and H is a nonlocal

operator associated to the jump and cost induced by an action:

Hv(t, x) = s u peE

v(t, (x, e)) + c(x, e)

.

Various applications in finance:

Transaction costs and liquidity risk models, where trading times take place dis-

cretely

Real options and firm investment problems, where decisions represent change of

regimes or production technologies


30/77

Lecture 2 : Viscosity solutions and stochastic control

Non smoothness of value functions: a motivating financial example

Introduction to viscosity solutions

Viscosity properties of the dynamic programming equation

Comparison principles

Application: Super-replication in uncertain volatility models


31/77

I. Non smoothness of value functions: a motivating financial example

Consider the controlled diffusion process

dXs = sXsdWs,

with an unbounded control valued in A = R+: Uncertain volatility model.

Consider the stochastic control problem

v(t, x) = s u pA

E[g(Xt,xT )], (t, x) [0, T] (0, ),

Superreplication cost of an option payoff g(XT).

If v were smooth, it would be a classical solution to the HJB equation:

vt + supaR+

12

a2x2vxx

= 0, (t, x) [0, T) (0, ). (1)

But, for the supremum in a R to be finite and HJB equation (1) to be well-posed,

we must have

vxx 0, i.e. v(t, .) is concave in x, for any t [0, T).


32/77

Now, by taking the zero control in the definition of v, we get

v(t, x) g(x),

which combined with the concavity of v(t, .), implies:

v(t, x) g(x), t < T ,

where g is the concave envelope of g: the smallest concave function above g.

Moreover, since g g, and by Jensens inequality and martingale property of X,

we have

v(t, x) supA

E[g(Xt,xT )]

supA

g

E[Xt,xT ]

= g(x).

Therefore,

v(t, x) = g(x), (t, x) [0, T) (0, ).

There is a contradiction with the smoothness of v, whenever g is not smooth!,for example when g is concave (hence equal to g) but not smooth.

Need to consider the case where the supremum in HJB can explode (singular case)

and to define weak solutions for HJB equation

Notion of viscosity solutions (Crandall, Ishii, P.L. Lions)


33/77

II. Introduction to viscosity solutions

Consider nonlinear parabolic second-order partial differential equations:

F(t,x,w,w

t, Dxw, D

2xxw) = 0, (t, x) [0, T) O, (2)

where O is an open subset ofRn

and F is a continuous function of its arguments,satisfying the ellipticity condition: for all (t, x) [0, T) O, r R, (q, p) R Rn,

M, M Sn,M M = F(t,x,r,q,p,M) F(t,x,r,q,p, M), (3)

and the parabolicity condition: for all t [0, T), x O, r R, q, q R, p Rn,

M Sn,

q q = F(t,x,r,q,p,M) F(t,x,r, q ,p,M ). (4)

Typical example: HJB equation

F(t,x,r,q,p,M) = q H(x,p,M),

where H is the Hamiltonian function of the stochastic control problem:

H(x,p,M) = s u paA

b(x, a).p +

1

2tr ( (x, a)M) + f(x, a)


34/77

Intuition for the notion of viscosity solutions

Assume that w is a smooth supersolution to (2). Let be a smooth test function

on [0, T) O, and (t, x) [0, T) O be a minimum point of w :

0 = (w )(t, x) = min(w ).

In this case, the first and second-order optimality conditions imply

(w )

t(t, x) 0 (= 0 ift > 0)

Dxw(t, x) = Dx(t, x) and D2xw(t, x) D

2x(t, x).

From the ellipticity and parabolicity conditions (3) and (4), we deduce that

F(t, x, (t, x), t

(t, x), Dx(t, x), D2x(t, x))

F(t, x, w(t, x),w

t(t, x), Dxw(t, x), D

2xw(t, x)) 0,

Similarly, if w is a classical subsolution to (2), then for all test functions , and

(t, x) [0, T) O such that (t, x) is a maximum point of w , we have

F(t, x, (t, x),

t(t, x), Dx(t, x), D

2x(t, x)) 0.


35/77

General definition of (discontinuous) viscosity solutions

Given a locally bounded function w on [0, T] O, we define its upper-semicontinuous

(usc) envelope w and lower-semicontinuous (lsc) envelope w by

w(t, x) = lim s upt


36/77

III. Viscosity properties of the DPE

We turn back to the stochastic control problem:

v(t, x) = supA

E

Tt

f(Xt,xs , s)ds + g(Xt,xT )

, (t, x) [0, T] Rn,

with Hamiltonian function H on Rn

Rn

Sn:

H(x,p,M) = s u paA

b(x, a).p +

1

2tr ((x, a)M) + f(x, a)

.

We introduce the domain of H as

dom(H) = {(x,p,M) Rn Rn Sn : H(x,p,M) < } ,

and make the following hypothesis (DH):

H is continuous on int(dom(H))

and there exists a continuous function G on Rn Rn Sn such that

(x,p,M) dom(H) G(x,p,M) 0.

Example. In the example considered at the beginning of this lecture:

H(x,p,M) = s u paR

12

a2x2M

,

and so

G(x,p,M) = M.


37/77

Viscosity property inside the domain

Theorem .1 The value function v is a viscosity solution to the HJB variational

inequality

min vt

H(x, Dxv, D2xv) , G(x, Dxv, D2xv) = 0, on [0, T) Rn.Remark. In the regular case when the Hamiltonian H is finite on the whole domain

RnRn Sn (this occurs typically when the control space is compact), the condition

(DH) is satisfied with any choice of strictly positive continuous function G. In this

case, the HJB variational inequality is reduced to the regular HJB equation:

v

t(t, x) H(x, Dxv, D

2xv) = 0, (t, x) [0, T) R

n,

which the value function satisfies in the viscosity sense. Hence, the above Theorem

states a general viscosity property including both the regular and singular case.


38/77

Proof of viscosity supersolution property

Let (t, x) [0, T) Rn and let C2([0, T) Rn) be a test function such that

0 = (v )(t, x) = min[0,T)Rn

(v ). (5)

By definition of v(t, x), there exists a sequence (tm, xm)m in [0, T) R

n

such that

(tm, xm) (t, x) and v(tm, xm) v(t, x),

when m goes to infinity. By the continuity of and by (5) we also have that

m := v(tm, xm) (tm, xm) 0.

Let A, a constant process equal to a A, and Xtm,xms the associated controlled

process. Let m = inf{s tm : |Xtm,xms xm| }, with > 0 a fixed constant. Let

(hm) be a strictly positive sequence such that

hm 0 andmhm

0.

We apply the first part of the DPP for v(tm, xm) to m := m (tm + hm) and get

v(tm, xm) E mtm

f(s, Xtm,xms , a)ds + v(m, X

tm,xmm ) .

Equation (5) implies that v v , thus

(tm, xm) + m E

mtm

f(s, Xtm,xms , a)ds + (m, Xtm,xmm

)

.


39/77

Apply Itos formula to (s, Xtm,xms ) between tm and m:

mhm

+ E

1

hm

mtm

t La f

(s, Xtm,xms , a)ds

0. (6)

Now, send m to infinity: by the mean value theorem, and the dominated convergence

theorem, we get

t(t, x) La(t, x) f(t, x, a) 0.

Since a is arbitrary in A, and by definition of H, this means:

t(t, x) H(x, Dx(t, x), D

2x(t, x)) 0.

In particular, (x, Dx(t, x), D2x(t, x)) dom(H), and so

G(x, Dx(t, x), D2x(t, x)) 0.

Therefore,

min


2x(t, x)) , G(x, Dx(t, x), D

2x(t, x))

0,

which is the required supersolution property.


40/77

Proof of viscosity subsolution property

Let (t, x) [0, T) Rn and let C2([0, T) Rn) be a test function such that

0 = (v )(t, x) = max[0,T)Rn

(v ). (7)

As before, there exists a sequence (tm, xm)m in [0, T) Rn

s.t.

(tm, xm) (t, x) and v(tm, xm) v(t, x),

m := v(tm, xm) (tm, xm) 0.

We will show the result by contradiction, and assume on the contrary that


2x(t, x)) > 0,

and G(x, Dx(t, x), D2x(t, x)) > 0.

Under (DH), there exists > 0 such that

t(t, y) H(y, Dx(t, y), D

2x(t, y)) > 0, for (t, x) B(t, ) B(x, ). (8)

Observe that we can assume w.l.o.g. in (7) that (t, x) achieves a strict maximum so

that

maxpB((t,x),)

(v ) =: < 0, (9)

where pB((t, x), ) = [t, t + ] B (x, ) {t + } B(x, ).


41/77

We apply the second part of DP: there exists m A s.t.

v(tm, xm)

2 E

mtm

f(Xtm,xms , ms )ds + v(m, X

tm,xmm

)

, (10)

where m = inf{s tm : (s, Xtm,xms ) / B(t, ) B(x, )}. Observe by continuity of

the state process that (m, Xtm,xm

m) pB((t, x), ) so that from (9)-(10):

(tm, xm) + m

2 E

mtm

f(Xtm,xms , ms )ds + (m, X

tm,xmm

)

.

Apply Itos formula to (s, Xtm,xms ) between tm and m, we then get after noting that

the stochastic integral vanishes in expectation:

m

2+ E

m

tm t Lms f (s, Xtm,xms , ms )ds . (11)Now, from (8) and definition of H, we have

t(s, Xtm,xms ) L

ms (s, Xtm,xms ) f(Xtm,xms ,

ms )

t(s, Xtm,xms ) H(s, Dx(s, X

tm,xms ), D

2x(s, X

tm,xms ))

> 0, for tm s m.

Plugging into (11), this implies

m

2 , (12)

and we get the contradiction by sending m to infinity: /2 .


42/77

Terminal condition

Due to the singularity of the Hamiltonian H, the value function may be discontinuous

at T, i.e. v(T, x) may be different from g(x). The right terminal condition is given

by the relaxed terminal condition:

Theorem .2 The value function v is a viscosity solution to

min

v g , G(x, Dxv, D2xv)

= 0, on {T} Rn. (13)

This means that v(T, .) is a viscosity supersolution to

min

v(T, x) g(x) , G(x, Dxv(T, x), D

2xv(T, x))

0, on Rn. (14)

and v(T, .) is a viscosity subsolution to

min

v(T, x) g(x) , G(x, Dxv(T, x), D2xv

(T, x))

0, on Rn. (15)

Remark. Denote by g the upper G-envelope of g, defined as the smallest function

above g, and viscosity supersolution to

G(x, Dg, D2g) 0.

Then v(T, x) g(x). On the other hand, since g is a viscosity supersolution to (14),

and if a comparison principle holds for (13), then v(T, x) g(x). This implies

v(T, x) = v(T, x) = v(T, x) = g(x).

In the regular case, we have g = g, and v is continuous at T.


43/77

IV. Strong comparison principles and uniqueness

Consider the DPE satisfied by the value function

min

v

t H(x, Dxv, D

2xv) , G(x, Dxv, D

2xv)

= 0, on [0, T) Rn. (16)

min v(T, x) g(x) , G(x, Dxv, D2xv) = 0, on {T} Rn. (17) We say that a strong comparison principle holds for (16)-(17) when the follow-

ing statement is true:

Ifu is an usc viscosity subsolution to (16)-(17) and w is a lsc viscosity supersolution

to (16)-(17), satisfying some growth condition, then u w.

Remark. The arguments for proving comparison principles are:

- dedoubling variables technique

- Ishiis Lemma

Standard reference: users guide of Crandall, Ishiis Lions (92).


44/77

Consequence of strong comparison principles

Uniqueness and continuity

Suppose that v and w are two viscosity solutions to (16)-(17). This means that v is

a viscosity subsolution to (16)-(17), and w is a viscosity supersolution to (16)-(17),

and vice-versa. By the strong comparison principle, we get:

v w and w v.

Since w w, v v

, this implies:

v = v = w = w.

Therefore,

v = w, i.e. uniqueness

v = v, i.e. continuity of v on [0, T) Rn.

Conclusion

The value function of the stochastic control problem is the unique continuous viscosity

solution to (16)-(17) (satisfying some growth condition).

A li i li i i i l ili d l


45/77

V. Application: superreplication in uncertain volatility model

Consider the controlled diffusion

dXs = sXsdWs, t s T,

with the control process valued in A = [a, a], where 0 a a . Given a

continuous function g on R+, we consider the stochastic control problem:

v(t, x) = s u pA

E

g(Xt,xT )

, (t, x) [0, T] (0, ).

Financial interpretation

represents the uncertain volatility process of the stock price X, and the functiong represents the payoff of an European option of maturity T. The value function

v is the superreplication cost for this option, that is the minimum capital required

to superhedge (by means of trading strategies on the stock) the option payoff at

maturity T whatever the realization of the uncertain volatility.

Th H il i f hi h i l bl i


46/77

The Hamiltonian of this stochastic control problem is

H(x, M) = supa[a,a]

12

a2x2M

, (x, M) (0, ) R.

We shall then distinguish two cases: a finite or not.

Bounded volatility: a < .

In this regular case, H is finite on the whole domain (0, ) R, and is given by

H(x, M) =1

2a2(M)x2M,

with

a(M) = a if M 0a if M < 0. v is continuous on [0, T] (0, ), and is the unique viscosity solution with linear

growth condition to the so-called Black-Scholes-Barenblatt equation

vt +1

2a2 (vxx) x

2vxx = 0, (t, x) [0, T) (0, ),

satisfying the terminal condition

v(T, x) = g(x), x (0, ).

Remark. If g is convex, then v is equal to the Black-Scholes price with volatility a,

which is convex in x, so that a(vxx) = a.

U b d d l tilit


47/77

Unbounded volatility: a = .

In this singular case, the Hamiltonian is given by

H(x, M) =

12

a2x2M if G(M) := M 0

if M < 0.

v is the unique viscosity solution to the HJB variational inequality

min

vt 1

2a2x2vxx , vxx

= 0, on [0, T) (0, ), (18)

min

v g , vxx

= 0, on {T} (0, ). (19)

Explicit solution to (18)-(19)

Denote by g the concave envelope of g, i.e. the solution to

min[g g , gxx] = 0.

Let us consider the Black-Scholes price with volatility a of the option payoff g, i.e.

w(t, x) = E

g

Xt,xT

,

where

dXs = aXsdWs, t s T, Xt = x.

Then,

v = w, on [0, T) (0, ).

P f


48/77

Proof.

Indeed, the function w is solution to the Black-Scholes equation:

wt +1

2a2x2wxx = 0, on [0, T) (0, )

w(T, x) = g(x), x (0, ).

Moreover, w inherits from g the concavity property, and so

wxx 0, (t, x) [0, T) (0, ).

(This holds true in the viscosity sense)

Therefore, w satisfies the same HJB variational inequality as v:

min

wt 1

2a2x2wxx , wxx

= 0, on [0, T) (0, ),

min

w g , wxx

= 0, on {T} (0, ).

We conclude by uniqueness result.

Remark. When a = 0, we have w = g, and so v(t, x) = g(x) on [0, T) (0, ).

L t 3 B k d St h ti Diff ti l E ti d


49/77

Lecture 3 : Backward Stochastic Differential Equations and

stochastic control

Introduction

General properties of BSDE

The Markov case : nonlinear Feynman-Kac formula. Simulation of BSDE

Application: CRRA utility maximization

Reflected BSDE and optimal stopping problem

BSDE with constrained jumps and quasi-variational inequalities

I Introduction


50/77

I. Introduction

BSDEs first introduced by Bismut (73): adjoint equation in Pontryagin maximum

principle (linear BSDEs)

Emergence of the theory since the seminal paper by Pardoux and Peng (90): general

BSDEs

BSDEs widely used in stochastic control and mathematical finance

Hedging and pricing problems linear and reflected BSDE

Portfolio optimization, risk measure nonlinear BSDE, reflected and constrained

BSDEs

Improve existence and uniqueness of BSDEs, especially quadratic BSDEs

BSDE provide a probabilistic representation of nonlinear PDEs: nonlinear Feynman-

Kac formulae

Numerical methods for nonlinear PDEs

II General results on BSDEs


51/77

II. General results on BSDEs

Let W = (Wt)0tT be a standard d-dimensional Brownian motion on (, F,F, P)

where F = (Ft)0tT is the natural filtration of W, and T is a fixed finite horizon.

Notations

P: set of progressively measurable processes on [0, T]

S2(0, T): set of elements Y P such that

E

sup0tT

|Yt|2

< ,

H2(0, T)d: set of elements Z P, Rd-valued, such that

E T0

|Zt|2dt < .

Definition of BSDE

A (one-dimensional) Backward Stochastic Differential Equation (BSDE in short) is

written in differential form as

dYt = f(t, Yt, Zt)dt Zt.dWt, YT = , (1)

where the data is a pair (, f), called terminal condition and generator (or driver):

L2(, FT,P) , f(t ,,y,z ) is P B(R Rd)-measurable.

A solution to (1) is a pair (Y, Z) S2(0, T) H2(0, T)d such that

Yt = + T

t

f(s, Ys, Zs)ds T

t

Zs.dWs, 0 t T.

Under some specific assumptions on the generator f there is existence and uniqueness


52/77

Under some specific assumptions on the generator f, there is existence and uniqueness

of a solution to the BSDE (1).

Standard Lipschitz assumption (H1)

f is uniformly Lipschitz in (y, z), i.e. there exists a positive constant C s.t. for

all (y , z , y, z):

|f(t ,y,z ) f(t, y, z)| C

|y y| + |z z|

, dt dP a.e.

The process {f(t, 0, 0), t [0, T]} H2(0, T)

Theorem (Pardoux and Peng 90) Under (H1), there exists a unique solution (Y, Z)

to the BSDE (1).Proof. (a) Assume first the case where f does not depend on (y, z), and consider

the martingale

Mt = E

+

T0

f(t, )dt

Ft ,which is square-integrable under (H1), i.e. M S2(0, T). By the martingale

representation theorem, there exists a unique Z H2

(0, T)d

s.t.

Mt = M0 +

t0

Zs.dWs, 0 t T.

Then, the process

Yt := E

+

Tt

f(s, )dsFt

= Mt

t0

f(s, )ds, 0 t T,

satisfies (with Z) the BSDE (1).

Proof (b) Consider now the general Lipschitz case As in the deterministic case


53/77

Proof. (b) Consider now the general Lipschitz case. As in the deterministic case,

we give a proof based on a fixed point method. Let us consider the function on

S2(0, T)m H2(0, T)d, mapping (U, V) S2(0, T) H2(0, T)d to (Y, Z) = (U, V)

defined by

Yt = + Tt f(s, Us, Vs)ds T

t Zs.dWs.

This pair (Y, Z) exists from Step (a). We then see that (Y, Z) is a solution to the

BSDE (1) if and only if it is a fixed point of .

Let (U, V), (U, V) S2(0, T)H2(0, T)d and (Y, Z) = (U, V), (Y, Z) = (U, V).

We set (U , V) = (U U, V V), (Y , Z) = (Y Y, Z Z) and ft = f(t, Ut, Vt)

f(t, U

t, V

t ). Take some > 0 to be chosen later, and apply Itos formula to es

|Ys|

2

between s = 0 and s = T:

|Y0|2 =

T0

es

|Ys|2 2Ys.fs

ds

T0

es|Zs|2ds 2

T0

esYs Zs.dWs.

By taking the expectation, we get

E|Y0|2 + E

T0

es

|Ys|2 + |Zs|

2

ds

= 2E T

0

esYs.fsds

2CfE T

0

es|Ys|(|Us| + |Vs|)ds

4C2fE T

0

es|Ys|2ds

+

1

2E T

0

es(|Us|2 + |Vs|

2)ds

Proof continued (b) Now we choose = 1 + 4C2 and obtain


54/77

Proof continued. (b) Now, we choose = 1 + 4Cf, and obtain

E T

0

es

|Ys|2 + |Zs|

2

ds

1

2E T

0

es(|Us|2 + |Vs|

2)ds

.

This shows that is a strict contraction on the Banach space S2(0, T) H2(0, T)d

endowed with the norm

(Y, Z)

=E T

0

es

|Ys|2 + |Zs|

2

ds 1

2

.

We conclude that admits a unique fixed point, which is the solution to the BSDE

(1).

Non-Lipschitz conditions on the generator

f is continuous in (y, z) and satisfies a linear growth condition on (y, z). Then,

there exists a minimal solution to the BSDE (1). (Lepeltier and San Martin 97)

f is continuous in (y, z), linear in y, and quadratic in z, and is bounded. Then,

there exists a unique bounded solution to the BSDE (1) (Kobylanski 00).

III The Markov case: non-linear Feynman-Kac formulae


55/77

III. The Markov case: non linear Feynman Kac formulae

Linear Feynman-Kac formula

Consider the linear parabolic PDE

v

t

(t, x) + Lv(t, x) + f(t, x) = 0, on [0, T) Rd (2)

v(T, .) = g, on Rd, (3)

where L is the second-order differential operator

Lv = b(x).Dxv +1

2tr( (x)D2xv).

Consider the (forward) diffusion process

dXt = b(Xt)dt + (Xt)dWt.

Then, by Itos formula to v(t, Xt) between t and T, with v smooth solution to (2)-(3):

v(t, Xt) = g(XT) +

Tt

f(s, Xs)ds

Tt

Dxv(s, Xs)(Xs)dWs.

It follows that the pair (Yt, Zt) = (v(t, Xt), (Xt)Dxv(t, Xt)) solves the linear BSDE:

Yt = g(XT) + Tt

f(s, Xs)ds Tt

ZsdWs.

Remark

We can compute the solution v(0, X0) = Y0 by the Monte-Carlo expectation:

Y0 = E

g(XT) +

T0

f(s, Xs)ds

.

Non linear Feynman-Kac formula


56/77

Non linear Feynman Kac formula

Consider the semilinear parabolic PDE

v

t+ Lv + f(t,x, v, Dxv) = 0, on [0, T) R

d (4)

v(T, .) = g, on Rd, (5)

The corresponding BSDE is

Yt = g(XT) +

Tt

f(s, Xs, Ys, Zs)ds

Tt

ZsdWs, (6)

in the sense that:

the pair (Yt, Zt) = (v(t, Xt), (Xt)Dxv(t, Xt)) solves (6)

Conversely, if (Y, Z) is a solution to (6), then Yt = v(t, Xt) for some deterministic

function v, which is a viscosity solution to (4)-(5).

The time discretization and simulation of the BSDE (6) provides a numerical

method for solving the semilinear PDE (4)-(5)

Simulation of BSDE: time discretization


57/77

Simulation of BSDE: time discretization

Time grid = (ti) on [0, T] : ti = it, i = 0, . . . , N , t = T /N

Forward Euler scheme X for X : starting from Xt0 = x,

Xti+1 := X

ti + b(X

ti)t + (X

ti)Wti+1 Wti

Backward Euler scheme (Y, Z) for (Y, Z) : starting from YtN = g(XtN

),

Yti = Yti+1

+ f(Xti , Yti+1

, Zti)t Zti

.

Wti+1 Wti

(7)

and take conditional expectation:

Yti = EYti+1 + f(Xti , Yti+1, Zti)tXti

To get the Z-component, multiply (7) by Wti+1 Wti and take expectation:

Zti =1

tE

Yti+1(Wti+1 Wti)Xti

Simulation of BSDE: numerical methods


58/77

S S s

How to compute these conditional expectations! several approaches:

Regression based algorithms (Longstaff, Schwartz)

Choose q deterministic basis functions 1, . . . , q, and approximate

Zti = E

Yti+1(Wti+1 Wti)Xti q

k=1

kk(Xti

)

where = (k) solve the least-square regression problem:

arg infRq

E

Yti+1(Wti+1 Wti)

q

k=1kk(X

ti

)

2

Here E is the empirical mean based on Monte-Carlo simulations of Xti , Xti+1

, Wti+1

Wti.

Efficiency enhanced by using the same set of simulation paths to compute all

conditional expectations.

Other methods:

Malliavin Monte-Carlo approach (P.L. Lions, Regnier)

Quantization methods (Pages)

Important literature: Kohatsu-Higa, Pettersson (01), Ma, Zhang (02), Bally and

Pages (03), Bouchard, Ekeland, Touzi (04), Gobet et al. (05), Soner and Touzi (05),

Peng, Xu (06), Delarue, Menozzi (07), Bender and Zhang (08), etc ...

IV. Application: CRRA utility maximization


59/77

pp y

Consider a financial market model with one riskless asset S0 = 1, and n stocks of

price process

dSt = diag(St)tdt + tdWt

,

wher W is a d-dimensional Brownian motion (with d n), b, bounded adapted

processes, of full rank n.

Consider an agent investing in the stocks a fraction of his wealth X at any time:

dXt = Xtt diag(St)

1dSt = Xt(ttdt +

ttdWt) (8)

A0: set ofF-adapted processes valued in A closed convex set ofRn

, and satisfying:T0 |

tt|dt +

T0 |

tt|

2dt < , (8) is well-defined.

Given a utility function U on (0, ), and starting from initial capital X0 > 0, the

objective of the agent is:

V0 := supA

E[U(XT)]. (9)

Here, X is the solution to (8) controlled by A0, and starting from X0 at time 0,

and A is the subset of elements A0 s.t. {U(X ), T0,T} is uniformly integrable.

We solve (9) by dynamic programming and BSDE.

Value function processes:


60/77

p

For t [0, T], and A, we denote by:

At() = { A : .t = .t} ,

and define the family ofF-adapted processes

Vt() := ess supAt()

E

U(XT)Ft, 0 t T.

Dynamic programming (DP)

For any A, the process {Vt(), 0 t T} is a supermartingale

There exists an optimal control A to V0 if and only if the martingale property

holds, i.e. the process {Vt(), 0 t T} is a martingale.

In the sequel, we exploit the DP in the case of CRRA utility functions: U(x) =

xp/p, p < 1. The key observation is the property that the F-adapted process

Yt :=Vt()

U(Xt )> 0 does not depend on A, and YT = 1.

We adopt a BSDE verification approach: we are looking for (Y, Z) solution to

Yt = 1 +

Tt

f(s ,,Y s, Zs)ds

Tt

ZsdWs, (10)

for some generator f to be determined such that

For any A, the process {U(Xt )Yt, 0 t T} is a supermartingale

There exists A for which {U(Xt )Yt, 0 t T} is a martingale.

By applying Itos formula to U(Xt )Yt, the supermartingale property for all


61/77

y pp y g ( t ) t, p g p p y

A, and the martingale property for some imply that f should be equal to

f(t, Yt, Zt) = p supaA

(tYt + tZt).a

1 p

2Yt|ta|

2

, (11)

with a candidate for the optimal control given by

t arg maxaA

(tYt + tZt).a

1 p2

Yt|ta|2, 0 t T. (12)

Existence and uniqueness of a solution to the BSDE (10)-(11):

Change of variables Y = ln Y, Z = Z/Y

(Y , Z) satisfy a quadratic BSDE. Then, we rely on results by Kobylanski (00)

Existence and uniqueness of (Y, Z) S(0, T) H2(0, T)d

Verification argument: let (Y, Z) be the solution to (10)-(11)

By construction U(Xt )Yt is a (local)-supermartingale + integrability conditions

on A: it is a supermartingale supAE[U(XT)] U(X0)Y0.

By BMO techniques, we show that defined in (12) lies in A U(Xt )Yt is amartingale E[U(XT)] = U(X0)Y0

We conclude that V0 := supAE[U(XT)] = U(X0)Y0, and is an optimal control.

Markov cases


62/77

Merton model: the coefficients of the stock price (t) and (t) are deterministic

In this deterministic case, the BSDE (10)-(11) is reduced to an ODE:

Y(t) = 1 + T

t

f(s, Y(s))ds, f (t, y) = y p supaA (t).a

1 p

2|(t)a|2 =: y(t)

and the solution is given by: Y(t) = eTt(s)ds we find again the solution to the

Merton problem:

V0 = U(X0)exp T

0

(s)ds

.

Factor model: the coefficients of the stock price (t, Lt) and (t, Lt) depend on


63/77

( ) ( )

a factor process

dLt = (Lt)dt + dWt.

In this case, the BSDE for (Y , Z) = (ln Y,Z/Y) is written as:

Yt =Tt

f(s, Ls, Ys, Zs)ds Tt

ZsdWs

with a quadratic (in z) generator

f(t,,y,z) = p supaA

((t, ) + (t, )z).a

1 p

2|(t, )a|2

+

1

2z2.

Yt

= (t, Lt), with a corresponding semilinear PDE for :

t+ ().D +

1

2tr(D2) + f(t,,,D) = 0, (T, ) = 0.

Value function:

V0 = U(X0)exp

(0, L0)

.

Remark The BSDE approach and dynamic programming is also well-suitable for

exponential utility maximization:

Many papers: El Karoui, Rouge (00), Hu, Imkeller, Muller (04), Sekine (06),

Becherer (06), etc ...

V. Reflected BSDE and optimal stopping problem


64/77

We consider a class of BSDEs where the solution Y is constrained to stay above a

given process L, called obstacle. An increasing process K is introduced for pushing

the solution upwards, above the obstacle Notion of reflected BSDE:

Given pair of terminal condition/generator (, f) and a continuous obstacle process

(Lt) s.t. LT, find a triple of adapted processes (Y ,Z,K) with K nondecreasing

s.t.

Yt = +

Tt

f(s, Ys, Zs)ds

Tt

ZsdWs + KT Kt, 0 t T, (13)

Yt Lt, 0 t T, (14)T0

(Yt Lt)dKt = 0. (15)

Connection with optimal stopping problem: in the case where f(t, ) does

not depend on (y, z), there exists a unique solution to (13)-(14)-(15) given by

Yt = esssupTt,T E

tf(s)ds + L1


65/77

Snell envelope of Ht :=t0

f(s)ds + Lt1t


66/77

rator, and L is a continuous obstacle in S2(0, T).

Existence and approximation by penalization

For each n N, we consider the (unconstrained) BSDE

Ynt = +Tt

f(s, Yns , Zns )ds + K

nT K

nt

Tt

Zns .dWs, (17)

where Knt = nTt

(Ysn Ls)ds existence and uniqueness of (Yn, Zn).

State a priori uniform estimates on the sequence (Yn, Zn, Kn): there exists ome

positive constant C s.t.

E sup0tT

|Ynt |2 + T0

|Znt |2dt + |KnT|2 C, n N. By comparison principle for BSDE, (Yn)n is an increasing sequence, and it con-

verges to some Y S2(0, T), and the convergence also holds in H2(0, T), i.e.

limnE T

0 |Ynt Yt|

2dt

= 0. Moreover, Yt Lt.

Prove that (Zn, Kn)n is a Cauchy sequence in H2(0, T)d S2(0, T): use Itos

formula to |Ynt Ymt |

2, and inequality 2ab a2+ 1

b2 for suitable . Consequently,

(Zn, Kn)n converges to some (Z, K) in H2(0, T)d S2(0, T). Pass to the limit in

(17) in order to obtain the existence of (Y ,Z,K) solution to the reflected BSDE.

Remark: Alternative formulation of the Skorokhod condition.


67/77

The definition of a solution to the reflected BSDE (13)-(14) with the Skorokhod

condition (15)T0 (YtLt)dKt = 0 can be formulated equivalently in terms of minimal

solution:

We say that (Y ,Z,K) is a minimal solution to (13)-(14) if for any other solution(Y , Z, K) solution to (13)-(14), we have Yt Yt, 0 t T.

Any solution to the reflected BSDE (13)-(14)-(15) is a minimal solution to (13)-

(14), and the converse is also true.

Connection with variational inequalities in the Markov case


68/77

Consider the case where:

= g(XT), f(t ,,Y t, Zt) = f(t, Xt, Yt, Zt), Lt = h(Xt), 0 t T,

with g h, and where X is a diffusion process on Rn

dXt = b(Xt)dt + (Xt)dWt.

Then, the solution to the reflected BSDE is given by Yt = v(t, Xt) for some deter-

ministic function v, viscosity solution to the variational inequality:

min

v

t Lv f(t,x,v,Dxv) , v h

= 0, on [0, T) Rn

v(T, .) = g on Rn.

VI. BSDE with constrained jumps and quasi-variational inequalities


69/77

Consider the impulse control problem:

v(t, x) = supE

g(XT) +

Tt

f(Xs )ds +

t


70/77

a continuation region C in which v(t, x) > Hv(t, x) and

v

t Lv f = 0

an action region D in which:

v(t, x) = Hv(t, x) = supeE

v(t, x + (x, e)) + c(x, e).

Various applications of impulse control problems:

Financial modelling with discrete transaction dates, due e.g. to fixed transaction

costs or liquidity constraints

Optimal multiple stopping: swing options

Projects investment and real options: management of power plants, valuation

of gas storage and natural resources, forest management, ...

...

Impulse control: widespread economical and financial setting with many prac-

tical applications More generally to models with control policies that do not

accumulate in time.

Main theoretical and numerical difficulty in the QVI (18) :


71/77

The obstacle term contains the solution itself

It is nonlocal

Classical approach : Decouple the QVI (18) by defining by iteration the sequence

of functions (vn)n:

min

vn+1

t Lvn+1 f , vn+1 Hvn

= 0 , vn+1(T, .) = g

associated to a sequence of optimal stopping time problems (reflected BSDEs)

Furthermore, to compute vn+1, we need to know vn on the whole domain

heavy computations, especially in high dimension (state space discretization): nu-

merically challenging!

Idea of our approach


72/77

Instead of viewing the obstacle term as a reflection ofv onto Hv (or vn+1 into Hvn),

consider it as a constraint on the jumps of v(t, Xt) for some suitable forward jump

process X:

Let us introduce the uncontrolled jump diffusion X :

dXt = b(Xt)dt + (Xt)dWt +

E

(Xt, e)(dt, de),

where is a Poisson random measure whose intensity is finite and supports the

whole space E.

We randomize the state space!

Take some smooth function v(t, x) and define:

Yt := v(t, Xt), Zt := (Xt)Dxv(t, Xt),

Ut(e) := v(t, Xt + (Xt, e)) v(t, Xt) + c(Xt, e)

= (Hev v)(t, Xt)

Apply Itos formula to Yt = v(t, Xt) between t and T:


73/77

Yt = YT +

Tt

f(Xs)ds + KT Kt

Tt

Zs.dWs

+

Tt

E

[Us(e) c(Xs, e)](ds, de),

where

Kt :=

t0

(v

t Lv f)(s, Xs)ds

Now, suppose that min[v

t Lv f, v Hv] 0, and v(T, .) = g :

Then (Y ,Z,U,K) satisfies

Yt = g(XT) + Tt

f(Xs)ds + KT Kt Tt

Zs.dWs

+

Tt

E

[Us(e) c(Xs, e)](ds, de), (19)

K is a nondecreasing process, and U satisfies the nonpositivity constraint :

Ut(e) 0, 0 t T, e E. (20)

View (19)-(20) as a Backward Stochastic Equation with jump constraints

We expect to retrieve the solution to the QVI (18) by solving the minimal solution

to this constrained BSE.

General definition of BSDEs with constrained jumps


74/77

Minimal Solution : find a solution (Y ,Z,U,K) S2 H2(0, T)d L2() A2 to

Yt = g(XT) +

Tt

f(Xs, Ys, Zs)ds + KT Kt

Tt

Zs.dWs

T

t E(Us(e) c(Xs, Ys, Zs, e))(ds, de) (21)with

h(Ut(e), e) 0, dP dt (de) a.e. (22)

such that for any other solution (Y , Z, U , K) to (21)-(22) :

Yt Yt, 0 t T, a.s.

Assumptions on the coefficients:

Forward SDE : b and Lipschitz continuous, bounded and Lipschitz contin-

uous w.r.t. x uniformly in e:

|(x, e) (x, e)| k|x x| e E

Backward SDE : f, g and c have linear growth, f and g Lipschitz continuous,

c Lipschitz continuous w.r.t. y and z uniformly in x and e

|c(x ,y,z,e) c(x, y, z, e)| kc(|y y| + |z z|)

Constraint : h Lipschitz continuous w.r.t. u uniformly in e:

|h(u, e) h(u, e)| kh|u u|

and

u h(u, e) nonincreasing. (e.g. h(u, e) = u)

Existence and approximation via penalization


75/77

Consider for each n the BSDE with jump:

Ynt = g(XT) +

Tt

f(Xs, Yns , Z

ns )ds + K

nT K

nt

Tt

Zns .dWs

TtE

[Uns (e) c(Xs, Yns, Zns , e)](ds, de) (23)

with a penalization term

Knt = n

t0

E

h(Uns (e), e)(de)ds

where h = max(h, 0).

For each n, existence and uniqueness of (Yn, Zn, Un) solution to (23) from Tang

and Li (94), and Barles et al. (97).

Convergence of the penalized solutions

Theorem .1 Under (H1), there exists a unique minimal solution

(Y ,Z,U,K) S2 H2(0, T)d L2() A2

with K predictable, to (21)-(22). Y is the increasing limit of (Yn) and also in

S2(0, T), K is the weak limit of (Kn) inS2(0, T), and for any p [1, 2),

Zn ZHp(0,T)

+ Un ULp()

0,

as n goes to infinity.

Sketch of proof.


76/77

Convergence of (Yn): by comparison results (under the nondecreasing property of

h) Yn Yn+1

Convergence of (Zn, Un, Kn) : more delicate!

A priori uniform estimates on (Yn, Zn, Un, Kn)n in L2

weak convergence of (Zn, Un, Kn) in L2

Moreover, in general, we need some strong convergence to pass to the limit in

the nonlinear terms f(X, Yn, Zn), c(X, Yn, Zn) and h(Un(e), e).

Control jumps of the predictable process K via a random partition of the

interval (0,T) and obtain a convergence in measure of (Zn, Un, Kn)

Convergence of (Zn, Un, Kn) in Lp, p [1, 2)

Related semilinear QVI


77/77

By Markov property, the minimal solution to the constrained BSDE with jumps is

Yt = v(t, Xt) for some deterministic function v.

The function v is the unique viscosity solution to the QVI:

min

v

t Lv f(. ,v ,Dxv), inf

eEh(Hev v, e)

= 0 on [0, T) Rn, (24)

together with the relaxed terminal condition:

min

v g , infeE

h(Hev v, e)

= 0 on {T} Rn. (25)

Probabilistic representation of semilinear QVIs, and in particular ofimpulse control problems by means of BSDEs with constrained jumps.

Numerical implications for the resolution of QVIs by means of simulation of the pe-

nalized BSDE: PhD thesis of M. Bernhart, in partnership with EDF for the valuation

of swing options and gas storage contacts.

Cours Controle Marrakech

Documents

Transcript of Cours Controle Marrakech