Course on Optimal ControlSOD311 Ensta Paris Tech

and Optimization master, U. Paris-Saclay

Part I: the Pontryagin approachVersion of Aug. 21, 2019

1

J. Frédéric Bonnans

Centre de Mathématiques Appliquées, Inria, Ecole Polytechnique, 91128 Palaiseau, France(Frederic.Bonnans@inria.fr).

1Updates and additional material on the pagehttp://www.cmap.polytechnique.fr/∼bonnans/notes/oc/oc.html

Preface

These are lecture notes for the Optimal control course for students of both Ensta Paris Techand the Optimization master, U. Paris-Saclay. We consider optimization problems for controlsystems modelized with ordinary differential equations. These lecture note deal only with thePontryagin approach, in which we mainly discuss necessary conditions for a trajectory to beoptimal.

The notes are organized as follows. We give several examples of optimal control problemsin chapter 1. Chapter 2 recalls general results in convex analysis and differential calculus (witha special focus on the notion of reduction) and obtains first order optimality conditions.

Chapter 3 states Pontryagin’s principle for a problem with control and initial-final con-straints, as well as its extension for problems with decision variables (not depending on time),and variable horizon. It presents an application to minimal time problems, including geodesicsin a Riemaniann setting, and a discussion of qualification conditions. The proof of Pontryagin’sprinciple is based on Ekeland’s principle.

Chapter 4 is devoted to the study of the shooting algorithm, that sometimes allows to reducean optimal control problem to the computation of a zero of a finite dimensional shooting function.This assumes that Pontryagin’s principle allows to express the control as a function of the stateand costate. We analyze problems that are unconstrained or with initial-final constraints. It isshown how the local well-posedness of the shooting function allows to obtain error estimates fordiscretized problems. Finally we show that this well-posedness can be characterized by somesecond order optimality conditions.

Chapter 5 deals with problems with running state constraints (constraints for each time).We state and prove a version of Pontryagin’s principle for this class. Junction conditions areanalyzed, based on the notion of order of the constraints. As an illustration we discuss problemsof elastic string and beam with obstacle.

Some useful references related to this course are: [15] for optimization theory (convex anal-ysis, duality, Lagrange multipliers), [14] for an introduction to automatic control and optimalcontrol, and [5], [6] for algorithmic aspects. Concerning applications : in spatial mechanics andquantum mechanics in the second case) [16], [17], in medecine, part IV of [27], in ecology [24],in chemistry [6], and on bioprocesses, e.g. [4]. We also mention the more advanced book [38],and [2] on the shooting approach.

3

Contents

Chapter 1. Motivating examples 91. Simple linear quadratic problems 91.1. Quadratic regulator 91.2. Fuller’s problem 102. Population dynamics 102.1. Fish harvesting 102.2. Cell population control 112.3. Predator-Prey 123. Bioreactors 123.1. Chemostat 123.2. Bio-gaz production 134. Flight mechanics 134.1. The Goddard problem 134.2. Space trajectories 13

Chapter 2. First steps 151. Calculus in Rn and in Banach spaces 151.1. Some notations 151.2. Normal cones and separation of convex sets 171.2.1. Preliminary 171.2.2. Minimization over a convex set 171.2.3. Normal cones 171.2.4. Separation of sets 181.3. Taylor expansions 182. Lagrange multipliers 202.1. Implicit function theorem 202.2. On Newton’s method 212.3. Reduced equation 212.4. Reduced cost 222.4.1. First order analysis 222.4.2. Computation of the derivative of the reduced cost 222.4.3. Reduction Lagrangian 232.4.4. Second order analysis 232.5. Equality constrained optimization 242.5.1. First order optimality conditions 242.5.2. Second order optimality conditions 252.5.3. Link with the reduction approach 252.6. Calculus over L∞ spaces 263. Optimal control setting 273.1. Weak derivatives 273.2. Controlled dynamical system and associated cost 283.3. Derivative of the cost function 303.4. Optimality conditions with control constraints 314. Examples 33

5

4.1. Multiple integrators 334.1.1. Example: double integrator, quadratic energy 334.2. Control constraints 33

Chapter 3. Pontryagin’s principle 351. Pontryagin’s minimum principle 351.1. The easy Pontryagin’s minimum principle 351.1.1. Main result 351.1.2. A related nonlinear expansion of the state 381.2. Pontryagin’s principle with two point constraints 381.2.1. Main result 381.2.2. Specific structures 401.2.3. Link with Lagrange multipliers; convex problems 411.3. Variations of the pre-Hamiltonian 411.3.1. Pontryagin extremal over (0, T ) 411.3.2. Constant pre-Hamiltonian for autonomous problem 421.3.3. Variation of the pre-Hamiltonian for non autonomous problem 422. Extensions 432.1. Decision variables 432.1.1. A design problem 442.2. Variable horizon 452.3. Varying initial and final time 472.4. Interior point constraints 482.4.1. A general setting 482.4.2. Variable durations 502.4.3. Variable dynamics structure 502.5. Minimal time problems, geodesics 502.5.1. General results 512.5.2. Linear dynamics 512.5.3. Geometric optics and Riemaniann geometry 523. Qualification conditions 533.1. Lagrange multipliers 533.1.1. General framework 533.1.2. Specific structures 543.2. Pontryagin multipliers 554. Proof of Pontryagin’s principle 564.1. Ekeland’s principle 564.2. Ekeland’s metric and the penalized problem 574.3. Proof of theorem 3.13 58

Chapter 4. Shooting algorithm, second order conditions 611. Shooting algorithm 611.1. Unconstrained problems 611.2. Problems with initial-final state constraints 631.3. Time discretization 642. Second-order optimality conditions 662.1. Expansion of the reduced cost 662.2. More on the Hessian of the reduced cost 672.3. Case of initial-final equality constraints 682.4. Links with the shooting algorithm 683. Control constraints 694. Notes 70

6

Chapter 5. State constrained problems 711. Pontryagin’s principle 711.1. Setting 711.2. Duality in spaces of continuous functions 721.2.1. Bounded variation functions 721.2.2. Costate equation 731.2.3. Link with Lagrange multipliers; convex problems 741.3. Variation of the pre-Hamiltonian 741.3.1. Constant pre-Hamiltonian for autonomous systems 751.3.2. Non autonomous systems 751.4. Decision variables, variable horizon 761.4.1. Decision variables 761.4.2. Variable horizon 762. Junction conditions 772.1. Representation of control constraints 772.2. Constraint order and junction conditions 772.2.1. Order of a state constraint 782.2.2. Costate jumps 782.2.3. Continuity of the multipliers 782.2.4. Continuity of the control 792.2.5. Jumps of the derivative of the control 802.3. Examples 813. Proof of Pontryagin’s principle 833.1. Some results in convex analysis 833.2. Renormalization and distance to a convex set 843.3. Diffuse perturbations. 853.4. Proof of theorem 5.5 86

Bibliography 91

Index 93

7

1Motivating examples

Contents

1. Simple linear quadratic problems 91.1. Quadratic regulator 91.2. Fuller’s problem 102. Population dynamics 102.1. Fish harvesting 102.2. Cell population control 112.3. Predator-Prey 123. Bioreactors 123.1. Chemostat 123.2. Bio-gaz production 134. Flight mechanics 134.1. The Goddard problem 134.2. Space trajectories 13

We present in this chapter some typical examples of optimal control problems arising in thecomputation of regulators, for population dynamics problem in ecology or biology, bioprocesses,and flight mechanics. The examples were solved numerically using the software bocop.org; someof them are taken from the collection [8].

1. Simple linear quadratic problems

In this section we present two academic examples, who may seem close one to each other.It appears, however, that the solutions have completely different behavior.

1.1. Quadratic regulator. Consider the problem:

min 12

∫ T

0(x2

1(t) + x22(t))dt;

s.t. x1(t) = x2(t); x2(t) = u(t), t ∈ (0, T ); x1(0) = 0, x2(0) = 1,

(1.1)

for t ∈ [0, T ], T = 5, subject to the control and state bound constraint:

(1.2) − 1 ≤ u(t) ≤ 1, x2(t) ≥ −0.2, t ∈ (0, T ).

9

We write the problem in the Mayer form (i.e., with a final cost only), defining an extra statevariable with dynamics

(1.3) x3(t) = 12(x2

1(t) + x22(t)), x3(0) = 0.

In this new setting the cost function can be expressed as x3(T ). The optimal trajectory, displayedin figure 1, appears to have three arcs (connected by two junction points): lower control bound,active state constraint, and a final unconstrained arc; the control is discontinuous at the junctionpoints.

Figure 1. x1 : position, x2 : speed, x3 : cost, u : acceleration

1.2. Fuller’s problem. Here is a variant of the previous problem, due to [23]:

(1.4) Min

∫ T

0x2(t)dt; x(t) = u(t) ∈ [−10−2, 10−2]; x(0) = x1; x(0) = x2 given.

For large enough T , the solution is as follows. There exists τ ∈ (0, T ) such that: (i) over (0, τ),the control has values ±1, the arc lenghts converging geometrically to 0, and (ii) over (τ, T ),x(t) = u(t) = 0. These switches are not easy to reproduce numerically. Those in figure 2 wereobtained with 1000 time steps, T = 3.5, x(0) = 0, x(0) = 1, x(T ) = x(T ) = 0.

2. Population dynamics

We next discuss examples of population dynamics problems, that are widely used in ecologyand biology.

2.1. Fish harvesting. The state equation is

(1.5) x(t) = B(x(t))x(t)− au(t)x(t)

Here x(t) is the size of the fish population, B(·) is the birth rate, a > 0 is the harvestingefficiency, u(t) is the harvesting effort; the cost function is

(1.6)∫ T

0(c(u(t))− u(t)x(t))dt

10

Figure 2. Fuller problem: chattering control (with zoom); x and v.

where c(u) is the harvesting cost, with constraint 0 ≤ u(t) ≤ 1 and x(T ) ≥ xT .Clasical birth rate models are the exponential, logistics and Gomperz ones, with positive

parameters a, b:

(1.7) BE(x) := ax; BL(x) := a(1− x/b); BG(x) := a log(b/x).

In the uncontrolled case, that is, when u(t) = 0 for all t, for the logistics and Gomperz models,x(t) → b in a monotone way, with an exponential speed of convergence: for some C > 0 andd > 0 depending on the initial state x(0):

(1.8) |x(t)− b| ≤ Ce−dt.

One can consider the optimization of a steady-state solution: compute u solution of

(1.9) Minu

c(u)− aux; B(x)x = aux; u ∈ [0, uM ].

About this and related optimal control problems, see [24, Ch. 4].

2.2. Cell population control. Similar state equation

(1.10) x(t) = B(x(t))x(t)− aw(t)x(t)

are used for modelling cell populations. Here w(t) ≥ 0 is the drug concentration, subject to theupper bound w(t) ≤ wM , and possibly satisfying the dynamics

(1.11) w(t) = −bw(t) + v(t),

with b > 0 elimination rate and v(t) drug injection rate. One calls (1.10) the pharmacodynamicsequation (modelling the action of the drug) and (1.11) the pharmacokinetics equation (whichmodels the dynamics of the drug itself). We want to minimize x(T ), with possibly constraintson time integrals of w(t):

(1.12)∫ βi

αi

w(t)dt ≤ γi,

with 0 ≤ αi < βi ≤ T and γi > 0, for i = 1 to N . Extensions of such models are applied to theproblem of optimizing cancer treatments [39].

11

Figure 3. Trajectory in predator-prey space, beginning at the bottom of thefigure. Four arcs: lower control bound, state constraint (vertical segment on theright), lower control bound (nearly vertical segment on the right), unconstrained.

2.3. Predator-Prey. A simple extension to the controlled setting of the Lokta-Volterrasystem is, see [24, Ch. 5] or [25]:

(1.13)x = αx− βxy − a1u,

y = δxy − γy − a2u,

with positive parameters α, β, δ, a1, a2. The uncontrolled dynamics (with u(t) = 0 for all t)has a unique nonzero equilibrium point

(1.14) x = γ/δ; y = α/β.

The uncontrolled trajectories are a closed loop around this point. The objective is to reach (x, y)by minimizing the sum of final time and c times the integral of the control. For the numericalillustration the constants are α = β = γ = δ = 1; b1 = 0.9, b2 = 1, c = 1; see figure 3.

3. Bioreactors

3.1. Chemostat. The simplest bioreactor model is the chemostat one:

(1.15)

x(t) = (µ(s(t))− u(t)/v)x(t),

s(t) = −1

γµ(s(t))x(t) +

u(t)

v(sin − s(t)).

The state variables are the biomass concentration x and substrate s. We can take the growthfunction µ of Monod type: µ(s) = µmaxs/(k+s), with µmax > 0 and k > 0. The yield coefficientis γ > 0. The control is the flow u ∈ [0, uM ]. The volume v > 0 is either constant, of for batchprocesses a function of time, subject to the dynamics

(1.16) v(t) = u(t)

The problem is usually to reach a given state in minimal time. See the analysis of the solutionin [33].

12

3.2. Bio-gaz production. The state equation is

(1.17)

y(t) = ν(t)y(t)/(1 + y(t))− (r + u(t))y(t),

x(t) = (µ(s(t))− βu(t))x(t),

s(t) = βu(δy(t)− s(t))− µ(s(t))x(t).

Here y is the mass of algae, x is the biomass concentration and s is the substrate; β > 0 is thevolume ratio between the algae tanks and the bioreactor. To function to be maximized is

(1.18)∫ T

0µ(s(t))x(t)dt.

Given a light model ν(t) that is periodic with period of a day, the solution tends to have aperiodic behavior when optimizing with a large horizon; see [3].

4. Flight mechanics

4.1. The Goddard problem. This is the problem of a vertical ascent of a rocket, see [40],State variables: position h, speed v, mass m. Control variable u: normalized thrust (maximalthrust denoted by Tmax) Cost: opposite of final mass (minimize fuel consumption) T : free finaltime, D(h, v) is the drag function. The state equation is

max m(T ),

s.t. h = v,

v = −1/h2 +1

m(Tmax u−D(h, v)),

m = −b Tmax u,

0 ≤ u ≤ 1, D(h, v) ≤ Dmax,

h(0) = 0, v(0) = 0, m(0) = 1, h(T ) = 1.

(1.19)

We take the simplest model of drag function: cρ(h)v2, with c > 0 the aerodynamic coefficient,and ρ volumic mass expressed as function of altitude, in the exponential volumic mass model:ρ(h) = αe−βh for some positive α, β. The optimal solution has four arcs, see figure 4. For athree dimensional extension of this problem, see [13].

4.2. Space trajectories. There is a large literature on the subject; let us just mention thecase of space lauchers [30], and atmosphere reentry [11].

13

Figure 4. Four arcs: full thrust, maximum drag, unconstrained arc, zero thrust.Discontinuity of control at all junction points.

14

2First steps

Contents

1. Calculus in Rn and in Banach spaces 151.1. Some notations 151.2. Normal cones and separation of convex sets 171.3. Taylor expansions 182. Lagrange multipliers 202.1. Implicit function theorem 202.2. On Newton’s method 212.3. Reduced equation 212.4. Reduced cost 222.5. Equality constrained optimization 242.6. Calculus over L∞ spaces 263. Optimal control setting 273.1. Weak derivatives 273.2. Controlled dynamical system and associated cost 283.3. Derivative of the cost function 303.4. Optimality conditions with control constraints 314. Examples 334.1. Multiple integrators 334.2. Control constraints 33

1. Calculus in Rn and in Banach spaces

1.1. Some notations. We denote by Rn the Euclidean space of dimension n, whose el-ements are identified with vertical vectors, with norm |x| := (

∑ni=1 x

2i )

1/2 and scalar productx · y :=

∑ni=1 xiyi, for all x, y in Rn. The dual space Rn∗ is identified with the set of n dimen-

sional horizontal vectors. Other useful norms in Rn are the `s norm |x|s := (∑n

i=1 |xi|s)1/s, fors ∈ [1,∞[, and the uniform norm |x|∞ := max|xi|, 1 ≤ i ≤ n. Note that norms in Rn aredenoted with a single bar. Let A, B be n dimensional symmetric matrices. We write A B ifA−B is semi positive definite, and A B if A−B is positive definite.

LetX be a Banach space (a normed vector space that is complete, i.e., every Cauchy sequencehas a limit), with norm denoted by ‖x‖X or ‖x‖ if there is no ambiguity. If Y is another Banachspace, we denote by L(X,Y ) the set of linear continuous mappings X → Y . Endowed with the

15

norm

(2.1) ‖A‖ := sup‖Ax‖Y ; ‖x‖X ≤ 1,L(X,Y ) is a Banach space. Note that a linear mapping A : X → Y is continuous iff the abover.h.s. is finite.

Exercice 2.1. Prove that, similarly, if Z is a third Banach space and a : X × Y → Z isbilinear, then it is continuous iff

(2.2) ‖a‖ := sup‖a(x, y)‖Z ; ‖x‖X ≤ 1, ‖y‖Y ≤ 1,is finite, and that the set of continuous bilinear forms endowed with this norm is a Banach space.

The set of linear continuous forms over X (linear and continuous applications X → R) isdenoted by X∗, and the action of x∗ ∈ X∗ over x ∈ X is denoted by 〈x∗, x〉X . The space X∗ isa Banach space, endowed with the dual norm

(2.3) ‖x∗‖X∗ := sup〈x∗, x〉X ; ‖x‖X ≤ 1.If A ∈ L(X,Y ) with X and Y Banach spaces, we denote by A† ∈ L(Y ∗, X∗) the transposeoperator defined, for all y∗ ∈ Y ∗, by(2.4) 〈A†y∗, x〉X = 〈y∗, Ax〉Y , for all x ∈ X.

That A† is continuous follows from the relations

(2.5) ‖A†y∗‖X∗ = sup‖x‖≤1

〈A†y∗, x〉X ≤ ‖A‖‖y∗‖Y ∗ .

We say that the Banach space X is a Hilbert space if there exists a symmetric continuous bilinearform a(·, ·) over X such that

(2.6) ‖x‖2 = a(x, x), for all x ∈ X.A Hilbert space H is endowed with the scalar product

(2.7) (x, x′)X := a(x, x′), for all x, x′ in X.

By the Fréchet-Riesz representation theorem, if X is a Hilbert space, we have that

(2.8) Given x∗ ∈ X∗, there exists x′ ∈ X such that 〈x∗, x〉X = (x′, x)X , for all x ∈ X.

Exercice 2.2. Check that x′ is uniquely defined, and prove the Fréchet-Riesz representationtheorem by showing that we can take for x′ the unique solution of the optimization problembelow:

(2.9) Minx∈X

12‖x‖

2X − 〈x∗, x〉X .

In the sequel X is a Banach space. If A and B are subsets of X, their Minkowski sum anddifference are

(2.10)A+B := a+ b; a ∈ A, b ∈ B;A−B := a− b; a ∈ A, b ∈ B.

If E ⊂ R we define the product

(2.11) EA := ea; e ∈ E, a ∈ A.If f : X → Y , A ⊂ X and B ∈ Y , we set

(2.12)f(A) := f(a); a ∈ A;f−1(B) := x ∈ Rn; f(x) ∈ B.

The closure of A ⊂ X is the intersection of closed sets containing A, and is denoted by cl(A).The interior and boundary of A are denoted by

(2.13)

int(A) := x ∈ X; y ∈ A if y is close enough to x;∂A := cl(A) \ int(A).

16

The segment [x, y], where x and y belong to X is

(2.14) [x, y] := αx+ (1− α)y; α ∈ [0, 1].We say that A ⊂ X is convex if

(2.15) [x, y] ⊂ A, for all x, y in A.

By 0X we denote the set having for unique element the zero of X. We say that a subset C ofa vector space Y is a cone if α y ∈ C, for all α > 0 and y ∈ C. The positive (resp. negative) coneof Rn is the set of element of Rn whose all coordinates are nonnegative (resp. nonpositive), andis denoted by Rn+ (resp. Rn−). More generally, if X is a space of real valued functions (perhapsdefined only a.e.), we call positive (resp. negative) cone of X and denote by X+ (resp. X−) theset of nonnegative (resp. nonpositive) (perhaps only a.e.) functions of X. We then denote byX∗+ the dual positive cone defined by

(2.16) X∗+ := x∗ ∈ X∗; 〈x∗, x〉 ≥ 0, for all x ∈ X+.

1.2. Normal cones and separation of convex sets.1.2.1. Preliminary. Again in the sequelX is a Banach space. The orthogonal space to E ⊂ X

is the closed vector space of X∗ defined by

(2.17) E⊥ := x∗ ∈ X∗; 〈x∗, x〉X = 0, for all x ∈ E.

Remark 2.3. When X is a Hilbert space and E ⊂ X we have another notion of primalorthogonal space which is (we remind that (·, ·)X denotes the scalar product in X):

(2.18) x′ ∈ X; (x′, x)X = 0, for all x ∈ E.There should be no confusion in our applications.

1.2.2. Minimization over a convex set. Let K be a convex subset of a Banach space X, andlet f : X → R be differentiable. We say that f attain a local minimum over K at x ∈ K iff(x) ≤ f(x), for all x ∈ K sufficiently close to x.

Lemma 2.4. Let f attain a local minimum over K at x. Then

(2.19) Df(x)(x− x) ≥ 0, for all x ∈ K.

Conversely, if f is convex, (2.19) implies that f attains its (global) minimum over K at x.

Proof. Let x ∈ K. For t ∈ [0, 1], xt := (1− t)x+ tx belongs to K, and xt → x when t ↓ 0,so that f(x) ≤ f(xt), for small enough t. Therefore the conclusion follows from

(2.20) 0 ≤ limt↓0

1

t(f(xt)− f(x)) = Df(x)(x− x).

Conversely, if f is convex, and x ∈ K, f(x) − f(x) ≥ Df(x)(x − x), so that (2.19) impliesf(x)− f(x) ≥ 0.

1.2.3. Normal cones.

Definition 2.5. Let K be a closed convex subset of X and x ∈ K. The normal cone to Kat x is the set

(2.21) NK(x) := x∗ ∈ X∗; 〈x∗, x′ − x〉 ≤ 0, for all x′ ∈ K.Its elements are called normal directions. This is a closed and convex cone. When X = Rn weidentify X with its dual and see normal directions as elements of Rn.

Exercice 2.6. Check that (i) If K is a singleton, the normal cone is X∗. (ii) If x ∈ int(K),the normal cone is the singleton 0. (iii) If K is a vector subspace, the normal cone coincideswith the orthogonal space. (iv) If K has a smooth boundary, the normal cone at x ∈ K is ofthe form R+y, where y is the outward normal at x to K.

17

Exercice 2.7 (Product form). Let X = X ′ ×X ′′, where X ′ and X ′′ are Banach spaces,and K = K ′ ×K ′′, with K ′ ⊂ X ′ and K ′′ ⊂ X ′′ closed and convex. Let x = (x′, x′′) ∈ K withx′ ∈ K ′ and x′′ ∈ K ′′. Check that

(2.22) NK(x) = NK′(x′)×NK′′(x

′′).

Exercice 2.8. Let K = 0Rn × Rm− . Check that, for x ∈ K, we have

(2.23) NK(x) = Rn × λ ∈ Rm+ ; λixn+i = 0, i = 1, . . . ,m.

Definition 2.9. Let E ⊂ X. Its negative polar cone is

(2.24) E− := x∗ ∈ X∗; 〈x∗, x〉 ≤ 0, for all x ∈ E.Its positive polar cone is E+ := −E−. By ’polar cone’ we mean the negative one.

We see that the polar cone is a closed convex cone, intersection of closed half spaces.

Lemma 2.10. Let C ⊂ X be a closed convex cone, and x ∈ C. Then(2.25) NC(x) = C− ∩ x⊥.

Proof. Let x∗ ∈ C−∩x⊥ and y ∈ C. Then 〈x∗, y〉 ≤ 0 and 〈x∗, x〉 = 0, so that 〈x∗, y−x〉 ≤0, proving that x∗ ∈ NC(x).Conversely, let x∗ ∈ NC(x). For y ∈ C, since C is a convex cone, z := x+ y = 2(1

2x+ 12y) ∈ C,

and therefore 0 ≥ 〈x∗, z − x〉 = 〈x∗, y〉, proving that x∗ ∈ C−. Since 0 ∈ C we also have0 ≥ 〈x∗, 0 − x〉 = −〈x∗, x〉, and the converse inequality holds since x∗ ∈ C−. The conclusionfollows.

Exercice 2.11. Let K := Rn−. Deduce from the previous lemma that if x ∈ K, thenλ ∈ NK(x) iff λ ∈ Rn+ and λixi = 0, i = 1 to n.

1.2.4. Separation of sets.

Definition 2.12. Let A and B be two convex subsets of a Banach space X. Let x∗ ∈ X∗,x∗ 6= 0. We say that x∗ separates A and B if

(2.26) 〈x∗, a〉 ≤ 〈x∗, b〉, for all a ∈ A and b ∈ B.

We start with the following geometric form of the Hahn-Banach theorem.

Theorem 2.13. Let A and B be two convex subsets of a Banach space X, with emptyintersection. If A−B has a nonempty interior, then there exists x∗ separating A and B.

Proof. (a) The result is obtained in [18, Ch. 1, Thm 1.6], assuming that either A of B isopen.(b) Set E := int(B−A). Since A∩B = ∅, E does not contain zero. By step (a), there exists x∗separating 0 and E. Now let a ∈ A and b ∈ B, and set e := b−a. Then e is the limit of a sequenceek in E (take ek := (1− 1/k)e+ (1/k)e, with e ∈ E). Therefore, 〈x∗, b− a〉 = limk〈x∗, ek〉 ≥ 0,as was to be proved.

Remark 2.14. Let X be finite dimensional, and A, B be convex subsets with empty inter-section. Then there exists x∗ separating A and B. See (the stronger result) [36, Thm 11.3].

1.3. Taylor expansions. Let f : X → Y , where X and Y are Banach spaces. We calldirectional derivatives of f at x ∈ X in direction h ∈ X the following amount, if it exists:

(2.27) f ′(x, h) := limt↓0

f(x+ th)− f(x)

t.

We say that f is differentiable, or Fréchet differentiable, at x ∈ X, if there exists a linearcontinuous mapping X → Y , denoted by Df(x) or f ′(x), and called derivative of f at x, suchthat

(2.28) ‖f(x+ h)− f(x)− f ′(x)h‖Y = o(‖h‖X).

18

In this case

(2.29) f ′(x, h) = Df(x)h, for all h ∈ X.

When X = Rn and Y = Rp, we identify f ′(x) with the usual p× n Jacobian matrix:

(2.30) f ′ij(x) =∂fi(x)

∂xj, 1 ≤ i ≤ p; 1 ≤ j ≤ n.

When Y = R, and X is a Hilbert space, we denote by ∇f(x) and call gradient of f at x theelement of X associated with f ′(x) by the Fréchet-Riesz representation theorem, characterizedby

(2.31) f ′(x)h = (∇f(x), h)X , for all h ∈ X.

When X = Rn, ∇f(x) is the element of Rn whose coordinates are equal to the partial derivativesof f at x, and f ′(x) = ∇f(x)†.

If Z is another Banach space and f : X × Y → Z, we denote by e.g. Dxf(x, y) or fx(x, y)its partial derivatives. If in addition Z = R, we denote by ∇xf(x, y) its partial gradient, andthen, if X = Rn and Y = Rq, fxy(x, y) is identified with the n × q matrix with general term∂f(x, y)/∂xi∂yj .

We recall the Taylor expansion with integral term: if f : X → Y is (n+1) times continuouslydifferentiable, with n ≥ 0, then

(2.32) f(x+ h) = f(x) + · · ·+ 1

n!Dnf(x)(h)n +

∫ 1

0

(1− t)n

n!Dn+1f(x+ th)(h)n+1dt.

For n = 0 and 1, this gives

(2.33)

f(x+ h) = f(x) +

∫ 1

0Df(x+ th)hdt,

f(x+ h) = f(x) +Df(x)h+

∫ 1

0(1− t)D2f(x+ th)(h)2dt.

The Taylor expansion (2.32) may be expressed as

(2.34) f(x+ h) = f(x) + · · ·+ 1

(n+ 1)!Dn+1f(x)(h)n+1 + rn(x, h)

where the remainder satisfies

(2.35)

(i) rn(x, h) = a(x, h)(h)n+1;

(ii) a(x, h) =

∫ 1

0

(1− t)n

n!

(Dn+1f(x+ th)−Dn+1f(x)

)dt.

The above relation uses the identity

(2.36)∫ 1

0

(1− t)n

n!dt =

1

(n+ 1)!

Since f : X → Y is (n+ 1) times continuously differentiable if follows that

(2.37) ‖rn(x, h)‖Y = o(‖h‖n+1X ) when h→ 0, for any given x ∈ X.

Sometimes we need more precise estimates of the remainder, as the one below:

Lemma 2.15. Let f : X → Y be (n + 1) times continuously differentiable, Dn+1f(x) beingLipschitz with constant L. Then

(2.38) ‖rn(x, h)‖Y ≤L

(n+ 2)!‖h‖n+2

X .

19

Proof. By (2.35), we have that

(2.39) ‖rn(x, h)‖Y ≤ L‖h‖n+2X

∫ 1

0t(1− t)n

n!dt.

Integrating by parts,we see that the integral is equal to ((n+ 1)!)−1∫ 1

0 (1− t)n+1dt = 1/(n+ 2)!The conclusion follows.

Some more accurate estimates of the remainder are based on the following notion.

Definition 2.16. (i) Let X and Y be Banach spaces, E ⊂ X and f : E → Y . The modulusof continuity of f , over E, at x ∈ E is the nondecreasing function ωf,x : R+ → R+ ∪ +∞defined by

(2.40) ωf,x,E(ε) := sup‖f(x)− f(x)‖; ‖x− x‖ ≤ ε, x ∈ E.(ii) The modulus of continuity of f over E is

(2.41) ωf,E(ε) := sup‖f(x′)− f(x)‖; ‖x′ − x‖ ≤ ε, x′, x ∈ E.

The moduli of continuity are nondecreasing functions with value 0 at 0, and the restrictionof f to E is continuous at x iff ωf,x,E(ε) is continuous at 0, i.e., if ωf,x,E(ε) ↓ 0 when ε ↓ 0.

Definition 2.17. We say that f is uniformly continuous over E if ωf,E(ε) ↓ 0 when ε ↓ 0.

We recall the link between continuity and uniform continuity.

Lemma 2.18. Let E be a compact subset (any open covering contains a finite covering) of aBanach space X. Then a continuous function over E is uniformly continuous over E.

Proof. We must prove that if xk, x′k in E satisfy ‖x′k−xk‖ → 0, then ‖f(x′k)−f(xk)‖ → 0.If this does not hold, extracting a subsequence we may assume that lim infk ‖f(x′k)− f(xk)‖ >0. Since any sequence in a compact metric space has a convergent subsequence, extracting ifnecessary a subsequence, we may assume that (x′k, xk)→ (x′, x). Since ‖x′k − xk‖ → 0 we musthave x′ = x. A compact set being closed, x ∈ E. From the continuity of f at x, it follows that‖f(x′k)− f(xk)‖ → 0. We have obtained a contradiction. The conclusion follows.

2. Lagrange multipliers

2.1. Implicit function theorem. Let U , Y and Z be Banach spaces, and Ψ : U ×Y → Zbe of class C1. Let (x, y) ∈ U × Y satisfy

(2.42) Ψ(u, y) = 0; DyΨ(u, y) is invertible.

We recall that A ∈ L(U, Y ) is said to be invertible if it is a bijection. It can then be provedthat its inverse A−1 (obviously linear) is continuous (as a consequence of the celebrated openmapping theorem, see [18, Corollary 2.7]).

Theorem 2.19. [IFT] If (2.42) holds, there exist an open neighbourhood V of u and a C1

mapping ϕ : V → Y and γ > 0 such that the equation

(2.43) Ψ(u, y) = 0; u ∈ V ; ‖y − y‖Y ≤ γholds iff y = ϕ(u).

Given u ∈ V , we have that Ψ(u, ϕ(u)) = 0. The derivative of this function is also equal tozero, i.e.

(2.44) Ψu(u, ϕ(u)) + Ψy(u, ϕ(u))Dϕ(u) = 0.

Taking u = u, we find that

(2.45) Ψu(u, y) + Ψy(u, y)Dϕ(u) = 0.

Since Ψy(u, y) is invertible, we deduce that

(2.46) Dϕ(u) = −Ψy(u, y)−1Ψu(u, y).

20

From this expression, and since the inverse mapping A 7→ A−1 is of class C∞ over the (open)set of invertible mappings Y → Z, we easily deduce that:

Corollary 2.20. Under the hypotheses of theorem 2.19, if Ψ is of class Cp, p ≥ 1, then ϕis also of class Cp. If DpΨ is Lipschitz, then so is Dpϕ.

2.2. On Newton’s method. Let X and Y be Banach spaces and F : X → Y be of classC1. We say that x ∈ X is a zero of F if F (x) = 0, and that it is a regular zero if in additionDF (x) is invertible. Newton’s method for finding a zero of F consists in computing the Newtonsequence xk in X such that

(2.47) F (xk) +DF (xk)(xk+1 − xk) = 0, k ∈ N,starting from some point x0 ∈ X. The sequence is well-defined as long asDF (xk) is an invertible.It is easily checked that the set of linear invertible elements of L(X,Y ) is open. Since DF (x) isa continuous function of x it follows that DF (x) is inversible in a vicinity of x. It can be provedthat:

Theorem 2.21. if x0 is close enough to the regular zero x, the Newton sequence is well-defined and xk → x superlinearly, i.e.,

(2.48)‖xk+1 − x‖‖xk − x‖

→ 0.

If in addition x 7→ DF (x) is Lipschitz near x, then we have quadratic convergence, i.e.

(2.49) ‖xk+1 − x‖ = O(‖xk − x‖2

).

Proof. See e.g. [9, Ch. 6].

2.3. Reduced equation. Consider a system with two (vector) equations and unknowns,such that the second variable can be eliminated from the second equation. We will compare theoriginal system of two equations, with the reduced one obtained after elimination of the secondvariable. Namely, let U , Y , W , Z be Banach spaces, Φ : U × Y →W and Ψ : U × Y → Z be ofclass C1. Consider the equations, in W × Z:(2.50) Φ(u, y) = 0; Ψ(u, y) = 0.

Let (u, y) ∈ U × Y satisfy

(2.51) (u, y) is a zero of (Φ,Ψ), and DyΨ(u, y) is invertible.

By the IFT, in the vicinity of (u, y), we have that

(2.52)

The relation Ψ(u, y) = 0 is equivalent to y = χ(u),for some C1 function χ : V → Y , where V is a neighbourhood of u,such that whenever y = χ(u):DuΨ(u, y) +DyΨ(u, y)Dχ(u) = 0.

We will call Ψ(u, y) = 0 the state equation. Locally, (2.50) is equivalent to the reduced equation

(2.53) Ξ(u) = 0, where Ξ(u) := Φ(u, χ(u)).

In addition

(2.54)DΞ(u) := DuΦ(u, y) +DyΦ(u, y)Dχ(u)

= DuΦ(u, y)−DyΦ(u, y)DyΨ(u, y)−1DuΨ(u, y).

Given (v, w) ∈ U ×W , one easily checks that

(2.55)DΞ(u)v = w iff there exists z ∈ Y such thatDΦ(u, y)(v, z) = w; DΨ(u, y)(v, z) = 0.

Lemma 2.22. The point u is a regular zero of Ξ iff (u, y) is a regular zero of (Φ,Ψ).

21

Proof. Consider the system

(2.56)DuΦ(u, y)v +DyΦ(u, y)z = a;DuΨ(u, y)v +DyΨ(u, y)z = b.

Since DyΨ(u, y) is invertible, we can eliminate z from the second equation. Using (2.54), we seethat the above system is equivalent to

(2.57) DΞ(u)v = a−DyΦ(u, y)DyΨ(u, y)−1b.

The conclusion easily follows.

2.4. Reduced cost.2.4.1. First order analysis. We next consider an optimization problem of the form

(2.58) Minu,y

f(u, y); Ψ(u, y) = 0.

Here U , Y , Z are Banach spaces, f : U × Y → R and Ψ : U × Y → Z are of class C1. Let (u, y)be a zero of Ψ, such that DyΨ(u, y) is invertible. Then for (u, y) close to (u, y), the consequence(2.52) of the IFT applies. Let F (u) := f(u, χ(u)) be the reduced cost. Given a neighbourhoodV of u, we may define a (localized) reduced problem as

(2.59) Minu

F (u); u ∈ V.

2.4.2. Computation of the derivative of the reduced cost. In view of (2.52), the derivative ofF at u close to u is, writing y = χ(u):

(2.60) DF (u)v = Duf(u, y)v +Dyf(u, y)Dχ(u)v= Duf(u, y)v −Dyf(u, y)DyΨ(u, y)−1DuΨ(u, y)v.

Therefore,

(2.61) DF (u) = Duf(u, y)−Dyf(u, y)DyΨ(u, y)−1DuΨ(u, y).

However, we must pay attention to the fact that we usually represent the action of the linearform DF (u) over v with the dual notation 〈DF (u), v〉. Note that the transpose of an invertiblelinear mapping is invertible, and that transposition and inversion commute1, which justifies thenotation A−†. We have that

(2.62) 〈DF (u), v〉 = 〈Duf(u, y), v〉U + 〈Dyf(u, y), Dχ(u)v〉Y= 〈Duf(u, y) +Dχ(u)†Dyf(u, y), v〉U ,

that is,

(2.63) DF (u) = Duf(u, y) +Dχ(u)†Dyf(u, y)= Duf(u, y)−DuΨ(u, y)†DyΨ(u, y)−†Dyf(u, y).

One must be careful of avoiding confusions between (2.61) and (2.63), which, despite an apparentdifference, represent the same operator.

1Indeed, if A ∈ L(X,Y ) is invertible, solving A†y∗ = x∗, with y∗ ∈ Y ∗ and given x∗ ∈ X∗, amountsto 〈y∗, Ax〉 = 〈x∗, x〉 for all x ∈ X, or equivalently 〈y∗, y〉 = 〈x∗, A−1y〉 = 〈(A−1)†x∗, y〉, for all y ∈ Y .Since y 7→ 〈(A−1)†x∗, y〉 is linear and continuous, the existence and uniqueness of y∗ follows, and we havethat (A†)−1x∗ = (A−1)†x∗.

22

2.4.3. Reduction Lagrangian. In practice, one obtains the expression of DF (u) using thereduction Lagrangian, where (u, y, p) ∈ U × Y × Z∗:

(2.64) LR(u, y, p) := f(u, y) + 〈p,Ψ(u, y)〉Z ,

and the costate equation, where y = χ(u):

(2.65) 0 = DyLR(u, y, p) = Dyf(u, y) +DyΨ(u, y)†p = 0.

Note the use of the dual notation. Given (u, y) ∈ U × Y , y being the state associated with u,since DyΨ(u, y) (and hence, DyΨ(u, y)†) is invertible, the costate equation has a unique solutionp ∈ Z∗, called the costate associated with u:

(2.66) p = −DyΨ(u, y)−†Dyf(u, y).

Observe now that

(2.67)DuLR(u, y, p) = Duf(u, y) +DuΨ(u, y)†p,

= Duf(u, y)−DuΨ(u, y)†DyΨ(u, y)−†Dyf(u, y),= DF (u),

where the last equality uses (2.63), and is therefore equal to 0 if u is a local solution. We saythat u ∈ U is a stationary point of F (u) if DF (u) = 0. We have proved the following useful rule:

Lemma 2.23. (i) The derivative of the reduced cost F is equal to the partial derivative w.r.t.the control of the reduction Lagrangian, computed at the control and associated state and costate.(ii) An element u of U is a stationary point of F (u) iff there exists p ∈ Z∗ such that, y beingthe associated state:

(2.68) DuLR(u, y, p) = 0; DyLR(u, y, p) = 0.

2.4.4. Second order analysis. The costate approach is also useful for obtaining a simpleexpression of the Hessian (the second derivative seen as a bilinear form) of the reduced cost,assuming now that f and Ψ are C2. The Hessian of the reduction Lagrangian (w.r.t. the controland costate variables) is the quadratic form Q : U × Y → R defined by

(2.69) Q(v, z) := D2f(u, y)(v, z)2 + 〈p,D2Ψ(u, y)(v, z)2〉.

Here we assume that y and p are the state and costate associated with the control u. We denoteby z[v] the solution of the linearized state equation

(2.70) DuΨ(u, y)v +DyΨ(u, y)z = 0.

Given v ∈ U it has a unique solution denoted by z[v] in Y , namely

(2.71) z[v] := −DyΨ(u, y)−1DuΨ(u, y)v.

Lemma 2.24. The Hessian of the reduced cost satisfies

(2.72) D2F (u)(v, v) = Q(v, z[v]), for all v ∈ U .

Proof. Given σ ≥ 0, set uσ := u+ σv. The associated state denoted by yσ satisfies

(2.73) ‖yσ − y − σz[v]‖Y = O(σ2).

So we have that

(2.74) F (uσ) = f(uσ, yσ) = L(uσ, yσ, p)= L(u, y) + σDuL(u, y, p)v + 1

2σ2Q(v, z[v]) + o(σ2),

where we have used the fact that DyL(u, y, p) = 0. The result follows.

Corollary 2.25. Let F have a local minimum at u ∈ U . Then Q(v, z[v]) ≥ 0, for allv ∈ U .

23

Proof. From the Taylor expansion of F it follows that, for v ∈ U and t ∈ R:

(2.75) F (u+ tv) = F (u) + tDF (u)v + 12 t

2D2F (u)(v, v) + o(t2),

and since DF (u) = 0, the local optimality of u implies

(2.76) 0 ≤ limt↓0

F (u+ tv)− F (u)12 t

2= D2F (u)(v, v).

We conclude using lemma 2.24.

2.5. Equality constrained optimization.2.5.1. First order optimality conditions. Consider next an ’abstract’ optimization problem

with equality constraints:

(2.77) Minxf(x); g(x) = 0.

Here g : X → Y (Banach spaces) and f : X → R are of class C2. This is obviously a gener-alization of the previous setting, but without the possibility of reduction to an unconstrainedreduced probem. Let x be a local solution of this problem, in the sense that for some ε > 0:

(2.78) f(x) ≤ f(x), whenever g(x) = 0 and ‖x− x‖X ≤ ε.

We assume that x satisfies the following qualification condition

(2.79) The operator Dg(x) is surjective.

The Lagrangian of the problem is the function Lg : X × Y ∗ → R defined by

(2.80) Lg(x, λ) := f(x) + 〈λ, g(x)〉Y .

Theorem 2.26. There exists a unique Lagrange multiplier λ ∈ Y ∗ such that (dual notation)

(2.81) 0 = DxLg(x, λ) := Df(x) +Dg(x)†λ.

The proof is based on the following lemmas below. The starting point is the concept ofmetric regularity.

Lemma 2.27. Let (2.79) hold. Then the following metric regularity property is satisfied:there exists cg > 0 such that, if x ∈ X is close enough to x, there exists x′ ∈ X such that

(2.82) ‖x′ − x‖X ≤ cg‖g(x)‖Y and g(x′) = 0.

Proof. See e.g. [9, Ch. 3].

Exercice 2.28. Take X = Y = R and g(x) = x2. Show that the metric regularity propertydoes not hold at x = 0. Also, show that the problem of minimizing f(x) = x under the constraintg(x) = 0 has solution x = 0, with which no Lagrange multiplier is associated.

Corollary 2.29. If (2.79) holds, any h ∈ Ker g′(x) is tangent to the manifold g−1(0) inthe sense that, setting xσ := x+ σh, for σ ∈ R:

(2.83) There exists x′σ ∈ U such that g(x′σ) = 0 and ‖x′σ − xσ‖ = o(|σ|).

Proof. That h ∈ Ker g′(x) implies ‖g(xσ)‖ = o(σ). We conclude with lemma 2.27.

Corollary 2.30. If (2.79) holds, then f ′(x) ∈ (Ker g′(x))⊥.

Proof. Otherwise there would exists h ∈ Ker g′(x) such that f ′(x)h 6= 0. Changing h into−h if necessary we may assume that f ′(x)h < 0. Define x′σ as above. By the previous corollary,

(2.84) limσ↓0

f(x′σ)− f(x)

σ= f ′(x)h < 0,

contradicting the local optimality of x. The conclusion follows.

24

Proof of theorem 2.26. By the above corollary, f ′(x) ∈ (Ker g′(x))⊥. Since g′(x) issurjective, its image is closed. Therefore, the orthogonal of its kernel coincides with the imageof its transpose operator, see e.g. [18, Thm. 2.19]. So, there exists λ ∈ Y ∗ such that f ′(x) +g′(x)†λ = 0, as was to be proved.

Remark 2.31. In a finite dimensional setting, the orthogonal of the kernel of a linear oper-ator always coincides with the image of its transpose. For a counterexample in a Banach spacesetting, let A be the injection from L2(0, 1) (identified with its dual) into L1(0, 1). Its imageis dense, but not closed. Now A† : L∞(0, 1) → L2(0, 1) is defined, for all w ∈ L∞(0, 1) andu ∈ L1(0, 1) by 〈A†w, u〉L2(0,T ) = 〈w,Au〉L1(0,T ) =

∫ 10 w(t)u(t)dt. Since A is injective its kernel

is reduced to 0 and the orthogonal of the kernel is L2(0, 1). On the other hand the image ofits transpose is L∞(0, 1).

2.5.2. Second order optimality conditions. We next obtain second order necessary conditionsas an easy consequence of the previous results:

Proposition 2.32. Let x be a qualified local solution of (2.77), with associated Lagrangemultiplier λ. Then

(2.85) D2xxLg(x, λ)(h, h) ≥ 0, for all h ∈ Ker g′(x).

Proof. Set as before xσ := x+ σh and let x′σ be given by corollary 2.29. Since g(x′σ) = 0,and DxLg(x, λ) = 0 by (2.81), we have for |σ| small enough

(2.86) 0 ≤ f(x′σ)− f(x) = Lg(x′σ, λ)− Lg(x, λ)

= 12σ

2D2xxLg(x, λ)(h, h) + o(σ2).

Dividing by σ2 and making σ → 0 we obtain the conclusion.

Exercice 2.33. Let X be a Hilbert space, x0 ∈ X, M ∈ L(X) symmetric and invertible,and for x ∈ X, g(x) := 1

2(x,Mx)X − 12 . Write the first and second order optimality conditions

for the problem of projecting x0 over g−1(0) (we do not discuss the existence of a solution).Hint: Set f(x) := 1

2‖x − x0‖2. Check that (i) ∇g(x) = Mx (deduce that any feasible point isqualified), (ii) if x is solution, then x−x0 +λMx = 0, (iii) the Hessian of Lagrangian is I+λM .

2.5.3. Link with the reduction approach. We next establish the relation with the ’non re-duced’ formulation, assuming that x = (u, y), i.e.

(2.87) Minu,y

Φ(u, y); Ψ(u, y) = 0; g(u, y) = 0.

We assume that DyΨ(u, y) is invertible so that ’locally’ Ψ(u, y) = 0 iff y = ϕ(u) where ϕ isgiven by the IFT. So the corresponding reduced problem is

(2.88) MinuF (u); G(u) = 0,

where

(2.89) F (u) := Φ(u, ϕ(u)); G(u) := g(u, ϕ(u));

The Lagrangian functions associated with the original and reduced problems are resp.

(2.90)L(u, y, p, λ) := Φ(u, y) + 〈p,Ψ(u, y)〉+ 〈λ, h(u, y)〉,`(u, λ) := F (u) + 〈λ,G(u)〉.

The costate equation at the point x = (u, y), with y the state associated with u, reads

(2.91) 0 = DyL(u, y, p, λ) = DyΦ(u, y) +DyΨ(u, y)†p+Dyg(u, y)†λ.

The optimality conditions for the the original and reduced problems are resp. (assuming in thecase of the original problem that the costate equation holds)

(2.92)

(i) 0 = DuL(u, y, p, λ) = DuΦ(u, y) +DuΨ(u, y)†p+Dug(u, y)†λ.

(ii) 0 = Du`(u, λ) = F ′(u) +G′(u)†λ.

25

Lemma 2.34. Let the costate equation (2.91) hold. Then λ satisfies (2.92)(i) iff it satisfies(2.92)(ii). In other words, the Lagrange multipliers associated with the original and reducedformulation coincide.

Proof. It suffices to eliminate p thanks to the costate equation (2.91) in (2.92)(i), and toexpress F ′(u) and G′(u) (thanks to the IFT) in (2.92)(ii).

Remark 2.35. In the same way we can check that the second order necessary optimalityconditions for the original and reduced problem give the same information, since

(2.93) D2uu`(u, λ)(v, v) = D2

(u,y)2L(u, y, p, λ)(v, z)2

where (v, z) ∈ U × Y satisfy DΨ(u, y)(v, z) = 0.

2.6. Calculus over L∞ spaces. We denote by L0(0, T ) the set of measurable functionsover (0, T ). With f : R × Rn → Rm we associate the Nemitskii mapping F : L0(0, T )n →L0(0, T )m defined by

(2.94) F (y)(t) := f(t, y(t)) for a.a. t ∈ (0, T ).

Lemma 2.36. If f is of class Cp, then F is of class Cp: L∞(0, T )n → L∞(0, T )m, andsatisfies

(2.95) (DjF (y)(z)j)(t) = Djyjf(t, y(t))(z(t))j a.e.,

for all j ≤ p, and so, for all y, z in L∞(0, T )n we have the following Taylor expansion:

(2.96) F (y + z)(t) = f(t, y(t)) +

p−1∑j=1

1

j!Djyjf(t, y(t))(z(t))j + r(t); ‖r‖∞ = o(‖z‖p∞).

Proof. Being continuous, F is bounded over bounded sets. In addition, the composititionof a measurable mapping by a continuous one is measurable. So, F has image in L∞(0, T )m.

In view of the Taylor expansion (2.32), the equality in (2.96) holds with

(2.97) r(t) := a(t, y(t), z(t))(z(t))p,

where the symmetric p linear form a is defined by

(2.98) a(t, y, z) :=

∫ 1

0

(1− t)p−1

(p− 1)!

(Dpypf(t, y + sz)−Dp

ypf(t, y))

ds,

and therefore (the norm of the multilinear form is defined analogously to (2.2))

(2.99) |r(t)| ≤ ‖a(t, y(t), z(t))‖|z(t)|p.Since continuous functions are uniformly continuous over compact sets, a(t, y(t), z(t)) → 0 inL∞(0, T )m when z → 0 in L∞(0, T )m. So, (2.96) holds. Writing it with p = 1, we obtain that(2.95) holds for p = 1. Let it hold for some j < p. Then for y, w, z in L∞(0, T )n, we have a.e.:

(2.100)(DjF (y + w)(z)j)(t) = Dj

yjf(t, y(t) + w(t))(z(t))j

= (DjyjF (y)(z)j)(t) +Dj+1

yj+1f(t, y(t))(w(t), (z(t))j) + ρ(t)

with |ρ(t)| = o(|w(t)||z(t)|j) uniformly in time. This proves (2.95) for j + 1, and so, the resultfollows by induction.

Analyzing the above proof we may guess that smoothness w.r.t. time can be weakened,provided that the functions are measurable and that the property of uniformly small remaindersin (2.96) holds.

Definition 2.37. Let gt be a family of mappings Rn → Rm, defined for a.a. t ∈ (0, T ). Wesay that gt is a Carathéodory mapping if t 7→ gt(x) is measurable for each x ∈ X, and x 7→ gt(x)is continuous for a.a. t.(ii) We say that g has a uniform (in time) modulus of continuity over bounded sets if for any

26

R > 0, there exists a nondecreasing function ωR : R+ → R+∪+∞, such that ωR(ε)→ 0 whenε ↓ 0, and

(2.101) |gt(x)− gt(y)| ≤ ωR(|x− y|), whenever |x| < R and |y| < R.

Lemma 2.38. Let gt be a Carathéodory function. Let f : [0, T ] → Rm be measurable. Thent 7→ gt(f(t)) is measurable.

Proof. (a) We start with the case when f is a simple function, say f(t) =∑

k akχAk(t),where the sum is finite, the Ak are measurable subsets of [0, T ], with negligible intersection, andχAk is the characteristic function of Ak (with value 1 over Ak and 0 otherwise). Then

(2.102) gt(f(t)) =∑k

gt(ak)χAk(t)

is measurable (being a sum of products of measurable mappings).(b) Since any measurable function f : [0, T ] → Rm is the limit a.e. of simple functions say fj ,and since gt(·) is a.e. continuous, we have that gt(f(t)) = limj gt(fj(t)) a.e., and the limit a.e.of measurable functions is measurable.

Definition 2.39. We say that f : R×Rn → Rm is uniformly quasi Cp, for p ∈ N, if f(t, x)is a measurable function of time for each x, is a Cp function of x for a.a. t, such that for any0 ≤ j ≤ p, the function Djf (partial derivative j times w.r.t. x of f):

(1) has a uniform (over time) modulus of continuity w.r.t. x, continuous at 0, on boundedsets,

(2) for fixed x, is an essentially bounded function of time.

Remark 2.40. A uniformly quasi Cp function is a Carathéodory function. This is also truefor Djf for j ≤ p (using the fact that derivatives are limits of quotients). From the inequality|f(t, x)| ≤ |f(t, 0)| + |f(t, x) − f(t, 0)| we easily deduce that a uniformly quasi Cp function isbounded over bounded sets.

Lemma 2.41. The conclusion of lemma 2.36 still holds if we assume only that f is uniformlyquasi Cp.

Proof. Follows from the previously discussed arguments.

3. Optimal control setting

3.1. Weak derivatives.

Definition 2.42. Let D(0, T ;Rn) denote the set of C∞ function with compact support in(0, T ) and value in Rn, and let y ∈ L1(0, T ;Rn). We say that g ∈ L1(0, T,Rn) is the weakderivative of y if it satisfies

(2.103)∫ T

0y(t) · ϕ(t)dt+

∫ T

0g(t) · ϕ(t)dt = 0, for all ϕ ∈ D(0, T ;Rn).

Lemma 2.43. The weak derivative is unique: given y ∈ L1(0, T ;Rn), there is at most oneg ∈ L1(0, T,Rn) satisfying (2.103).

Proof. (a) It is enough to consider the scalar case n = 1. Let g′ and g′′ be weak derivativesof y. Then g := g′′ − g′ satisfies

(2.104)∫ T

0g(t) · ϕ(t)dt = 0, for all ϕ ∈ D(0, T ;Rn).

A quick conclusion is obtained in the case when g ∈ L2(0, T ); since D(0, T ;Rn) is known to bea dense subset of L2(0, T ), there exists a sequence ϕk ∈ D(0, T ;Rn) converging to g in L2(0, T ),so that

(2.105)∫ T

0g(t)2dt = lim

k

∫ T

0g(t) · ϕk(t)dt = 0

27

which implies that g = 0 so that the result holds.(b) In the general case, observe that if g 6= 0, there exists ε > 0 and a measurable subset A of(0, T ), of positive measure, such that (changing g into −g if necessary), g(t) > ε a.e. on A. It isknown that there exists a compact setK ⊂ A with positive measure, see [7, Ch. 1]. Obviously wecan take K as a subset of [ε1, T−ε1] for some ε1 > 0. Set Ψk(t) := (1−k distK(t))+, where distKdenotes the distance to the set K. By the dominated convergence theorem,

∫ T0 g(t)Ψk(t)dt →∫

K g(t)dt > 0. So there exists k such that∫ T

0 g(t)Ψk(t)dt > 0. Observe that Ψk is continuousand (taking k large enough) with compact support in (0, T ). So there exists a sequence ϕ` inD(0, T ;Rn) that uniformly converges to Ψk. By the dominated convergence teorem,

(2.106) 0 <

∫ T

0g(t)Ψk(t)dt = lim

`

∫ T

0g(t)ϕ`(t)dt,

but the r.h.s. is zero by the definition, which gives the desired contradiction.

Conversely, a given weak derivative is the one of a unique function up to a constant, as showsthe following result : we give a proof in the L2 setting.

Lemma 2.44 (Du Bois-Reymond). Let y ∈ L2(0, T,Rn) be such that

(2.107)∫ T

0y(t) · ϕ(t)dt = 0, for all ϕ ∈ D(0, T ;Rn).

Then y is constant.

Proof. We may assume that n = 1. If y satisfies (2.107), then so does the functiony − (1/T )

∫ T0 y(t)dt. So, we may assume that

∫ T0 y(t)dt = 0. Since D(0, T ) is a dense subset

of L2(0, T ), there exist a sequence ψk in D(0, T ) converging to y in L2(0, T ), so that αk :=∫ T0 ψk(t)dt → 0. Let η ∈ D(0, T ) have a unit integral. Then ψ′k(t) = ψk(t) − αkη(t) is anothersequence in D(0, T ) converging to y in L2(0, T ), with zero integral. The primitives ϕk(t) :=∫ t

0 ψ′k(s)ds also belong to D(0, T ), so that by hypothesis

(2.108) 0 = limk

∫ T

0y(t)ϕk(t)dt = lim

k

∫ T

0y(t)ψ′k(t)dt→

∫ T

0y2(t)dt.

The conclusion follows.

For s ∈ [1,∞] we define the Sobolev space

(2.109) W 1,s(0, T,Rn) := y ∈ Ls(0, T,Rn); there exists y ∈ Ls(0, T,Rn),where here y denotes the weak derivative of y, endowed with the norm

(2.110) ‖y‖W 1,s(0,T,Rn) := ‖y‖Ls(0,T,Rn) + ‖y‖Ls(0,T,Rn).

One easily checks thatW 1,s(0, T,Rn) is a Banach space. The following is well-known, see Royden[37, Ch. 5]:

Lemma 2.45. Let y ∈ L1(0, T,Rn) and s ∈ [1,∞]. Then y is the primitive of a function gin Ls(0, T,Rn) iff y ∈W 1,s(0, T,Rn), and then g is the weak derivative of y.

3.2. Controlled dynamical system and associated cost. Consider a controlled dynam-ical systems of the type

(2.111)

(i) y(t) = f(t,u(t),y(t)), for a.a. t ∈ [0, T ];(ii) y(0) = y0.

The data are the horizon or final time T > 0, the initial condition y0 ∈ Rn, and the dynamics

(2.112) f : Rm × Rn → Rn, uniformly quasi Cr, r ≥ 1, Lipschitz w.r.t. y.

We call u(t) and y(t) the control and state at time t. The control and state spaces are

(2.113) U := L∞(0, T,Rm); Y := W 1,∞(0, T,Rn).

28

We may see (2.111)(i) as an equality in L∞(0, T )n. By the Cauchy-Lipschitz theorem(adapted to the uniformly quasi Cr setting, with the same proof based on a fixed-point the-orem) for all (u, y0) ∈ U × Rn, the state equation (2.111) has a unique solution in Y, denotedby y[u, y0] (or y[u] if y0 is fixed). In addition, if f is uniformly Lipschitz in u, by Gronwall’slemma, for some Cf depending only on the Lipschitz constant of f :

(2.114) ‖y[u′, (y0)′]− y[u, y0]‖∞ ≤ Cf(‖u′ − u‖1 + |(y0)′ − y0|

).

We denote by z[v, z0], or z[v] if z0 = 0, the unique solution of the linearized state equation

(2.115)

(i) z(t) = f ′(t,u(t),y(t))(v(t), z(t)), for a.a. t ∈ [0, T ];(ii) z(0) = z0.

Here, by f ′(t, u, y), we denote the partial derivative of f w.r.t. (u, y). The mapping z[v, z0] iswell defined and continuous U ×Rn → Y. It has a continuous extension2 from Ls(0, T,Rm)×Rninto W 1,s(0, T,Rn), for any s in [1,∞].

Proposition 2.46. The mapping U × Rn → Y, (u, y0) 7→ y[u, y0], is of class Cr.

Proof. Let F be the mapping U × Y × Rn to L∞(0, T,Rn) × Rn, that with (u,y, y0)associates the state equation (2.111), that is,

(2.116) F(u,y, y0) :=

(y(t)− f(t,u(t),y(t)), t ∈ (0, T )y(0)− y0

).

By lemma 2.41, F is of class Cr. So the conclusion will follow from the implicit function theorem,provided that the partial derivative of F w.r.t. the state is invertible. This holds iff, for any(g, e) ∈ L∞(0, T,Rn) × Rn, the following variant of the linearized state equation (2.115) has aunique solution in Y:

(2.117) (i) z(t) = f ′(u(t),y(t))(v(t), z(t)) + g(t), for a.a. t ∈ [0, T ]; (ii) z(0) = e,

which obviously is the case. The conclusion follows.

We say that (u,y) ∈ U × Y is a trajectory if y = y[u,y(0)]. In the sequel we perform ananalysis around the nominal trajectory (u, y). We denote

(2.118) f(t) := f(t, u(t), y(t)), f ′(t) := (f ′u(t, u(t), y(t)), f ′y(t, u(t), y(t))),

the nominal dynamics and its derivative w.r.t. control and state, with a similar convention forderivatives at any order of (u(t), y(t)), the partial derivatives being denoted by e.g. fu(t). Byproposition 2.46, z[v, z0] is the directional derivative of y[u, y0] at the point (u, y0) in direction(v, z0) ∈ U × Rn, i.e.,

(2.119) z[v, z0] = lims→0

1

s(y[u + sv, y0 + sz0]− y[u, y0]).

With the controlled system (2.111) we associated the cost function

(2.120) J(u,y) :=

∫ T

0`(t,u(t),y(t))dt+ ϕ(y(T )),

sum of an integral cost, with integrand `, and of a final cost ϕ; recalling the hypothesis on f , weassume that

(2.121) f and ` are uniformly quasi Cr, ϕ is Cr.

Since a composition of Cr mappings is Cr, J : U × Y → R is Cr, as well as the reduced cost

(2.122) JR(u, y0) := J(u,y[u, y0]).

2Let X and Y be Banach spaces and E be a dense vector subspace of X. Let A be a linear mapping E → Y ,such that for some c > 0, ‖Ax‖Y ≤ c‖x‖X , for all x ∈ E. Then there exists a unique A′ ∈ L(X,Y ) that extendsA, i.e., such that A′x = Ax for all x ∈ E, and A′ satisfies ‖A′‖ ≤ c. We call A′ the continuous extension of A.

29

In addition, for (v, z0) ∈ U × Rn, writing z = z[v, z0], the expression of the derivative of thereduced cost is (adopting the notations ¯, ¯′ similar to f , f ′):

(2.123) J ′R(u, y0)(v, z0) =

∫ T

0

¯′(t)(v(t), z(t))dt+ ϕ′(y(T ))z(T ).

Our next step is to consider local (in time) control constraints of the type

(2.124) u(t) ∈ Uad, for a.a. t ∈ [0, T ],

where Uad is a closed subset of Rm. Typical examples are the case of bound constraints, andthe closed unit ball:

(2.125) Πmi=1[ai, bi]; B :=

u ∈ Rm;

m∑i=1

u2i ≤ 1

;

3.3. Derivative of the cost function. The reduction Lagrangian (defined in (2.64)) hasin the present setting the following expression:

(2.126) J(u,y) +

∫ T

0p(t) · (f(t,u(t),y(t))− y(t)) dt+ q · (y(0)− y0).

Here we choose L∞(0, T ;Rn) as space for the state equation. So, a priori, we search for a costatein the dual space. But we assume more regularity, namely p ∈ Y, so that the above integralmakes sense. Define the pre-Hamiltonian H : Rm × Rn × Rn → R by

(2.127) H(t, u, y, p) := `(t, u, y) + p · f(t, u, y).

Integrating by parts, we can rewrite the reduction Lagrangian (2.126) as

(2.128)∫ T

0(H(t,u(t),y(t),p(t)) + p(t) · y(t)) dt+ϕ(y(T ))−p(T )·y(T )+(p(0)+q)·y(0)−q·y0.

Computing this amount at the point (u(t), y(t), p(t)), and setting to zero its derivative w.r.t. yin an arbitrary direction z ∈ Y, we obtain the relation

(2.129)0 =

∫ T

0

(∇yH(t, u(t), y(t), p(t)) + ˙p(t)

)· z(t)dt

+(∇ϕ(y(T ))− p(T )) · z(T ) + (p(0) + q) · z(0).

Taking z arbitrary with zero values at time 0 and T (which is a dense subset of L2(0, T ;Rn)) wesee that the above integrand has to be equal to zero, and then taking z(0) and z(T ) arbitrarywe see that q = −p(0) and that the costate p ∈ Y, is solution of the costate!equation

(2.130)

(i) − ˙p(t) = ∇yH(t, u(t), y(t), p(t)), for a.a. t ∈ [0, T ];(ii) p(T ) = ∇ϕ(y(T )).

Given the trajectory (u, y), this equation is backwards (the final condition is given) and has aunique solution in Y. We adopt the notation H(t) in the spirit of (2.118), i.e., for instance:

(2.131) H(t) := H(t, u(t), y(t), p(t)), ∇yH(t) := ∇yH(t, u(t), y(t), p(t)).

Note that the r.h.s. of the costate dynamics is

(2.132) ∇yH(t) = ∇y ¯(t) + fy(t)†p(t),

Similarly, we have that

(2.133) ∇uH(t) = ∇u ¯(t) + fu(t)†p(t).

It follows from (2.132) that, for any z ∈ Y:

(2.134) ∇yH(t) · z(t) = ∇y ¯(t) · z(t) + p(t)†fy(t)z(t).

30

Lemma 2.47. The reduced cost JR, defined in (2.122), has a derivative at (u, y0) character-ized by

(2.135) J ′R(u, y0)(v, z0) = p(0) · z0 +

∫ T

0∇uH(t) · v(t)dt.

Proof. This follows from the theory of reduction Lagrangian, see lemma 2.23, but we willgive a direct argument. Using (2.115) and (2.130)-(2.134), we get:

∇ϕ(y(T )) · z(T ) = p(T ) · z(T )

= p(0) · z(0) +

∫ T

0

d

dt(p(t) · z(t)) dt

= p(0) · z(0) +

∫ T

0

(˙p(t) · z(t) + p(t) · z(t)

)dt

= p(0) · z(0) +

∫ T

0(p(t)†fu(t)v(t)−∇y ¯(t) · z(t))dt.

The result follows using (2.123) and the expression of ∇uH(t) given in (2.133).

We say that (u, y0) is a local minimum point of JR in U × Rn if

(2.136) JR(u, y0) ≤ JR(u, y0), if ‖u− u‖∞ + |y0 − y0| is small enough.

We define in the same way local minima of arbitrary function, paying attention to the fact thatthe definition depends on the norm. In the case of the space U × Rn, local minimum pointsare also called weak minima. Since the directional derivatives of a function (if they exists) arenonnegative at a local minimum point, we deduce from lemma 2.47 that:

Theorem 2.48. (i) If u is a local minimum point of JR(·, y0), then

(2.137) Hu(t, u(t), y(t), p(t)) = 0, for a.a. t ∈ [0, T ].

(ii) If y0 is a local minimum point of JR(u, ·), then p(0) = 0.

Example: quadratic regulator. Consider the case with dynamics f(u, y) := Ay + Bu, costintegrand `(u, y) := 1

2(u†Ru + y†Qy), final cost ϕ(y) := 12y†QT y, and fixed initial state, the

matrices R, Q, QT being symmetric. The costate equation is

(2.138)− ˙p(t) = A†p(t) +Qy(t), for a.a. t ∈ [0, T ],p(T ) = QT y(T ),

and the local optimality condition (2.137) reads

(2.139) Ru(t) +B†p(t) = 0, for a.a. t ∈ [0, T ].

If R is invertible, eliminating the control with the previous equation, we see that (y, p) is solutionof the two point boundary value problem

(2.140)

y(t) = Ay(t)−BR−1B†p(t), for a.a. t ∈ [0, T ],−p(t) = A†p(t) +Qy(t), for a.a. t ∈ [0, T ],y(0) = y0; p(T ) = QTy(T ).

3.4. Optimality conditions with control constraints. We assume here that the initialstate is given, and that there are only control constraints of type (2.124). Eliminating the initialstate as argument of the reduced cost, we can write the reduced problem in the form

(2.141) MinuJR(u) s.t. (2.124).

The set of admissible, or feasible controls is

(2.142) Uad := u ∈ U ; u(t) ∈ Uad for a.a. tWe assume in this subsection that

(2.143) Uad is a closed convex set.

31

Lemma 2.49. The set Uad is a closed convex subset of U .

Proof. The convexity of Uad follows from the one of Uad. Now let uk → u in U , uk ∈ Uad.From any subsequence we can extract another subsquence converging a.e., for which, by thedominated convergence theorem,

∫ T0 dist(u(t), Uad)dt is the limit of

∫ T0 dist(uk(t), Uad)dt, equal

to 0. The result follows.

Proposition 2.50. Let u be a weak minimum (local minimum in U) of (2.141). Then

(2.144) Hu(t)(u− u(t)) ≥ 0, for all u ∈ Uad, for a.a. t ∈ [0, T ].

Proof. (a) Let uk be a dense sequence in Uad. We first check that (2.144) is equivalentto

(2.145) Hu(t)(uk − u(t)) ≥ 0, for all k ∈ N, for a.a. t ∈ [0, T ].

Obviously (2.144) implies (2.145). Conversely, set Ik := t ∈ [0, T ]; Hu(t)(uk − u(t)) < 0. If(2.145) holds, the set I := ∪kIk has null measure, being a countable union of null measure sets.Over [0, T ]\ I, we have Hu(t)(uk− u(t)) ≥ 0 for all k, and so Hu(t)(u− u(t)) ≥ 0 for all u ∈ Uadsince uk is a dense sequence in Uad, and so, (2.144) holds.(b) Let u be admissible, and s ∈]0, 1[. Then us := u+ s(u− u) is admissible since it is a convexcombination of u and u, and hence, when s is small enough, JR(u) ≤ JR(us). By lemma 2.47 :

(2.146) 0 ≤ lims↓0

JR(us)− JR(u)

s= J ′R(u)(u− u) =

∫ T

0Hu(t)(u(t)− u(t))dt.

(c) If (2.145) does not hold, there exists E ⊂ [0, T ] measurable with positive measure and k ∈ Nsuch that Hu(t)(uk − u(t)) < 0 a.e. on E. Define u ∈ U by u(t) = uk if t ∈ E, and u(t) = uotherwise. Then

(2.147) J ′R(u)(u− u) =

∫EHu(t)(uk − u(t))dt < 0,

contradicting (2.146). The conclusion follows.

We obtain a characterization of optimality in the case of a convex problem.

Corollary 2.51. Let u 7→ JR(u) be convex. Then u is solution of (2.141) iff (2.144).

Proof. By proposition 2.50, (2.144) is a necessary condition for optimality. Assume nowthat (2.144) holds. Let u ∈ Uad, and set v := u − u. Using the convexity of JR, lemma 2.47,and (2.144), we obtain:

(2.148) JR(u)− JR(u) ≥ DJR(u)v =

∫ T

0H(t)v(t)dt ≥ 0.

The conclusion follows.

Remark 2.52. A sufficient condition for the convexity of JR is that f is an affine functionof (u, y), ` is for a.a. t a convex function of (u, y), and ϕ is convex.

Remark 2.53. (i) If Uad = Rm, (2.144) boils down to (2.137).(ii) The relation (2.144) is often written in the form

(2.149) −∇uH(t) ∈ NUad(u(t)), for a.a. t ∈ [0, T ].

We can also say that u(t), for a.a. t, attains the minimum of u 7→ Hu(t)u, for u ∈ Uad.

We easily extend the previous results to the case when the initial state is not fixed, butsubject to a constraint of the form

(2.150) y0 ∈ K0,

where K0 is a convex subset of Rn. The optimal control problem can be stated as

(2.151) Minu,y0

JR(u, y0) s.t. (2.124) and (2.150).

32

The proof of the next lemma is left as an exercice.

Lemma 2.54. If (u, y0) is a local solution of (2.151), then (2.144) holds, as well as thecondition

(2.152) p(0) · (y0 − y0) ≥ 0, for all y0 ∈ K0.

Conversely, if JR is convex and both (2.144) and (2.152) hold, then (u, y0) is solution of (2.151).

4. Examples

4.1. Multiple integrators.4.1.1. Example: double integrator, quadratic energy. Consider the following problem. The

state equation is

(2.153) h = v; v = u,

with h(t) the distance, v(t) the velocity and u(t) the acceleration. The initial state is given, andthe cost function is 1

2

∫ T0 u(t)2dt+ϕ(h(T ), v(T )). The pre-Hamiltonian is H = 1

2u2 + phv+ pvu.

The costate equation is

(2.154)

−ph = 0;−pv = ph;

ph(T ) = ∇hϕ(h(T ), v(T ));pv(T ) = ∇vϕ(h(T ), v(T )).

So ph is constant, and u = −pv is an affine function of time. It follows that the optimal state isa cubic function of time.

As an illustration, consider the case when ϕ(h, y) := 12h

2: the criterion is to minimizedistance to 0. Then

(2.155) ph(t) = ph(T ) = h(T ); pv(T ) = 0 ⇒ u(t) = −pv(t) = (t− T )h(T ).

Therefore

(2.156) v(t) = v0 + (12 t

2 − tT )h(T ); h(t) = h0 + tv0 + (16 t

3 − 12 t

2T )h(T )

Setting t = T we can compute h(T ) = (h0 + Tv0)/(1 + T 3/3), which in turn allows to computethe optimal control.

Exercice 2.55. Extend the analysis to the case of n integrations.

4.2. Control constraints.

Example 2.56. Bound constraints: Uad = Πmi=1[ai, bi] with bi > ai. We have for a.a. t, and

for i = 1 to m : Hui(t) ≥ 0 if ui(t) = ai, Hui(t) ≤ 0 if ui(t) = bi, Hui(t) = 0 if ui(t) ∈]ai, bi[.

Example 2.57. Constraint on Euclidean norm: Uad = B (closed unit ball). We have fora.a. t, if Hu(t) 6= 0, then u(t) = −Hu(t)/|Hu(t)|.

33

3Pontryagin’s principle

Contents

1. Pontryagin’s minimum principle 351.1. The easy Pontryagin’s minimum principle 351.2. Pontryagin’s principle with two point constraints 381.3. Variations of the pre-Hamiltonian 412. Extensions 432.1. Decision variables 432.2. Variable horizon 452.3. Varying initial and final time 472.4. Interior point constraints 482.5. Minimal time problems, geodesics 503. Qualification conditions 533.1. Lagrange multipliers 533.2. Pontryagin multipliers 554. Proof of Pontryagin’s principle 564.1. Ekeland’s principle 564.2. Ekeland’s metric and the penalized problem 574.3. Proof of theorem 3.13 58

1. Pontryagin’s minimum principle

1.1. The easy Pontryagin’s minimum principle.1.1.1. Main result. Consider again problem (2.141) (with fixed initial state and control con-

straints) with now Uad a nonempty closed set, possibly nonconvex. Note that initial state isfixed and that there is no contraint on the final state. We will weaken the hypotheses, in orderto cover some cases when the dynamics and cost function are non differentiable functions of thecontrol. We assume that

(3.1)

f and ` are C1 functions of y. The functions f , fy, `, `y areCarathéodory functions, uniformly (in time) locally Lipschitz w.r.t. (u, y),and for given (u, y), essentially bounded functions of time.The final cost ϕ is C1 with a locally Lipschitz gradient.

35

Let (u, y) be a trajectory (it satisfies the state equation (2.111)). We recall that the pre-Hamiltonian is the function

(3.2) H(t, u, y, p) := `(t, u, y) + p · f(t, u, y)

and that the costate p is the unique solution of the costate equation (2.130), reproduced below:

(3.3)

(i) − ˙p(t) = ∇yH(t, u(t), y(t), p(t)), for a.a. t ∈ [0, T ],(ii) p(T ) = ∇ϕ(y(T )).

Definition 3.1. We say that (u, y) is a Pontryagin extremal if the Hamiltonian inequalitybelow holds:

(3.4) H(t, u(t), y(t), p(t)) ≤ H(t, u, y(t), p(t)) for all u ∈ Uad, for a.a. t ∈ [0, T ].

Note that (3.4) can be written in the form

(3.5) H(t, u(t), y(t), p(t)) = infu∈Uad

H(t, u, y(t), p(t)) for a.a. t ∈ [0, T ].

If the previous relation holds, we also say that (u, y, p) (or (y, p) if no ambiguity is possible) isa Pontryagin biextremal.

One easily deduces from the Hamiltonian inequality the following:

Lemma 3.2. Assume that Uad is convex. Then a Pontryagin extremal satisfies the first ordercondition (2.144).

Proof. By (3.4), for a.a. t, for any u in Uad, setting v := u− u(t) we have that:

(3.6)0 ≤ lim

σ↓0

1

σ(H(t, u(t) + σv, y(t), p(t))−H(t, u(t), y(t), p(t)))

= Hu(t, u(t), y(t), p(t))(u− u(t)).

The conclusion follows.

Pontryagin’s minimum principle (PMP) is (in our framework) the following statement:

Theorem 3.3. Let u be solution of (2.141). Set y := y[u]. Then (u, y) is a Pontryaginextremal.

We recall that the reduced cost was defined in (2.122) as

(3.7) JR(u, y0) := J(u,y[u, y0]).

The proof is based on the following lemma. Remember the definition of the control and statespaces U and Y in (2.113). In the sequel, for some M ≥ ‖u‖‖∞, set

(3.8) UM := u ∈ U ; ‖u‖∞ ≤M.

Lemma 3.4. There exists cM > 0 such that, for all u ∈ UM , we have that for some r(u) ∈ R:

(3.9) JR(u) = JR(u)+

∫ T

0(H(t,u(t), y(t), p(t))−H(t))dt+r(u), with |r(u)| ≤ cM‖u−u‖21.

Proof. Let y = y[u]. Integrating by parts and using the costate equation, we obtain

(3.10) JR(u) = JR(u) +

∫ T

0p(t) · (f(u(t),y(t))− y(t))dt = ∆1(y) + ∆2(u),

with∆1(y) := ϕ(y(T ))− ϕ′(y(T ))y(T ) + p(0) · y(0),

∆2(u) :=

∫ T

0(H(t,u(t),y(t), p(t))− Hy(t)y(t))dt.

Therefore, JR(u)−JR(u) = ∆1(y)−∆1(y)+∆2(u)−∆2(u). We have that u(t) and consequentlyy(t) remain in bounded subsets of U and Y, over which the functions f, `, ϕ and their partial

36

derivatives w.r.t. the state are uniformly Lipschitz. So, setting δy(t) := y(t)−y(t), that satisfiesδy(0) = 0:

(3.11) ∆1(y)−∆1(y) = ϕ(y(T ))− ϕ(y(T ))− ϕ′(y(T ))δy(T ) = O(|δy(T )|)2.

In addition

∆2(u)−∆2(u) =

∫ T

0(H(t,u(t), y(t), p(t))− H(t))dt+ ∆3,

where

∆3 :=

∫ T

0(H(t,u(t),y(t), p(t))−H(t,u(t), y(t), p(t))− Hy(t)δy(t))dt.

Since(3.12)

H(t,u(t),y(t), p(t))−H(t,u(t), y(t), p(t)) =

(∫ 1

0Hy(t,u(t), y(t) + sδy(t), p(t))ds

)δy(t),

we have that

(3.13) |∆3| = O ((‖u− u‖1 + ‖δy‖∞) ‖δy‖∞) .

We conclude by noting that ‖δy‖∞ = O(‖u− u‖1).

Proof of theorem 3.3. (a) We first show that, if the trajectory (u,y) is admissible, then

(3.14) H(t, u(t), y(t), p(t)) ≤ H(t,u(t), y(t), p(t)) a.e. on [0, T ].

Otherwise, there would exist ε > 0 and a measurable subset E of [0, T ] with positive measure,such that

(3.15) H(t,u(t), y(t), p(t)) ≤ H(t, u(t), y(t), p(t))− ε a.e. over E.

LetM := max(‖u‖∞, ‖u‖∞), and E′ be a measurable subset of E. Define u′ ∈ U by u′(t) = u(t)if t ∈ E′, and u′(t) = u(t) otherwise. Since ‖u′ − u‖∞ ≤ 2M , we have that ‖u′ − u‖1 ≤2M meas(E′). By Lemma 3.4,

(3.16) JR(u′) ≤ JR(u)− εmeas(E′) + 4cMM2 meas(E′)2.

This gives a contradiction by choosing E′ with a sufficiently small, positive measure.(b) Let uk be a dense sequence in Uad, and uk be the sequence in U defined by u0 := u, and

(3.17) uk(t) =

uk if H(t, uk, y(t), p(t)) < H(t,uk−1(t), y(t), p(t)),

uk−1(t) otherwise.

The density property of uk implies that H(t,uk(t), y(t), p(t))→ infu∈Uad H(t, u, y(t), p(t)) a.e.So, taking u = uk in (3.14) and passing to the limit in this inequality we get the conclusion.

Remark 3.5. If Uad = Rm, the Hamiltonian inequality (3.4) implies the following Legendre-Clebsch condition:

(3.18) Huu(t) 0 for a.a. t ∈ [0, T ].

We next introduce a new notion of minimum.

Definition 3.6. A sequence uk of U is said to converge to u in Pontryagin’s sense if thereexists M > 0 such that ‖uk‖∞ ≤ M , and that uk → u a.e. We say that u is a Pontryaginminimum if uk → u in Pontryagin’s sense implies that JR(u) ≤ JR(uk) for k large enough.

Remark 3.7. If uk → u in Pontryagin’s sense, by the dominated convergence theorem,uk → u in Ls(0, T )m for all s ∈ [1,∞[.

Remark 3.8. One easily checks that the proof of theorem 3.3 holds for a Pontryagin mini-mum. With any Pontryagin minimum is therefore associated a Pontryagin extremal.

37

1.1.2. A related nonlinear expansion of the state. We consider the nonlinear relation

(3.19) ξ(t) = fy(t)ξ(t) + f(t,u(t), y(t))− f(t, u(t), y(t)), for a.a. t ∈ (0, T ),

with u ∈ U , and initial condition ξ(0) = 0, whose solution in Y may viewed as a function of u,denoted by ξ[u].

Lemma 3.9. For every M > 0, there exists cM > 0 such that, if ‖u‖∞ ≤M , then

(3.20) ‖y[u]− y − ξ‖∞ ≤ cM‖u− u‖21.

Proof. We may assume that the data are autonomous. Denote y[u] by y, and let η :=y − y − ξ. Then, skipping time arguments

(3.21)η = f(u,y)− f(u, y)− fyξ

= fyη + f(u,y)− f(u, y)− fy(y − y)

= fyη +A(y − y)

where

(3.22) A = A(t) :=

∫ 1

0(fy(u, y + σ(y − y))− fy)dσ

is a n × n matrix, and since the state is uniformly bounded whenever ‖u‖∞ ≤ M , there existsc′M > 0 not depending on u such that

(3.23) |A(t)| ≤ c′M (|u(t)− u(t)|+ |y(t)− y(t)|).

Since ‖y − y(t)‖∞ = O(‖u− u(t)‖1), the conclusion follows with Gronwall’s lemma.

Alternative proof of lemma 3.4. It is enough to consider the autonomous case, withonly a final cost. Then, by (2.114) and (3.20):

(3.24)JR(u)− JR(u) = ∇ϕ(y(T )) · (y(T )− y(T )) +O(|y(T )− y(T )|2),

= p(T ) · ξ(T ) +O(|u(T )− u(T )|21).

Also,

(3.25) p(T ) ·ξ(T ) =

∫ T

0

(˙p(T ) · ξ(T ) + p(T ) · ξ(T )

)dt =

∫ T

0(H(t,u(t), y(t), p(t))− H(t))dt.

The result follows.

1.2. Pontryagin’s principle with two point constraints.1.2.1. Main result. We now allow the initial and final state to vary, subject to some con-

straints that may couple them (as in the case of periodic trajectories). So, consider the costfunction

(3.26) JIF (u,y) :=

∫ T

0`(t,u(t),y(t))dt+ ϕ(y(0),y(T )),

where ’IF’ stands for initial-final, and the two point constraints

(3.27) Φ(y(0),y(T )) ∈ KΦ.

The hypotheses on the data are as follows:

(3.28)

Hypothesis (3.1) holds and ϕ, Φ are C1 with locally Lipschitz derivatives.The set KΦ is a nonempty, closed convex subset of RnΦ .

Consider the optimal control problem

(3.29)

Minu,y JIF (u,y); s.t. the state equation (2.111)(i),control constraints (2.124), and (3.27).

38

The pre-Hamiltonian, now expressed in non qualified form, is the function H : R+ × R× Rm ×Rn × Rn → R defined by

(3.30) H(β, t, u, y, p) := β`(t, u, y) + p · f(t, u, y).

We call β the cost multiplier. Denoting by Ψ ∈ RnΨ the multiplier associated with the two pointconstraints. The end points Lagrangian is LIF : R+ × Rn × Rn × RnΦ defined by

(3.31) LIF (β, y0, yT ,Ψ) := βϕ(y0, yT ) + Ψ · Φ(y0, yT ).

The Lagrangian of problem (3.29) (sum of weighted cost function and constraints) is(3.32)

L(β,u,y,p,Ψ) := βJIF (u,y) +

∫ T

0p(t) · (f(t,u(t),y(t))− y(t))dt+ Ψ · Φ(y(0),y(T ))

=

∫ T

0(H(β, t,u(t),y(t),p(t))− p(t) · y(t)) dt+ LIF (β,y(0),y(T ),Ψ).

As usually we will obtain a costate equation by expressing the condition of stationarity (i.e.,zero partial derivative) of the Lagrangian w.r.t. the state. After an integration by parts (validsince we assume that p ∈ Y), this boils down to the fact that, for all z ∈ Y :

(3.33)DyLz =

∫ T

0

(∇yH(β, t, u(t), y(t), p(t)) + ˙p(t)

)· z(t)dt

+ (∇y0LIF (β, y(0), y(T ), Ψ) + p(0)) · z(0)

+ (∇yTLIF (β, y(0), y(T ), Ψ)− p(T )) · z(T ) = 0.

By the same arguments than those following (2.128), the coefficients of the linearized state oneach line must be zero. So, let us define the costate equation as

(3.34)

(i) − ˙p(t) = ∇yH(β, t, u(t), y(t), p(t)), for a.a. t ∈ [0, T ],(ii) −p(0) = ∇y0LIF (β, y(0), y(T ), Ψ),(iii) p(T ) = ∇yTLIF (β, y(0), y(T ), Ψ).

Remark 3.10. For historical reasons, the above two last lines are called transversality con-ditions.

Remark 3.11. By the costate equation, if z = z[v, z0], the solution of the linearized stateequation (2.115), then

(3.35)βϕ′(y(0), y(T ))(y(T ))(z0, z(T )) + Ψ · Φ′(y(0), y(T ))(y(T ))(z0, z(T ))

+β∫ T

0 `′(u(t), y(t))(v(t), z(T ))dt =∫ T

0 Hu(t)v(t)dt.

Definition 3.12. We call Pontryagin multiplier associated to the nominal trajectory (u, y),any triple λ := (β, Ψ, p) verifying (3.34), as well as

(3.36) β ∈ 0, 1; Ψ ∈ NKΦ(Φ(y(0), y(T ))),

the non nullity relation

(3.37) β + |Ψ| > 0,

and the Hamiltonian inequality that generalizes (3.5) :

(3.38) H(β, u(t), y(t), p(t)) = infu∈Uad

H(β, u, y(t), p(t)) for a.a. t ∈ (0, T ).

We say that (u, y) is a Pontryagin extremal if the set ΛP (u, y) of associated Pontryagin multi-pliers is non empty.

Relation (3.37) prevents a multiplier to be zero. We say that an element of ΛP (u, y) is asingular or abnormal multiplier if β = 0, and a normal (Pontryagin) multiplier if β = 1.

Theorem 3.13. A solution of (3.29) is a Pontryagin extremal.

Proof. The proof being technical, we postpone it to section 4.

39

Example 3.14. Let us apply theorem 3.13 to the probem of minimizing∫ 1

0 u(t)(1−u(t))dts.t. y(t) = u(t) ∈ [0, 1] for a.a. t ∈ (0, 1), y(0) = 0, and y(0) = 1/2. The pre-Hamiltonian isH = βu(1− u) + pu so that − ˙p(t) = 0 a.e., whence p(t) = Ψ. If β = 0 then Ψ 6= 0 and so, u(t)is constant, either equal to 0 or 1, in contradiction with the initial-final state constraints. So,β = 1, Ψ = 0, and since H is a strictly concave function of the control, we have that u(t) ∈ 0, 1for a.a. t. By the initial-final state constraint, meas(t; u(t) = 0) = meas(t; u(t) = 1) = 1/2.In this simple nonconvex example, one can check that any control satsfyinhg these necessaryconditions is optimal.

1.2.2. Specific structures. We next consider several examples that illlustrate the above the-orem.

Example 3.15. Assume that we have finitely many initial-final equality and inequalityconstraints, i.e. (with obvious interpretation if either n1 or n2 equals zero):

(3.39)

Φi(y(0),y(T )) = 0, i = 1, . . . , n1,Φi(y(0),y(T )) ≤ 0, i = n1 + 1, . . . , n1 + n2.

This corresponds to the choice

(3.40) KΦ := 0Rn1 × Rn2− .

In that case we easily check that Ψ ∈ NKΦ(Φ(y(0),y(T ))) iff the following sign and complemen-

tarity conditions for inequality constraints hold:

(3.41) Ψi ≥ 0, ΨiΦi(y(0),y(T )) = 0, i = n1 + 1, . . . , n1 + n2.

Example 3.16. Problem (2.141), that has a fixed inital state and a free final state, is aparticular case of the present setting where ϕ depends only on the final state, Φ(y0, yT ) = y0,and KΦ is the singleton y0. The two point conditions of the costate equation reduce then to

(3.42) p(T ) = β∇ϕ(y(T )); p(0) = −Ψ.

The last relation expresses Ψ as a function of p(0). If β was equal to 0, we would have thatp(T ) = 0, and then p = 0 by the costate equation, so that Ψ = 0, contradicting (3.37).Therefore, there exists a regular Pontryagin multiplier, and no singular Pontryagin multiplier.We recover the conclusion of theorem 3.3.

Example 3.17. More generally, if the initial state is fixed, we may write ϕ as function ofthe final state only, and assume the two point constraints of the form

(3.43) Φ(y0, yT ) = (y0,ΦF (yT )); KΦ = y0Rn ×KF ,

with ΦF : Rn → RnF andKF convex and closed subset of RnF . Denoting by (Ψ0, ΨF ) ∈ Rn×RnFthe components of the multiplier Ψ associated with the initial and final state constraint, we mayreplace the transversality conditions (3.34(ii)-(iii)) by

(3.44)

p(T ) = β∇ϕ(y(T )) +DΦF (y(T ))†ΨF ;

ΨF ∈ NKF (ΦF (y(T ))),

−p(0) = Ψ0,

We see then that we can replace the non nullity condition (3.37) by

(3.45) β + |ΨF | > 0.

Indeed, if β = 0 and |ΨF | = 0, then p(t) = 0 for all t (since its final condition is zero and it issolution of an homgeneous linear ordinary differential equation). Then Ψ0 appears only in ther.h.s. of the initial condition for the costate, so that this initial condition brings no informationand we usually do not state it when dealing with problems with given initial state.

40

Example 3.18. Another generalization of example 3.16 is when we assume only that thefinal state is free, and we have the initial constraint Φ(y(0)) ∈ KΦ. If β = 0, then p(T ) =0, and by the costate equation, p(t) = 0 for all t ∈ [0, T ]. So, 0 = p(0) = DΦ(y(0))†Ψ,with 0 6= Ψ ∈ NKΦ

(y(0)). If DΦ(y(0)) is surjective, then its transpose is injective and thisgives a contradiction with (3.37). So, if DΦ(y(0)) is surjective, there is no singular Pontryaginmultiplier, and there exists a regular one, i.e., we must have β = 1.

1.2.3. Link with Lagrange multipliers; convex problems. When Uad is convex, we easily de-duce from the Hamiltonian inequality (3.38) that

(3.46)

For a.a. t ∈ [0, T ], we have thatHu(β, t, u(t), y(t), p(t))(u− u(t)) ≥ 0, for all u ∈ Uad.

Definition 3.19. When Uad is convex, we define the set ΛL(u, y) of Lagrange multipliersas the set of triple λ := (β, Ψ, p) satisfying (3.34)-(3.37) and (3.46). If β = 0 (resp. β = 1) wesay that the Lagrange multiplier is singular (resp. normal).

Since (3.38) implies (3.46), the following holds:

(3.47) If Uad is convex, then ΛP (u, y) ⊂ ΛL(u, y).

Lemma 3.20. Let λ := (β, Ψ, p) be a regular (β = 1) Lagrange multiplier, such that theassociated Lagrangian defined in (3.32) is a convex function of (u,y). Then (u, y) is solution ofproblem (3.29).

Proof. Let (u,y) satisfy the constraints of problem (3.29). Skipping the argument β, weget:

(3.48)JIF (u,y)− JIF (u, y) ≥ L(u,y, p, Ψ)− L(u, y, p, Ψ)

≥ DuL(u, y, p, Ψ)(u− u)

=∫ T

0 H(t)(u(t)− u(t))dt ≥ 0.

The first inequality follows from the state constraint and the relation

(3.49) Ψ · (Φ(y(0),y(T ))− Φ(y(0), y(T ))) ≤ 0,

which holds since Ψ ∈ NKΦ(Φ(y(0), y(T ))) and Φ(y(0),y(T )) ∈ KΦ. The second inequality

follows from the convexity hypothesis for the Lagrangian, remembering that the partial derivativeof the Lagrangian w.r.t. the state variable is zero (this is how is derived the costate equation),and then we use the definition of a Lagrange multiplier. The result follows.

Remark 3.21. For a sufficient condition for the convexity hypothesis on the Lagrangian wecan make the same hypotheses as in remark 2.52. It remains then to check that

(3.50) (y0, yT ) 7→ Ψ · Φ(y0, yT ) is convex.

This always holds if Φ is an affine mapping. In the case of n1 equality constraints and n2

inequality constraints on the initial and final state, i.e. when KΦ is of the form (3.39), sinceΨi ≥ 0 whenever i > n1 by (3.41), we see that (3.50) holds if the first n1 components of Φ areaffine, and the remaining ones are convex.

1.3. Variations of the pre-Hamiltonian.1.3.1. Pontryagin extremal over (0, T ). Let (u, y) be a trajectory, and p ∈ Y satisfy the first

part (3.34)(i) of the costate equation. Set

(3.51) h(t) := infu∈Uad

H(β, t, u, y(t), p(t)).

We say that (u, y, p) is a Pontryagin extremal over (0, T ) (rather than [0, T ]) if the Hamiltonianinequality (3.38) holds, that is, if

(3.52) h(t) = H(t), for a.a. t.

41

So, we forget the constraints on initial-final state as well as the transversality conditions for thecostate. Set

(3.53) Uad := Uad ∩ B(0, ‖u‖∞); h(t) := infu∈Uad

H(β, t, u, y(t), p(t)).

Then Uad is compact, and so, since |u(t)| ≤ ‖u‖∞ a.e.:

(3.54) h(t) = h(t) for a.a. t ∈ [0, T ].

Remark 3.22. Note that y and p are Lipschitz functions of time. It easily follows from(3.54) that if f(·, u, y) has a continuity modulus (resp. Lipschitz constant) uniformly in time,over bounded sets, then h is continuous (resp. Lipschitz).

1.3.2. Constant pre-Hamiltonian for autonomous problem. We say that the optimal controlproblem is autonomous if f and ` do not depend on time, which then is also the case for thepre-Hamiltonian. In that case we say that these functions are autonomous, and we may skipthe time from their arguments.

Lemma 3.23. Let (u, y, p) be a Pontryagin extremal over (0, T ) of an autonomous problem.Then h(t) is, up to a null measure set, equal to some constant over (0, T ).

Proof. a) Consider first the easy case when u(t) is differentiable over (0, T ), in the absenceof control constraints. Then Hu(t) = 0 for all t ∈ (0, T ), and (using the state and costateequations) h(t) has a continuous derivative satisfying

(3.55) ˙h(t) = Hy(t) ˙y(t) + Hp(t) ˙p(t) = − ˙p(t) · ˙y(t) + ˙y(t) · ˙p(t) = 0,

from which the result follows.b) We now now with the general case. By remark 3.22, h is a Lipschitz function; iIt can be shownthat this is equivalent to say that h ∈ W 1,∞(0, T,Rn). By lemma 2.45, h is the primitive of itsweak derivative, and it suffices to show that h has zero derivative for a.a. t. For a.a. t0 ∈ (0, T ),we have that (h, y, p) are differentiable at t0, H(β, ·, y(t0), p(t0)) attains its minimum over U ′adat u(t0), and the state and costate equation hold (at time t0). Therefore,

(3.56)

˙h(t0) ≤ limt↓t0

H(β, u(t0), y(t), p(t))−H(β, u(t0), y(t0), p(t0))

t− t0= Hy(β, u(t0), y(t0), p(t0)) ˙y(t0) +Hp(β, u(t0), y(t0), p(t0)) ˙p(t0)

= − ˙p(t0) · ˙y(t0) + ˙p(t0) · ˙y(t0) = 0.

By taking t ↑ t0 we would prove the opposite inequality in the same way. The result follows.

Remark 3.24. In the case of state constrained problems, we will see in lemma 5.7 anothermethod of proof based on a time dilation approach.

1.3.3. Variation of the pre-Hamiltonian for non autonomous problem. We may view the timeas an additional state variable say τ with initial condition zero and dynamics τ = 1. In thisnew setting, f = f(τ, u, y) and ` = `(τ, u, y) become autonomous functions. In view of (3.1), weneed to assume that

(3.57)

f and ` are C1 functions of (t, y).The functions f , fy, ft, `, `y, `t are locally Lipschitz.ϕ and Φ are C1 with locally Lipschitz derivatives.

The new pre-Hamiltonian is H(β, τ, u, y, p) + q, the additional costate q being solution of

(3.58) − ˙q(t) = Ht(t), for a.a. t ∈ (0, T ), q(T ) = 0.

Since τ(0) is fixed, q(0) is equal to the associated Lagrange multiplier (or its opposite) whichgives no information. So, q(t) =

∫ Tt Ht(s)ds. In this autonomous reformulation, by lemma 3.23,

the pre-Hamiltonian has constant value, equal to H(t)+∫ Tt Ht(s)ds. Equivalently, the total and

partial derivative of the pre-Hamiltonian along the trajectory are equal (and bounded), so that:

42

Lemma 3.25. Let (3.57) hold. Then we have that h(t) is Lipschitz, and

(3.59)d

dth(t) = Ht(t), a.e. on (0, T ).

2. Extensions

2.1. Decision variables. Quite often some variables not depending on time have to beoptimized as well as the control, think for instance of the mass of fuel to be taken by anairplane, or some design variables. So, consider, for (u,y, π) ∈ U × Y × Rnπ , the cost function

(3.60) J(u,y, π) :=

∫ T

0`(t,u(t),y(t), π)dt+ ϕ(y(0),y(T ), π),

and the state equation

(3.61) y(t) = f(t,u(t),y(t), π), for a.a. t ∈ [0, T ].

We also have initial-final state constraints

(3.62) Φ(y(0),y(T ), π) ∈ KΦ.

Here ` and f are mappings from R×Rm×Rn×Rnπ into R and Rn resp., and ϕ : Rn×Rnπ → R.We also have local (in time) control constraints of the type

(3.63) u(t) ∈ Uad, for a.a. t ∈ [0, T ].

The hypotheses on the data are as follows:

(3.64)

f and ` are C1 functions of (y, π); the mappings f , fy, fπ,`, `y, `π areCarathéodory functions, uniformly locally Lipschitz w.r.t. (u, y, π),and for given (u, y, π), essentially bounded functions of time.ϕ, Φ are C1 with locally Lipschitz derivatives.The set KΦ is a nonempty, closed convex subset of RnΦ .Uad is a nonempty closed subset of Rm.

The optimal control problem is

(3.65) Min J(u,y, π) s.t. (3.61)-(3.63).

We can reduce this problem to the general format (3.29) by interpreting π as an additionalstate variable denoted by π, with zero dynamics. Then the augmented state is ya := (y†,π)†.Identifying in the sequel y with the first n components of the augmented state, we can write thestate equation for ya as

(3.66) ya(t) =

(f(t,u(t),y(t),π(t))

0

)for a.a. t ∈ [0, T ].

The cost function in this augmented setting is (we choose to express the initial-final cost asfunction of the final value of π):

(3.67) Ja(u,ya,π) :=

∫ T

0`(t,u(t),y(t),π(t))dt+ ϕ(y(T ),y(T ),π(T )).

The resulting optimal control problem is

(3.68) Min Ja(u,ya,π) s.t. (3.62)-(3.63) and (3.66).

We next discuss the optimality conditions of this problem. We have an augmented costatepa := (p,q), where p(t) ∈ Rn and q(t) ∈ R are costates associated with y and π, resp. Thepre-Hamiltonian and end points Lagrangian in this setting are resp.

(3.69)

H(β, t, u, y, π, p, q) := β`(t, u, y, π) + p · f(t, u, y, π),

LIF (β, y0, yT , π,Ψ) := βϕ(y0, yT , π) + Ψ · Φ(y0, yT , π).

43

Since π(t) = π for all t, the costate equation (3.34) takes here the expression

(3.70)

− ˙p(t) = ∇yH(β, t, u(t), y(t), π, p(t)), for a.a. t ∈ [0, T ],

− ˙q(t) = ∇πH(β, t, u(t), y(t), π, p(t)), for a.a. t ∈ [0, T ],

−p(0) = ∇y0LIF (β, y(0), y(T ), π, Ψ),

p(T ) = ∇yTLIF (β, y(0), y(T ), π, Ψ),

q(0) = 0,

q(T ) = ∇πLIF (β, y(0), y(T ), π, Ψ).

It follows that

(3.71)0 = q(0) = −

∫ T0

˙q(t)dt+ q(T )

=

∫ T

0∇πH(β, t, u(t), y(t), π, p(t))dt+∇πLIF (β, y(0), y(T ), π, Ψ).

This is nothing but the condition of stationarity of the Lagrangian of the optimal control problemw.r.t π. We also have the Hamiltonian inequality for the control variables. We have proved thefollowing:

Lemma 3.26. For the optimal control problem (3.65) with decision variables, the PMP holdswith the same expression as for the corresponding problem with decision variables set at theirnominal value, with in addition the condition of stationarity of the Lagrangian w.r.t the decisionvariables.

Exercice 3.27. Show that we obtain the same result when in (3.67) we choose to expressthe initial-final cost function as ϕ(y(0),y(T ),π(0)).

2.1.1. A design problem. Consider the following model of a ground vehicle with rectilineartrajectory:

(3.72) h = v; v = (eu−D(γ, v))/m; m = −u,

where h is the distance to the initial position, v is the velocity, m is the (positive) mass, thecontrol is the mass flow u ∈ [0, U ], with U > 0, e > 0 is the ejection speed, and D(v, γ) is thedrag force, function of the speed and of a design variable (think of a parametrized shape of thevehicle) γ ∈ R. We assume that D(v, γ) and Dv(v, γ) have positive values (when v(t) > 0 whichwill be the case by hypothesis).

The initial state is given, and we have a final state constraint

(3.73) h(0) = 0; v(0) = 0; m(0) = m0 > 0; m(T ) ≥ mT .

We want to maximize the final distance, but have a cost c(γ) for the design, and so we decideto minimize rc(γ)− h(T ) where r > 0 is a parameter.

We next express the PMP. The pre-Hamiltonian is

(3.74) H = phv + pv(eu−D(γ, v))/m− pmu.

The costate equation is

(3.75)

−ph = Hh = 0; ph(T ) = −β,−pv = Hv = ph − pvDv/m; pv(T ) = 0,−pm = Hm = −pv(eu−D)/m2; pm(T ) = −Ψ ≤ 0.

If β = 0 then ph(t) = 0 for all t, so that pv/pv is bounded along the trajectory. Since pv(T ) = 0,it follows that pv(t) = 0 for all t, whence −pm = 0, implying pm(t) = pm(T ) = −Ψ < 0, so that

(3.76) Hu = epv/m− pmis constant and positive, meaning that u(t) = 0 for all time, which is optimal only when mT =m0. In the sequel we exclude this case and therefore we may assume that β = 1.

44

Obviously ph(t) = −1 for all t, and we have that

(3.77) pv = 1 + pvDv/m; pv(T ) = 1; pm(T ) = 0.

Since pv(T ) = 1 and pv(T ) = 0, pv(t) < 0 for t close to T . As pv > 0 when pv is close to zero,necessarily

(3.78) One has pv(t) < 0, for all t ∈ [0, T ).

It follows that pm has the sign of D − eu. Since pv(T ) = 0, pm(T ) = 0, and so:

(3.79) Hu(T ) = Ψ ≥ 0; Hu(T ) = e/m(T ) > 0.

Case when Ψ = 0. Then Hu(t) < 0 for t close to T , and so, the trajectory has a final fullthrust arc, say over (t0, T ). We assume that D ≤ eU (i.e., the full thrust allows to increase thespeed) over the trajectory. Over the final full thrust arc, pm is positive since it decreases withfinal condition zero. Reminding that pv(t) < 0 for all t, we see that Hu(t0) < −pm(t0) < 0 sothat t0 = 0: the optimal policy is the full thrust over (0, T ).Case when Ψ > 0. Then for t close to T , Hu > 0 and so u = 0: the trajectory ends with azero thrust arc (over which the mass equals mT ) say (t0, T ). Over this arc pm increases.Optimality w.r.t. γ. The condition of stationarity of the Lagrangian w.r.t. γ gives

(3.80) rc′(γ)−∫ T

0

pv(t)

m(t)Dγ(γ, v(t))dt = 0.

We may assume that, for instance, D(γ, v) = 12γv

2, i.e., γ is the aerodynamic coefficient, andthat c′(γ) < 0 (it is expensive to achieve a small aerodynamic coefficient). Then the aboveequation reads

(3.81) rc′(γ) = 12

∫ T

0

pv(t)

m(t)v(t)2dt.

As expected, in view of (3.78), both sides have the same negative sign.

2.2. Variable horizon. In many problems, the final time is itself an optimization parame-ter. For instance, in the minimum time problem, we aim to reach a certain target in a minimumtime. So, consider the case when the initial time is zero and the initial state is fixed, the finalstate being free, and assume that T is a decision variable. The original problem is

(3.82)Minu,T

∫ T

0`(t,u(t),y(t))dt+ ϕ(y(T ))

s.t. y(t) = f(t,u(t),y(t)) a.e. in (0, T ); y(0) = y0,

u(t) ∈ Uad a.e., ΦF (y(T )) ∈ KF .

We introduce the normalized time τ := t/T , so that τ ∈ [0, 1]. Then yN (τ) := y(τT ) is thetime normalized state, with a similar conventions for other variables such as the control. Thenthe normalized control and state satisfy the (normalized) state equation

(3.83) yN (τ) = Tf(Tτ,uN (τ),yN (τ)), τ ∈ [0, 1]; yN (0) = y0.

On the other hand the cost function (3.26) can, in this setting, be expressed as

(3.84) JN (uN ,yN ) := T

∫ 1

0`(Tτ,uN (τ),yN (τ))dτ + ϕ(yN (1)).

The resulting time normalized optimal control problem is

(3.85) MinuN

JN (uN ,yN ) s.t. (3.83); uN (τ) ∈ Uad for a.a. τ , ΦF (yN (1)) ∈ KF .

The pre-Hamiltonian for the original and normalized problems are resp.

(3.86)

H(β, t, u, y, p) := β`(t, u, y) + p · f(t, u, y);

HN (β, τ, T, uN , yN , pN ) := TH(β, Tτ, uN , yN , pN ).

45

We see that the Lagrangian function of the time normalized optimal control problem is

(3.87)LN :=

∫ 1

0

(TH(β, Tτ,uN (τ),yN (τ),pN (τ))− pN (τ) · yN (τ)

)dτ

+β ϕ(yN (1)) + ΨF · ΦF (yN (1)).

The horizon T appears then as a decision variable and we can apply the analysis of section 2.2.We know that we obtain the PMP for the same problem with a fixed horizon (theorem 3.13),with in addition the stationarity of the Lagrangian of the normalized problem w.r.t. the variableT (lemma 3.26). Set

(3.88) H(τ) := H(β, Tτ,uN (τ),yN (τ),pN (τ)).

This is a.e. equal to

(3.89) hN (τ) := infu∈Uad

H(β, Tτ, u,yN (τ),pN (τ)).

The costate pN for the normalized time problem is solution of

(3.90) − dpN (τ)) = T∇yH(τ), τ ∈ (0, 1),

with final condition

(3.91) pN (1) = βN∇ϕ(yN (1)) +DΦF (yN (1))†ΨNF ; ΨNF ∈ KF (ΦF (yN (1))).

Set p(t) := pN (t/T ). It is easily checked that p(t) is solution of the costate equation for theoriginal problem with multipliers (β, ΨF ) = (βN ,ΨNF ), that ΨNF ∈ KF (ΦF (y(T ))), and that theHamiltonian inequality for the normalized time problem induces the Hamiltonian inequality forthe original one. Indeed, one easily checks that this induces a bijective application between thePontryagin extremal s of the original problem with fixed horizon, and those of the normalizedtime problem (also with a fixed horizon) as follows:

(3.92) (β, ΨF ) = (βN ,ΨNF ); p(t) = pN (t/T )), t ∈ (0, T ).

As a consequence of the PMP with a fixed horizon, we have seen in section 1.3.3 how to refor-mulate non autonomous problems (with smooth enough data) into autonomous ones, leading torelation (3.59), which in the setting of the normalized formulation gives

(3.93) τ 7→ HN (τ) is Lipschitz, anddHN (τ)

dτ=∂HN (τ)

∂τa.e.

Since H(τ) := HN (τ)/T , we deduce that

(3.94)dH(τ)

dτ=∂H(τ)

∂τ= THt(τ) a.e.

Lemma 3.28. Assume that the initial time is fixed and the final time is free, and that (3.57)holds. Then the PMP has the same expression as in the case of a fixed final time, with in addition(assuming that the final time does not enter in the final cost or constraints) the condition of zerofinal infimum of pre-Hamiltonian:

(3.95) h(T ) = 0.

Proof. We recall that hN (τ) was defined in (3.89). Using lemma 3.26, (3.87) and (3.94),we get that:

(3.96)0 =

∂LN

∂T=

∫ 1

0

(H(τ) + TτHt(τ)

)dτ =

∫ 1

0

(hN (τ) + τ

dhN (τ)

dτ

)dτ

=

∫ 1

0

(d(τ hN (τ))

dτ

)dτ = hN (1) = h(T ).

The conclusion follows.

46

For an autonomous problem, we know by lemma 3.23 that the pre-Hamiltonian is constantfor any Pontryagin extremal (on the problem with fixed horizon, and therefore also with avariable one), and therefore:

Corollary 3.29. If in addition the problem is autonomous then the pre-Hamiltonian hasconstant value 0 over [0, T ].

Example 3.30. Let the problem be autonomous, and let the horizon enter in the initial-finalcosts and constraint functions (think of a rendez-vous problem), whose expression are resp.

(3.97) ϕ(y(0),y(T ), T ); Φ(y(0),y(T ), T ).

Then the condition of stationarity of the Lagrangian for the normalized formulation w.r.t. T ,denoting by h the value (constant in time) of H(t), is

(3.98) h+ βDTϕ(y(0), y(T ), T ) + Ψ ·DTΦ(y(0), y(T ), T ) = 0.

Example 3.31. In the case of a simple integrator y = αy + u, with y(t) ∈ Rn, consider theproblem of minimizing the cost function

∫ T0 (1 + 1

2 |u(t)|2)dt, the final time being free. Here weminimize a compromise of the horizon and the quadratic energy, with constraints y(0) = a andy(T ) = b. The pre-Hamiltonian (in qualified form) is

(3.99) H = 1 + 12 |u|

2 + p · (αy + u).

The costate dynamics is −p = αp, and so, p(t) = e−αtp(0) = −u(t).When α = 0, the trajectories are straight lines from a to b with constant speed say u =

(b− a)/T , p(t) = −u, y(t) = a+ t(b− a)/T , and so since the time is free

(3.100) 0 = H = 1− 12 u

2 = 1− 12 |b− a|

2/T 2.

This determines the optimal horizon T = |b − a|/√

2. The optimal speed has always modulus√2.

2.3. Varying initial and final time. Consider the case when both the initial time t0 andthe final time T are free, these times entering in the initial-final state constraints

(3.101) Φ(y(t0),y(T ), t0, T ) ∈ KΦ,

as well as in the cost function, of the form:

(3.102)∫ T

t0

`(t,u(t),y(t))dt+ ϕ(y(t0),y(T ), t0, T ).

The Lagrangian of the time normalized optimal control problem is

(3.103)LN =

∫ 1

0

((T − t0)H(β, t0 + (T − t0)τ,uN (τ),yN (τ))− pN (τ) · yN (τ)

)dτ

+β ϕ(yN (t0),yN (1), t0, T ) + Ψ · Φ(yN (t0),yN (1), t0, T ).

Lemma 3.32. Let (3.57) hold. If (u, y) is an optimal trajectory, there exist a Pontryaginmultiplier (β, Ψ, p) (for the problem with fixed initial and final time) such that

(3.104) −h(t0) + βDt0ϕ(y(t0), y(T ), T ) + Ψ ·Dt0Φ(y(t0), y(T ), t0, T ) = 0,

h(T ) + βDTϕ(y(t0), y(T ), T ) + Ψ ·DTΦ(y(t0), y(T ), t0, T ) = 0.

Proof. We only detail the derivation of the first relation. Redefine

(3.105) hN (τ) := infu∈Uad

H(β, (T − t0)τ, u,yN (τ),pN (τ)).

Again we apply lemma 3.26 to the time normalized optimal control problem where now [t0, T ]is mapped into [0, 1]. So, the partial derivative of the Lagrangian (3.103) w.r.t. t0 is equal to

47

0. Denoting by r the (easily computed) contributions of the initial-final cost and constraints,recalling the definition (3.88) of H(τ), we get that:

(3.106)

0 =∂LN

∂t0=

∫ 1

0

(−H(τ) + (1− τ)(T − t0)Ht(τ)

)dτ + r

=

∫ 1

0

(−hN (τ) + (1− τ)

d

dτhN (τ)

)dτ + r

=

∫ 1

0

d

dτ

((1− τ)hN (τ)

)dτ + r = −hN (0) = −h(t0) + r.

The first relation in (3.104) follows.

2.4. Interior point constraints.2.4.1. A general setting. It may happen that some constraints hold on the state at finitely

many times other that the initial and final ones. Typical examples are (i) interpolation problems,(ii) successive rendez-vous (e.g. of several asteroids with a satellite). We may formalize theseconstraints as follows. Consider a subdivision of [0, T ]:

(3.107) 0 = t0 < t1 < · · · < tN = T,

and the interior point constraints

(3.108) Φi(y(ti)) ∈ Ki, i = 0, . . . , N,

where Ki is a closed convex subset of RnΦ,i and Φi : Rn → RnΦ,i . Let us describe in generalterms how to reduce these constraints to the setting (3.29). As in the case of variable horizon wecan reparametrize each interval ]ti, ti+1[ so that it starts at time 0 and ends at time 1, puttingas decision variable the duration

(3.109) Ti := ti − ti−1; i = 1, . . . , N.

The dynamics (after renormalizing the time) that may vary from one interval to another, overthe ith interval, can be writen, assuming that we have autonomous problems, as

(3.110) yi(τ) = Tifi(ui(τ),yi(τ)) for a.a. τ in (0, 1), i = 1, . . . , N.

The cost function is

(3.111) J(u,y) :=N∑i=1

Ti

∫ 1

0`i(ui(τ),yi(τ))dτ + ϕ(y1(0),y1(1),y2(1), . . . ,yN (1)).

We also have the continuity equations

(3.112) yi(1)− yi+1(0) = 0, i = 1, . . . , N − 1,

whose Lagrange multipliers will be denoted by ηi, and control constraints

(3.113) ui(τ) ∈ U iad, for a.a. τ in (0, 1), i = 1, . . . , N.

as well as possible interior constraints other than (3.112):

(3.114) Φ(y1(0),y1(1),y2(1), . . . ,yN (1)) ∈ K,

with K closed and convex. The Lagrangian of this problem is L := L1 + L2, with

(3.115)

L1 :=N∑i=1

∫ 1

0

(βTi`(u

i(τ),yi(τ)) + pi(τ) · (Tif(ui(τ),yi(τ))− y(τ))

dτ,

L2 := βϕ(y1(0),y1(1), . . . ,yN (1)) + Ψ · Φ(y1(0),y1(1), . . . ,yN (1))

+Ψ0 · Φ0(y1(0)) +N∑i=1

Ψi · Φi(yi(1)) +N−1∑i=1

ηi · (yi(1)− yi+1(0)).

48

Set

(3.116) H i(β, u, y, p) := β`i(u, y) + p · f i(y, u).

Integrating by parts, we have that

(3.117) L1 =N∑i=1

∫ 1

0

(pi(τ) · yi(τ) + TiH

i(β,ui(τ),yi(τ),pi(τ)))

dτ+pi(0)·yi(0)−pi(1)·yi(1).

Therefore we have the costate equation at the reference trajectory (u, y):

(3.118)− ˙pi(τ) = Ti∇yH i(τ) = Tif

iy(u

i(τ), yi(τ))†pi(τ) + βTi∇y`i(ui(τ), yi(τ)),

i = 1, . . . , N,

with transversality conditions, skipping arguments when possible:

(3.119) 0 = ∇y0L = pi(0) + β∇y0ϕ+ (Dy0Φ)†Ψ + (DΦ0)†Ψ0,

(3.120) 0 = ∇yi0L = pi(0)− ηi−1, i = 2, . . . , N,

(3.121) 0 = ∇yi1L = ηi − pi(1) + β∇yiϕ+ (DyiΦ)†Ψ + (DΦi)†Ψi, i = 1, . . . , N − 1,

(3.122) 0 = ∇yN1 L = −pN (1) + β∇yN1 ϕ+ (DyN1Φ)†Ψ + (DΦN )†ΨN .

So, we can eliminate η and write the punctual conditions for the costate in the form

(3.123)

pN (1) = β∇yNϕ+ (DyNΦ)†Ψ + (DΦN )†ΨN ,

pi(1) = pi+1(0) + β∇yiϕ+ (DyiΦ)†Ψ + (DΦi)†Ψi, i = 1, . . . , N − 1,

−pi(0) = β∇y0ϕ+ (Dy0Φ)†Ψ + (DΦ0)†Ψ0.

The multiplier Ψ and the Ψi satisfy

(3.124)

Ψ ∈ NK(Φ(y1(0),y1(1), . . . ,yN (1)));

Ψ0 ∈ NK0(Φ0(y1(0))),

Ψi ∈ NKi(Φi(yi(1))), i = 1, . . . , N.

The nontriviality condition reads

(3.125) β + |Ψ|+N+1∑i=0

|Ψi| > 0.

The Hamiltonian inequality can be decomposed between separated problems for each interval:we get that

(3.126)

H i(β,ui(τ),yi(τ),pi(τ)) = minui∈U iad

H i(β, u,yi(τ),pi(τ)),

for a.a. τ ∈ (0, 1), i = 1, . . . , N.

By (an obvious extension of) lemma 3.23 it follows that

(3.127) t 7→ H i(β,ui(t),yi(t),pi(t)) is constant over (0, 1).

Exercice 3.33. Write the PMP in the original formulation, i.e., with costate function oft ∈ [0, T ].

49

2.4.2. Variable durations. In the study of interior point constraints, we assumed that thedurations Ti were given. But if some of them are free, say for i ∈ I, the others being fixed. Thenfor i ∈ I we have the additional condition

(3.128) 0 =∂L

∂T i=

∫ 1

0H i(τ)dτ.

Equivalently, by (3.127):

(3.129)

If the duration Ti is free, then the Hamiltonianhas value zero over the ith interval, i ≤ N .

Consider now the case when the final time of the ith interval is free. We can then see Ti andTi+1 as optimization variables, subject to the constraint Ti + Ti+1 = b (b given). So we mustadd γ(Ti + Ti+1 − b) to the Lagrangian of the problem and we get that

(3.130) 0 =∂L

∂T j= γ +

∫ 1

0Hj(τ)dτ, j = i, i+ 1.

Thanks again to (3.127) we obtain that

(3.131)

If the final time of the ith interval is free,then the Hamiltonian is continuous at that time.

Exercice 3.34. Extend this result to the case of non autonomous problems.Hint: follow the proof of lemma 3.28.

2.4.3. Variable dynamics structure. Think of the problem of the ascent trajectory of a spaceshuttle, with different stages that are expelled during the ascent. This fits in the present frame-work, but with a state yi whose dimension depends on the interval and, rather than continuityof the state, compatibility conditions on the form

(3.132) Φi(yi(1),yi+1(0)) ∈ Ki, i = 1, . . . , N − 1,

with again Ki closed convex subset of an Euclidean space and Φi mapping between Euclideanspaces of appropriate dimension. So, we consider the same setting as before, with variablestate dimension, and replacing the continuity equations (3.112) by (3.132). The multiplier Ψi

associated with constraint (3.132) must satisfy

(3.133) Ψi ∈ NKi(Φi(yi(1),yi+1(0)), i = 1, . . . , N − 1.

It appears that the costate dynamics has the same expression, the only changes are the end con-ditions for the costate, which instead of (3.119)-(3.122) now read, denoting by L the Lagrangianin the present setting (that we do not need to explicit):

(3.134) 0 = ∇y0L = pi(0) + β∇y0ϕ+ (Dy0Φ)†Ψ + (DΦ0)†Ψ0,

(3.135) 0 = ∇yi0L = pi(0) +Dyi0Φi(yi(1),yi+1(0))†Ψi, i = 2, . . . , N,

(3.136)0 = ∇yi1L = −pi(1) +Dyi1

Φi(yi(1),yi+1(0))†Ψi + β∇yiϕ+(DyiΦ)†Ψ + (DΦi)†Ψi, i = 1, . . . , N − 1,

(3.137) 0 = ∇yN1 L = −pN (1) + β∇yN1 ϕ+ (DyN1Φ)†Ψ + (DΦN )†ΨN .

2.5. Minimal time problems, geodesics.

50

2.5.1. General results. Minimal time problems are the particular case of problems with fixedinital state, free final time, zero final cost, cost integrand `(u, y) = 1, and final constraintΦF (y(T )) ∈ KF . The pre-Hamiltonian is

(3.138) H(β, t, u, y, p) = β + p · f(t, u, y).

So, the Hamiltonian inequality (3.38) reads

(3.139) p(t) · f(t, u(t), y(t)) = infu∈Uad

p(t) · f(t, u, y(t)) for a.a. t ∈ (0, T ).

The costate dynamics and non triviality conditions are (see (3.44)-(3.45)) :

(3.140)− ˙p(t) = fy(t)

†p(t) for a.a. t ∈ [0, T ];p(T ) = DΦF (y(T ))†ΨF ; ΨF ∈ NKF (y(T )).

(3.141) β ∈ 0, 1; β + |ΨF | > 0.

So the PMP here reduces to (3.139)-(3.141), with (since the final time is free), by lemma 3.28,the condition that the final Hamiltonian has value 0.

(3.142) h(T ) = 0; if the problem is autonomous, then h(t) = 0, for all t ∈ [0, T ].

One can prove the following key result:

Lemma 3.35. If (u, y) is solution of the minimal time problem, or more generally a Pon-tryagin extremal of this problem, then ΨF 6= 0, and if Φ′F (y(T )) is surjective, then p(t) 6= 0, forall t ∈ [0, T ].

Proof. If on the contrary ΨF = 0, then by (3.140), p(t) = 0 for all t. So the pre-Hamiltonian reduces to β, but it has zero value at final time, by lemma 3.28, so that β = 0,contradicting (3.141). So ΨF 6= 0. If Φ′F (y(T )) is surjective, then its transpose matrix isinjective, and so p(T ) = Φ′F (y(T ))†ΨF is not zero. Being the solution of an homogeneous linearODE, the costate is nonzero at any time. The result follows.

Remark 3.36. Since the final pre-Hamiltonian has zero value, the l.h.s. of (3.139) is nonpos-itive, with value 0 iff β = 0. So (provided u is continuous at time T ), p(T ) and f(T, u(T ), y(T ))make an obtuse angle if β > 0, and are orthogonal if β = 0. The same relation holds for anyt ∈ [0, T ] if the problem is autonomous (since then the pre-Hamiltonian has a constant value).

Remark 3.37. We could formulate minimal time problems with a zero integral cost and finalcost T . The resulting optimality conditions are equivalent to those above. By pre-Hamiltonian ofa minimal time problem one often understands the one of this second formulation, i.e., p ·f(u, y).

Example 3.38 (A case when β = 0). Let n = 2, ΦF be the identity, KF = x ∈ R2; x2 ≥x2

1, y0 = (−1, 0)†, with dynamics y1(t) = u(t) ∈ [0, 1], y2(t) = 0. Clearly the minimum timeis 1 and the optimal solution is u(t) = 1, y1(t) = t − 1, y2(t) = 0. Since the normal coneto KF at the final state y(1) = 0 is the half line generated by (0,−1)†, we may assume thatΨF = (0,−1)†; that ˙p(t) = 0 implies p(t) = (0,−1)†, over [0, T ]. The Hamiltonian inequality issatisfied, as expected. Since p(t) · f(u(t), y(t)) = 0, we have that β = 0.

We next discuss some important classes of optimal time problems.2.5.2. Linear dynamics. Let f(u, y) = Ay + Bu with A and B matrices of compatible di-

mensions. Then − ˙p(t) = A†p(t), and so, p(t) = e(T−t)A†p(T ) is an analytic function (i.e., has apowerseries expansion). Denote by p(t)(k) the kth derivative of p(t). The control must minimizeover Uad the function p(t)†Bu. If the latter vanishes identically, then for k = 0 to n − 1, wehave that

(3.143) 0 = B†p(t)(k) = (−1)kB†(A†)kp(t),

51

whence p(t) is orthogonal to A(k)B. It is said that the pair (A,B) is controllable, if (in thecontext of linear systems) the family A(k)B, k = 0 to n− 1, has rank n. If this condition holdsthen p vanishes identically, and we know that this is impossible when Φ′F (y(T )) is surjective.

Example 3.39 (Bound constraints). Assume that Uad = [−1, 1]m and denote by Bj the jthcolumn of B. Let j ∈ 1, . . . ,m. Then uj(t) minimizes uj 7→ ujBj · p(t) for a.a. t. If Bj · p(t)does not identicall vanish, since a nonconstant analytic function has finitely many zeros overa bounded interval, we deduce that uj(t) is bang-bang (equal to ±1 for a.a. t ∈ [0, T ]), andmore precisely, equal to −1 when Bj · p(t) > 0, and to 1 when Bj · p(t) < 0, with finitely manycommutation times.

In particular, if m = 1, the pair (A,B) is controllable, and Φ′F (y(T )) is surjective, thecontrol is bang-bang with fnitely many switching times.

Remark 3.40. It can be checked that, if A has only real eigenvalues, and Bj · p(t) does notidenticall vanish, then uj(t) has at most n−1 switches. This holds in particular for the standardnth order integrator (with here m = 1):

(3.144) y1 = y2, . . . , yn−1 = yn, yn = u.

About this and the explicit resolution of two dimensional problems, see e.g. [14, Part III].

2.5.3. Geometric optics and Riemaniann geometry.

Example 3.41 (Geometric optics). The nominal dynamics is ˙y(t) = F (y(t))u(t), where thepositive valued light speed F : Rn → R is of class C∞, and u(t) ∈ B (closed Euclidean unitball) for a.a. t. We assume Φ′F (y(T )) to be surjective, so that the costate does not vanish.The pre-Hamiltonian β + F (y(t))p(t) · u(t) attains its minimum over B at −p(t)/|p(t)|, andhas value 0 at that point, so that β cannot have the value 0, and so, 0 = h(t) = 1 − F (y)|p|.Eliminating u(t) = −p(t)/|p(t)|, we see that the state-costate pair has the following dynamics,skipping the time argument:

(3.145) ˙y = −F (y)p/|p|; ˙p = −∇yH = −(p · u)∇F (y) = |p|∇F (y).

If over some open subset E of Rn the function F is constant, then p and consequently u arealso constant, so that the extremal trajectories over E are lines. Since the final time is free, wehave that

(3.146) 0 = H(t) = 1− F (y(t))|p(t)|, for all t ∈ [0, T ].

Remark 3.42. Given an horizon T , the minimal energy transfer for geometric optics consistsin minimizing 1

2

∫ T0 u(t)2dt with the above dynamics and final constraints. The pre-Hamiltonian

is 12βu

2 + F (y)p · u. If β = 0 then p(t) = 0 for all t, otherwise the infinum of the pre-Hamiltonian has value −∞. Next conside the case when β = 1. The unique control minimizingthe pre-Hamiltonian being u(t) = −F (y(t))p(t), the state-costate pair has dynamics

(3.147) ˙y = −F (y)2p; ˙p = −∇yH = −(p · u)∇F (y) = F (y)|p|2∇F (y).

This dynamics is, at each point, proportional to the one in (3.145). Therefore, the orbits ofthe state-costate pair in geometrical optics coincide with those of the minimal energy transfer(by say the orbit of f(t) for t ∈ [0, T ] we mean f(t), t ∈ [0, T ]). Besides, in the minimalenergy transfert problem, since the pre-Hamiltonian has a constant value equal to −1

2 |u(t)|2,the solution of an extremal of a minimal energy transfert problem has constant control norm.

In the case of geometric optics, the medium is isotropic, that is, the speed at a given pointdoes not depend on the direction of propagation. We next present an extension to the case ofan anisotropic medium.

Example 3.43 (Riemaniann geometry). Consider the minimal time problem with nominaldynamics ˙y = A(y)u, the symmetric matrix A(y), of size n, being invertible for all y, and C∞function of y, with the constraint u(t) ∈ B for a.a. time, which means that ˙y†A(y)−2 ˙y ≤ 1. We

52

still can reduce to the case of a regular multiplier and show that p cannot vanish. The qualifiedpre-Hamiltonian (one easily checks that β > 0) 1+p†A(y)u attains, over a Pontryagin extremal,its minimum at −A(y)p/|A(y)p|, and so,

(3.148) ˙y = −A2(y)p/|A(y)p|.In addition, for k = 1 to n, setting Ak(y) := ∂A(y)/∂yk, we have that

(3.149) ˙pk = −∇ykH = −p†Ak(y)u = p†Ak(y)A(y)p/|A(y)p|.If over some open subset E of Rn the function A(y) is constant, then p and consequently u arealso constant, so that the extremal trajectories over E are lines. The corresponding minimalenergy transfert problem consists in minimizing 1

2

∫ T0 u(t)2dt over the previous dynamics and

constraints. The pre-Hamiltonian 12u

2 + p†A(y)u attains its minimum at u = −A(y)p, and so

(3.150) ˙y = −A2(y)p; ˙pk = −∇ykH = −p†Ak(y)u = p†Ak(y)A(y)p.

Here again, the orbits of the two problems coincide. One calls geodesics the orbits of y whenthe pair (y,p) satisfies the state-costate dynamics (forgetting the conditions at time 0 and T )for now t ∈ R. The pre-Hamiltonian having the constant value −1

2 |u(t)|2, the control norm|u(t)| = |A(y(t))p(t)| is also constant.

Remark 3.44. The usual way to introduce a variational problem related to these geodesicsis to set a smooth mapping G(y), for y ∈ Rn, with value in the set of positive definite symmetricmatrices and to define the lenght between y0 and y1 in Rn as the infimum of

(3.151) I(y) :=

∫ b

a(y(s)†G(y(s))y(s))1/2ds

over those absolutely continuous curves y(t) such that y(a) = y0 and y(b) = y1. It is easilyshown that under a time change of the form s′ = ϕ(s), with ϕ : R→ R of class C1 with positivederivative, the cost is invariant1. So it is possible to reduce the problem to the one of minimizingb − a under the constraint y(s)†G(y(s))y(s))1/2 ≤ 1. Taking A(y) := G(y)−1/2 we recover oursetting.

3. Qualification conditions

We saw in several particular casess how to check that singular multipliers do not exist. Seeexamples 3.16 and 3.18, and the above discussion of geometric optics and Riemaniann geometry.We now give a general approach to this question, for problem (3.29), starting with the case ofLagrange multipliers. Let (u, y) be an admissible trajectory for problem (3.29).

3.1. Lagrange multipliers.3.1.1. General framework. We assume here that Uad is a nonempty, closed convex subset of

Rm. Remember that z[v, z0] denotes the solution of the linearized state equation (2.115), andthat the Lagrange multipliers were introduced in definition 3.19. Consider the convex sets

(3.152)

C := z[v, z0]; z0 ∈ Rn; u(t) + v(t) ∈ Uad a.e.,C0T := (z(0), z(T )); z ∈ C;M := DΦ(y(0), y(T ))C0T ;

MΦ := M + Φ(y(0), y(T ))−KΦ.

The main result concerning Lagrange multiplier is as follows:

Proposition 3.45. The set of singular Lagrange multipliers associated with (u, y) is emptyiff the following qualification condition holds:

(3.153) MΦ is a neighbourhood of 0.

The proof needs some lemmas.

1Indeed, set z(s′) := y(s), and use ds′ = ϕ′(s)ds as well as dz(s′) = ϕ′(s)−1dy(s).

53

Lemma 3.46. The positive polar cone (definition 2.9) of MΦ satisfies

(3.154) (MΦ)+ = M+ ∩NKΦ(Φ(y(0), y(T ))).

Proof. Indeed, Ψ ∈ (MΦ)+ iff

(3.155) 0 ≤ Ψ · (m+ Φ(y(0), y(T ))− k), for all m ∈M and k ∈ K.

Choosing m = 0 (corresponding to v = 0) we deduce that Ψ · (k − Φ(y(0), y(T ))) ≤ 0, so thatΨ ∈ NKΦ

(Φ(y(0), y(T ))). Choosing k = Φ(y(0), y(T )), we get Ψ ·m ≥ 0, so that Ψ ∈M+. Wejust proved that (MΦ)+ is included in the r.h.s. of (3.154). Conversely, if Ψ belongs to the r.h.s.of (3.154), then

(3.156) Ψ ·m ≥ 0 and Ψ · (Φ(y(0), y(T ))− k) ≥ 0, for any m ∈M and k ∈ K,so that (3.155) holds, i.e., Ψ ∈ (MΦ)+. This ends the proof.

Lemma 3.47. Let Ψ ∈ RnΦ. Then there exists a singular Lagrange multiplier associated with(u, y), of the form (β = 0,Ψ, p), iff Ψ 6= 0 and Ψ ∈ (MΦ)+.

Proof. Let (β = 0,Ψ, p) be a singular Lagrange multiplier, so that Ψ 6= 0, and let z =z[v, z0]. By remark 3.11,

(3.157) Ψ†DΦ(y(0), y(T ))(z(0), z(T )) =

∫ T

0Hu(t)v(t)dt.

The last integral is nonnegative in view of (3.46). So, any singular multiplier Ψ belongs to M+

and therefore (since Ψ ∈ NKΦ(Φ(y(0), y(T )))) also to (MΦ)+ by lemma 3.46. Conversely, let

Ψ ∈ (MΦ)+, Ψ 6= 0. Let p satisfy the costate dynamics, with final condition

(3.158) p(T ) = DyTΦ(y(0), y(T )))†Ψ.

Set η := p(0) +∇y0Φ(y(0), y(T ))†Ψ. Then

(3.159)

0 ≤ Ψ†DΦ(y(0), y(T ))(z(0), z(T )) = [p · z]T0 + η · z(0)

=

∫ T

0( ˙p(t) · z(t) + p(t) · z(t))dt+ η · z(0)

=

∫ T

0Hu(t)v(t)dt+ η · z(0).

Since we may take z(0) arbitrarily if follows that η = 0, i.e., p is the costate associated with Ψ.We deduce then easily that Hu(t)v(t) ≥ 0 a.e., showing that (β = 0,Ψ, p) is a singular Lagrangemultiplier.

We can now prove the main result:

Proof of prop. 3.45. IfMΦ is a neighbourhood of 0, then its polar cone reduces to 0. Bythe above lemma, the set of singular multipliers is empty. Conversely, if the convex set M+

KΦis

not a neighbourhood of 0, by remark 2.14, we can separate MΦ from zero by some nonzero Ψ,which means that Ψ ∈ (MΦ)+; we conclude with lemma 3.47.

3.1.2. Specific structures. We end the subsection by considering some particular cases.

Exercice 3.48. (i) If KΦ = 0 (initial-final constraints of equality type), show that theset of singular Lagrange multipliers is empty iff M is a neighbourhood of 0.(ii) In in addition there is no control constraints (i.e., Uad = Rm) show that the set of singularLagrange multipliers is empty iff

(3.160) The mapping (v, z0) 7→ DΦ(y(0), y(T ))(z[v, z0](0), z[v, z0](T )) is surjective.

In the last case above we can be slightly more precise:

Lemma 3.49. If (3.160) holds and Uad = Rm, then there is no singular Lagrange multipliers,and there is a unique normal multiplier.

54

Proof. In view of proposition 3.45, β 6= 0; so, we may take β = 1. By computations similarto those in (3.157), we find that for any (v, z0) ∈ U × Rn, setting z = z[v, z0]:(3.161)

Dϕ(y(0), y(T ))(z(0), z(T )) + Ψ†DΦ(y(0), y(T ))(z(0), z(T )) =

∫ T

0DuH(t)v(t)dt ≥ 0.

Changing (v, z0) into its opposite we obtain the inequality in the reverse direction and so theintegral is equal to zero. Had we two possible multipliers say Ψ′ and Ψ′′, their difference Ψ :=Ψ′′ −Ψ′ would therefore satisfy

(3.162) Ψ†DΦ(y(0), y(T ))(z(0), z(T )) = 0, for all v ∈ U ,

which in view of (3.160) implies Ψ = 0. The conclusion follows.

We can also detail the case of inequality constraints, starting with the more general casewhen KΦ has a nonempty interior.

Lemma 3.50. Let KΦ have a nonempty interior. Then the set of singular Lagrange multi-pliers is empty iff there exists (v, z0) ∈ U ×Rn such that u + v satisfies the control constraints,and z := z[v, z0] satisfies

(3.163) Φ(y(0), y(T )) +DΦ(y(0), y(T ))(z(0), z(T )) ∈ int(KΦ).

Proof. By a translation argument we may assume that Φ(y(0), y(T )) = 0. We can thenwrite the above condition as int(KΦ) ∩M 6= ∅. So, in view of proposition 3.45, we must provethat MΦ is a neighbourhood of 0 iff int(KΦ) ∩M 6= ∅.

If the intersection is non empty it is easily seen that MΦ is a neighbourhood of 0. If theintersection is empty, we can separate these convex sets by some nonzero Ψ and (since, int(KΦ)being nonempty, its closure is equal to KΦ) we have that Ψ · x ≥ 0 for any x ∈ MΦ. But thenMΦ cannot be a neighbourhood of 0.

Exercice 3.51. When KΦ = RnΦ− (initial-final constraints of inequality type), show that

the set of singular Lagrange multipliers is empty iff there exists (v, z0) ∈ U ×Rn such that u+vsatisfies the control constraints, and z := z[v, z0] satisfies

(3.164) DΦk(y(0), y(T ))(z(0), z(T )) < 0 if Φk(y(0), y(T )) = 0, k = 1 to nΦ.

3.2. Pontryagin multipliers. We recall that (u, y) is an admissible trajectory for problem(3.29). We assume here that Uad is a nonempty, closed subset of Rm. Instead of the linearizedstate equation (2.115), we consider again the dynamics (3.19) for ξ, with u ∈ U , but in the caseof a varying initial condition ξ0. Its solution in Y may viewed as a function of an initial valueξ0 and u, denoted by ξ[u, ξ0]. Consider the sets

(3.165)

C ′ := ξ[u, ξ0]; ξ0 ∈ Rn; u(t) ∈ Uad a.e.,C ′0T := (ξ(0), ξ(T )); ξ ∈ C ′;M ′ := DΦ(y(0), y(T ))C ′0T ;

M ′′ := conv(M ′);

M ′Φ := M ′′ + Φ(y(0), y(T ))−KΦ.

By conv(M ′) we mean the convex hull of M ′, i.e., the smallest convex set containing M ′, whichis nothing but the set of convex combinations (linear combinations with nonnegative weightssumming to 1) of elements of M ′. Similarly to lemma 3.155, observing that M ′ and M ′′ havethe same polar sets, we can easily show that

(3.166) (M ′Φ)+ = (M ′)+ ∩NKΦ(Φ(y(0), y(T ))).

Lemma 3.52. Let (u, y) be an admissible trajectory for problem (3.29), and Ψ ∈ RnΦ. Thenthere exists a singular Pontryagin multiplier of the form (β = 0,Ψ, p) iff Ψ 6= 0 and Ψ ∈ (M ′Φ)+.

55

Proof. Let (β = 0,Ψ, p) be a singular Pontryagin multiplier, so that Ψ 6= 0, and let ξ ∈ C,ξ = ξ[u, ξ0] with u(t) ∈ Uad a.e. Then

(3.167)∆ := Ψ†DΦ(y(0), y(T ))(ξ(0), ξ(T )) = [p · ξ]T0

=

∫ T

0( ˙p(t) · ξ(t) + p(t) · ξ(t))dt =

∫ T

0

(H(0, t,u(t), y(t), p(t))− H(t)

)dt.

By the Hamiltonian inequality, the last integral is nonnegative, so that Ψ ∈ (M ′)+. SinceΨ ∈ NKΦ

(Φ(y(0), y(T )))), it follows that Ψ ∈ (MΦ)+. Conversely, let Ψ ∈ (M ′Φ)+, Ψ 6= 0. Letp satisfy the costate dynamics, with final condition

(3.168) p(T ) = DyTΦ(y(0), y(T )))†Ψ.

Set η := p(0) +∇y0Φ(y(0), y(T ))†Ψ. Then

(3.169)

0 ≤ Ψ†DΦ(y(0), y(T ))(ξ(0), ξ(T )) = [p · ξ]T0 + η · ξ(0)

=

∫ T

0( ˙p(t) · ξ(t) + p(t) · ξ(t))dt+ η · ξ(0)

=

∫ T

0

(H(0, t,u(t), y(t), p(t))− H(t)

)dt+ η · ξ(0).

Since we may take ξ(0) arbitrarily if follows that η = 0, i.e., p is the costate associated with Ψ.We deduce then easily that the Hamiltonian inequality holds, showing that (β = 0,Ψ, p) is asingular Pontryagin multiplier.

We obtain then the main result for the characterization of the absence of Pontryagin multi-pliers:

Proposition 3.53. The set of singular Pontryagin multipliers is empty iff the followingqualification condition holds:

(3.170) M ′Φ is a neighbourhood of 0.

Proof. Similar to the one of proposition 3.45.

Lemma 3.54. If KΦ = 0 (case of equality constraints), the set of singular Pontryaginmultipliers is empty iff M ′′ is a neighbourhood of 0.

Proof. Obvious consequence of proposition 3.53.

Lemma 3.55. Let KΦ have a nonempty interior. Then the set of singular Pontryagin mul-tipliers is empty if there exists u satisfying the control constraints and ξ0 ∈ Rn such thatξ := ξ[u, ξ0] satisfies

(3.171) Φ(y(0), y(T )) +DΦ(y(0), y(T ))(ξ(0), ξ(T )) ∈ int(KΦ).

Proof. Similar to the proof of lemma 3.50.

Example 3.56. Consider the problem of minimizing 12

∫ T0 u(t)2dt, with state equation y(t) =

u(t)3, and with fixed initial and final time y(0) = y(T ) = 0 and no control constraints. Theunique solution (u, y) is the null trajectory, and any costate has a constant value (as a functionof time). There exists a singular Lagrange multiplier with p(t) = 1 for all t, ΨF = 1. But thereexists no singular Pontryagin multiplier, due to the fact that a singular pre-Hamiltonian of theform p(t)u3 cannot have a minimum at zero (since p(t) = ΨF 6= 0).

4. Proof of Pontryagin’s principle

4.1. Ekeland’s principle. If f : X → R has a finite infimum, we say that x ∈ X is anε-optimal for the problem of minimizing f if f(x) ≤ inf f +ε. The following result demonstratesthat, under weak assumptions, it is possible then to construct another ε-optimal solution, closeto x, that becomes the minimizer of a slightly perturbed objective function.

56

Theorem 3.57. (Ekeland’s variational principle) Let (E, ρ) be a complete metric space andf : E → R ∪ +∞ a lower semicontinuous function. Suppose that infe∈E f(e) is finite and let,for a given ε > 0, e ∈ E be an ε-minimizer of f , i.e., f(e) ≤ infe∈E f(e) + ε. Then for anyk > 0, there exists a point e ∈ E such that ρ(e, e) ≤ k−1 and

f(e) ≤ f(e)− εkρ(e, e),(3.172)f(e)− εk ρ(e, e) < f(e), ∀ e ∈ E, e 6= e.(3.173)

Proof. Consider the multifunction M : E → 2E defined by

M(e) := e′ : f(e′) + kερ(e, e′) ≤ f(e).It is not difficult to see that M(·) is reflexive, i.e., e ∈ M(e), and transitive, i.e., e′ ∈ M(e)implies M(e′) ⊂M(e). Consider the function v : dom f → R defined by

v(e) := inff(e′) : e′ ∈M(e).We have that infE f ≤ v(e) ≤ f(e), and

(3.174) εkρ(e, e′) ≤ f(e)− v(e), ∀e′ ∈M(e).

Since f(e)− v(e) ≤ f(e)− infE f ≤ ε, it follows that(3.175) kρ(e, e) ≤ 1, ∀ e ∈M(e).

Consequently, the diameter (i.e., the supremum of distances between two points) of M(e) is lessthan or equal to 2k−1.

Consider a sequence en satisfying

e1 = e, en+1 ∈M(en) , and f(en+1) ≤ v(en) + ε2−n.

(By definition of v(·), such a sequence exists.) SinceM(en+1) ⊂M(en) (by transitivity ofM(·)),we have v(en) ≤ v(en+1), and since v(e) ≤ f(e), it follows that

v(en+1) ≤ f(en+1) ≤ v(en) + ε2−n ≤ v(en+1) + ε2−n,

implying 0 ≤ f(en+1)− v(en+1) ≤ ε2−n. Combining this and (3.174), we get that kρ(en+1, e) ≤2−n for all e ∈ M(en+1), and hence the diameter of M(en+1) tends to zero as n → ∞. Sincein addition en+1 ∈ M(en) and M(en+1) ⊂ M(en), it follows that en is a Cauchy sequence.By completeness of the space E it follows that en converges to some e ∈ dom(f), and sincethe diameters of M(en) tend to zero,

⋂∞n=1M(en) = e. Since en is included in the set M(e)

and M(e) is closed, we have by (3.175) that kρ(e, e) ≤ 1, and by the definition of M(e) thatf(e) + εkρ(e, e) ≤ f(e), so that (3.172) holds. In addition, since M(e) ⊂ M(en) for all n, anddiam(M(en))→ 0, we have M(e) = e, which implies (3.173).

Note that condition (3.172) of the above theorem implies that e is an ε-minimizer of f overE, and that condition (3.173) means that e is the unique minimizer of the “perturbed" functionf(·) + εk ρ(·, e) over E. In particular, by taking k = ε−1/2 we obtain that for any ε-minimizere of f there exists another ε-minimizer e such that ρ(e, e) ≤ ε1/2 and e is the minimizer of thefunction f(·) + ε1/2ρ(·, e).

4.2. Ekeland’s metric and the penalized problem. Consider the set

(3.176) Uad := L∞(0, T ;Uad)

endowed with Ekeland’s metric

(3.177) ρE(u,v) = meast; u(t) 6= v(t).It is not difficult to check that (Uad, ρE) is a complete metric space. Since the initial state is notfixed, we need to consider the product space Uad × Rn, endowed with the augmented Ekelandmetric

(3.178) ρA[(u, y0), (v, z0)] := |z0 − y0|+ ρE(u,v).

57

Again, it is easily checked that (Uad × Rn, ρA) is a complete metric space.We next introduce the composite cost function, defined for ε > 0:

(3.179) JεR(u, y0) :=([JR(u, y0)− J + ε2]2+ + dKΦ

[Φ(y0,y[u, y0](T ))]2)1/2

.

Remark 3.58. We will need the following results:(i) The Euclidean distance to KΦ, denoted by dKΦ

(·), has a continuously differentiable square,and the expression of its derivative is, denoting by PKΦ

the orthogonal projection over KΦ is,see e.g. [9, Thm 4.77]:

(3.180) ∇(

12d

2KΦ

(w))

= w − PKΦ(w), w ∈ RnΦ .

(ii) The function R→ R, x 7→ (x+)2 is of class C1 with derivative 2x+.

4.3. Proof of theorem 3.13. a) We first deal with the case when Uad is bounded. SettingJ := JIF (u, y), apply Ekeland’s principle to the perturbed problem

Minu,y0

JεR(u, y0); u ∈ Uad; y0 ∈ Rn. (Pε)

Since JεR is a nonnegative function, and JεR(u, y0) = ε2, we have that (u, y0) is a ε2-minimumof (Pε). Since Uad is bounded, it is easily checked that the function (u, y0) 7→ JεR(u, y0) iscontinuous for the augmented metric. By theorem 3.57, there exists (uε,yε) ∈ U × Y such that

(3.181) |yε(0)− y0|+ ρE(uε,u) ≤ ε,

(3.182)JεR(uε,yε(0)) ≤ JεR(u, y0) + ε(|y0 − yε(0)|+ ρE(u, uε)),

for all (u, y0) ∈ Uad × Rn.Necessarily JεR(uε,yε(0)) > 0, as was previously discussed. Define

βε :=(JR(uε,yε(0))− J + ε2)+

JεR(uε,yε(0)); Ψε :=

Φ(yε(0),yε(T ))− PKΦ(Φ(yε(0),yε(T )))

JεR(uε,yε(0));

Then βε ∈ [0, 1], and |Ψε| = JεR(uε,yε(0))−1dKΦ(Φ(yε(0),yε(T ))). Using the definition of

JεR(uε,yε(0)),

(3.183) β2ε + |Ψε|2 = 1.

Using for x > 0 and x′ > 0 the first-order expansion

(3.184)√x′ =

√x+ 1

2(x′ − x)/√x′ + o(|x′ − x|),

we get with remark 3.58, denoting by y the state associated with (u, y0):

(3.185)JεR(u, y0) = JεR(uε,yε(0)) + βε(JR(u, y0)− JR(uε,yε(0)))

+〈Ψε, DΦ(yε(0),yε(T ))(y0 − yε(0),y(T )− yε(T ))〉+o(‖u− uε‖1 + |y0 − yε(0)|).

Let pε ∈W 1,∞(0, T,Rn∗) be the unique solution of

(3.186)−pε(t) = ∇yH(βε, t,u

ε(t),yε(t),pε(t)), a.e. t ∈ (0, T );pε(T ) = ∇yTLIF (βε,y

ε(0),yε(T ),Ψε).

Lemma 3.59. The following expansion holds:(3.187)

JεR(u, y0) = JεR(uε,yε(0)) +

∫ T

0(H(βε, t,u(t),yε(t),pε(t))−H(βε, t,u

ε(t),yε(t),pε(t))) dt

+(p0 +∇y0LIF (βε,yε(0),yε(T ),Ψε))(y0 − yε(0)) + o(‖u− uε‖1 + |y0 − yε(0)|).

Proof. Combine (3.185) with arguments essentially similar to those in the proof of lemma3.4, taking into account the fact that the initial state may vary and that it enters into theinitial-final cost.

58

b) Setting u = uε in (3.182), obtain

(3.188) JεR(uε,yε(0)) ≤ JεR(uε, y0) + ε|y0 − yε(0)|, for all y0 ∈ Rn.Substituting in (3.188) the expression of JεR(uε, y0) given in (3.187) (note that, since u = uε,the integral term vanishes) we get

(3.189) 0 ≤ (pε0 +∇y0LIF (βε,yε(0),yε(T ),Ψε))(y0 − yε(0)) + ε|y0 − yε(0)|+ o(|y0 − yε(0)|).

Taking y0 = yε(0)− ρ(pε0 +∇y0LIF (βε,yε(0),yε(T ),Ψε)), with ρ ↓ 0, deduce that

(3.190)∣∣pε0 +∇y0LIF (βε,y

ε(0),yε(T ),Ψε)∣∣ ≤ ε.

c) Fixing now y0 = yε(0) in (3.182), obtain

(3.191) JεR(uε,yε(0)) ≤ JεR(u,yε(0)) + ερE(u,uε), for all u ∈ Uad(u).

Substituting in (3.191) the expression of JεR(u,yε(0)) provided by (3.187), get

(3.192) 0 ≤∫ T

0(H(βε, t,u(t),yε(t),pε(t))−H(βε, t,u

ε(t),yε(t),pε(t))) dt

+ερE(u,uε) + o(‖u− uε‖1).

Equivalently, denoting Iε := t ∈ (0, T ); u(t) 6= uε(t), we have that(3.193)

0 ≤∫Iε

(H(βε, t,u(t),yε(t),pε(t))−H(βε, t,uε(t),yε(t),pε(t)) + ε) dt+ o(meas(Iε)).

By techniques similar to those presented after (3.15), we obtain that

(3.194) H(βε, t,uε(t),yε(t),pε(t)) ≤ H(βε, t, v,y

ε(t),pε(t)) + ε,for all v ∈ Uad, for a.a. t ∈ (0, T ).

d) We now pass to the limit in (3.183)-(3.194). Since ρE(u,uε) → 0 and Uad is bounded, wehave that yε → y uniformly. By (3.183), (βε,Ψ

ε) has a unit norm limit point (β, Ψ), so thatthe Hamiltonian inequality holds. Also, passing to the limit in the relation (consequence of thedefinition of projection)

(3.195) Ψε · (k − PKΦ(Φ(yε(0),yε(T )))) ≤ 0, for all k ∈ KΦ,

and since Φ(y0, y(T )) ∈ KΦ, we get that Ψ ∈ NKΦ(Φ(y0, y(T ))). There is no difficulty in

passing to the limit in (3.190), and in (3.186) (the costate pε remains bounded since βε andΨε are bounded). Therefore the costate equation is satisfied. Finally, let εk be a sequenceassociated with the limit-point (β, Ψ). For each k, (3.194) is satisfied on a set Tk, of full measurein [0, T ]. We may then pass to the limit in (3.194) on the set ∩kTk, that is also of full measure.The conclusion follows.e) We end the proof by dealing with the case when Uad is not bounded. Set

(3.196) UR := Uad ∩BR,where BR is the ball of radius R and center 0 in Rm, and UR := L∞(0, T, UR). If R ≥ ‖u‖∞,then u is solution of the problem

Min JR(u); u ∈ UR. (PRε )

By point d), there exists (βR,ΨR) ∈ R+ × NKΦ

(Φ(y0, y(T ))), with (β,Ψ) 6= 0, and pR ∈ Y,such that

−pR(t) = ∇yH(βR, t, u(t), y(t),pR(t)), for a.a. t,(3.197)

u(t) ∈ argminu∈UR

H(βR, t, u, y(t),pR(t)) for a.a. t,(3.198)

−pR0 = ∇y0LIF (βR, y0, y(T ),ΨR);(3.199)

pR(T ) = ∇yTLIF (βR, y0, y(T ),ΨR);(3.200)

ΨR ∈ NKΦ(Φ(y0, y(T ))).(3.201)

59

Multiplying (βR,ΨR, pR) by |(βR,ΨR)|−1, we may assume that |(βR,ΨR)| = 1. It easily follows

that pR is bounded in Y. We conclude by passing to the limit in an extracted sequence forwhich (βR,Ψ

R) converges, in (3.197)-(3.201), using again the fact that a countable intersectionof sets of full measure is of full measure.

60

4Shooting algorithm, second order conditions

Contents

1. Shooting algorithm 611.1. Unconstrained problems 611.2. Problems with initial-final state constraints 631.3. Time discretization 642. Second-order optimality conditions 662.1. Expansion of the reduced cost 662.2. More on the Hessian of the reduced cost 672.3. Case of initial-final equality constraints 682.4. Links with the shooting algorithm 683. Control constraints 694. Notes 70

1. Shooting algorithm

1.1. Unconstrained problems. We consider for the sake of simplicity an autonomousunconstrained optimal control problem: the state equation and cost function are resp.

(4.1) y(t) = f(u(t),y(t)), for a.a. t ∈ (0, T ); y(0) = y0,

(4.2) J(u,y) :=

∫ T

0`(u(t),y(t))dt+ ϕ(y(T )).

Here y0 is given and the functions f , `, ϕ are of class C2 with locally Lipschitz second derivatives,and f is Lipschitz. As before the control and state spaces are

(4.3) U := L∞(0, T,Rm); Y := W 1,∞(0, T,Rn).

So, the control to state mapping y[u] is well-defined U → Y and we denote the reduced cost by

(4.4) JR(u) := J(u,y[u]).

The optimal control problem is

(4.5) Minu∈U

JR(u).

We assume that

(4.6) The control variable u ∈ U is a local solution, and a continuous function of time.

61

We denote by y := y[u] the associated state. Then, by theorem 2.48:

(4.7) Hu(u(t), y(t), p(t)) = 0, for all t ∈ [0, T ].

(We write this relation for all t and not a.e. since u is continuous.) We assume that the followingstrong Legendre-Clebsch condition holds, where Id is the identity matrix: there exists α > 0 suchthat

(4.8) Huu(t) αId, a.e. over [0, T ].

Remember that the corresponding condition with α = 0 is a necessary condition for Pontryaginminima, see remark 3.5.

Applying the implicit function theorem 2.19 to (4.7), we deduce thanks to (4.8) that, forτ ∈ [0, T ] and (u, y, p) close enough of (u(τ), y(τ), p(τ)), there exists Υ : Rn×Rn → Rm of classC1 such that the relation Hu(u, y, p) = 0 is equivalent to u = Υ(y, p). Putting if necessary thetime as an additional state, we can “stick” together the functions Υ for different times and weobtain in this way that for a certain ε > 0, and all t ∈ [0, T ]:

(4.9) If |y − y(t)|+ |p− p(t)| < ε, then Hu(u, y, p) = 0 iff u = Υ(y, p).

Set

(4.10) Υ(t) := Υ(y(t), p(t)), t ∈ [0, T ].

Remark 4.1. When in the sequel we make reference to (4.8), one should understand Huu(t)as the second derivative w.r.t. the control variable of the pre-Hamiltonian specific to eachproblem.

Remark 4.2. Derivating in time the relation ∇uH(u(t), y(t), p(t)) = 0, we get then

Huu(t)Υ(t) + Huy(t)f(t)− fu(t)†∇yH(t) = 0,

which allows to express ˙u(t) = Υ(t) as a function of (u(t), y(t), p(t)).

Eliminating the control in the state and costate equations we see that the pair (y, p) issolution of the autonomous differential equation

(4.11)

˙y(t) = f(Υ(t),y(t)),−p(t) = ∇yH(Υ(t),y(t),p(t)),

for all t ∈ [0, T ],

with initial-final conditions

(4.12) y(0) = y0; p(T ) = ∇ϕ(y(T )).

Remark 4.3. In the particular case of a linear quadratic problem, i.e. when the stateequation is linear and the cost function is quadratic, we recover relations (2.140) which arethemself linear.

Let us introduce the shooting parameter p0 ∈ Rn, and denote (if it exists) by (y[p0],p[p0])the solution of (4.11) with the initial condition (y0, p0). Consider the shooting function Rn → Rndefined by

(4.13) T (p0) := ∇ϕ(y[p0](T ))− p[p0](T ).

Then the following obviously holds:

Lemma 4.4. The two point boundary value problem (TPBVP) (4.11)-(4.12) has a solutionsuch that p(0) = p0 iff p0 is a zero of the shooting function.

So, we reduce the search for a solution of the optimality conditions of the optimal controlproblem, to the resolution of the finite dimensional shooting equation

(4.14) T (p0) = 0.

By our hypothesis of C2 data with Lipschitz second derivatives, we see that the shooting functionis C1 with Lipschitz derivatives. The shooting equation (4.14) is usually solved by a variant of

62

Newton’s method, see section 2.2 of chapter 2. The latter needs the evaluation of the Jacobianof T . Let us give its expression when p0 = p(0) and so (y[p0],p[p0]) = (y, p). By the chainrule, we have that

(4.15) DT (p0)q0 = D2ϕ(y(T ))z(T )− q(T ),

where (z(T ),q(T )) is solution of the linearization of (4.11) with initial condition

(4.16) (z(0),q(0)) = (0, q0).

We can express this linearization as follows. Relation (4.11) is locally equivalent to

(4.17)

y(t) = f(u(t),y(t)),−p(t) = ∇yH(u(t),y(t),p(t)),

0 = ∇uH(u(t),y(t),p(t)).for all t ∈ [0, T ],

Indeed, (4.11) was obtained by eliminating the control variable in the above system, and thelocal equivalence of (4.11) and (4.17) follow from the IFT. It can be easily deduced from the IFTthat both systems evaluated at a point where u(t) = Υ(y(t),p(t)), have equivalent linearizations.In particular, (z,q) satisfies the linearization of (4.11) at (y, p) iff there exists v ∈ U such that(v, z,q) satisfies the linearization of (4.17) at at (u, y, p) and the latter is

(4.18)

z(t) = f ′(t)(v(t), z(t)),−q(t) = fy(t)

†q(t) + Hyu(t)v(t) + Hyy(t)z(t)0 = Huu(t)v(t) + Huy(t)z(t) + fu(t)†q(t).

for all t ∈ [0, T ].

We have proved the following:

Lemma 4.5. The expression of the Jacobian of the shooting function is given by (4.15), where(v, z,q) satisfy (4.16) and (4.18).

1.2. Problems with initial-final state constraints. We restrict the analysis to the caseof the minimization of the function (already defined in (3.26), but in the autonomous case)

(4.19) JIF (u,y) :=

∫ T

0`(u(t),y(t))dt+ ϕ(y(0),y(T )),

with initial-final equality state constraints

(4.20) Φ(y(0), y(T )) = 0.

Now the initial state y0 is free. As before, we assume that the data are C2 with Lipschitz secondderivatives, and denote by y[u, y0] the state associated with (u, y0) ∈ U × Rn, and by JR thereduced cost defined by

(4.21) JR(u, y0) = JIF (u,y[u, y0]).

So, the optimal control problem is

(4.22) Minu∈U ,y0∈Rn

JR(u, y0) s.t. (4.20).

Let (u, y) ∈ U × Y be a locally optimal trajectory whose initial state is denoted by y0. Weassume that it satisfies the following qualification condition, taken from exercice 3.48:

(4.23) The mapping U × Rn → Y, (v, z0) 7→ DΦ(y(0), y(T ))(z[v, z0](0), z[v, z0](T )) is onto.

By lemma 3.49, the regular Lagrange multiplier (with β = 1) is unique. In this setting, theshooting function has for parameters (y0, p0,Ψ), the initial values of the state and costate, aswell as the multiplier associated with the constraints. This function expresses the initial-finalconditions for the costate as well as the constraints :

(4.24) T IF (y0, p0,Ψ) :=

p0 +∇y0LIF (1, y0,y(T ),Ψ)−p(T ) +∇yTLIF (1, y0,y(T ),Ψ)

Φ(y0,y(T ))

,

63

where (y,p) is the result of the integration of (4.11) with the initial condition (y0, p0), and LIFis defined in (3.31). Note that we have the same number of equations and unknown. here again,the shooting function is C1 with Lipschitz derivatives.

We recall that (y0, p(0), Ψ) is a regular zero of T IF if the Jacobian of T IF at this point isinvertible. This is known to ensure the local quadratic convergence of Newton’s method (and, bythe IFT, the stability w.r.t. to perturbations of the solution of the shooting function). We willsee in section 2.4 that this property is closely related to the second-order optimality conditions,and allows to ensure the convergence of the discretized problem.

1.3. Time discretization. We discretize the state equation (2.111) by the Euler methodwith constant time step, assuming the control to be constant over each time step:

(4.25) yk+1 = yk + hf(uk,yk), k = 0, . . . , N − 1; y0 = y0,

with h = T/N , N positive integer, uk ∈ Rm and yk ∈ Rn. The space for the discrete controlsvariables is UN := RNm, and the one for discrete states is YN := R(N+1)n. With each u ∈ UNis associated a unique state y[u, N ] ∈ YN solution of (4.25). The corresponding discretizationof the cost function (2.120) is

(4.26) JN (u,y) := h

N−1∑k=0

`(uk,yk) + ϕ(yN ).

So, the unconstrained discrete optimal control problem (with y0 fixed) is

(4.27) Min JN (u,y); subject to (4.25).

Writing the Lagrangian in the form

(4.28) ϕ(yN ) +

N−1∑k=0

(h `(uk,yk) + pk+1 · (yk + hf(uk,yk)− yk+1)) + q · (y0 − y0),

we obtain the discrete costate equation by setting to zero the partial derivative of the Lagrangianfunction w.r.t. the state, i.e.

(4.29)pk = pk+1 + h

(∇y `(uk,yk) + fy(u

k,yk)†pk+1), k = 1, . . . , N − 1;

pN = ∇ϕ(yN ),

−q = p1 + h(∇y `(u0,y0) + fy(u

0,y0)†p1).

The reduced cost is JNR (u) := JN (u,y[u, N ]). By the same kind of arguments as for the case ofcontinuous time, and denoting y = y[u, N ], we can check that

(4.30) DJNR (u)v =N−1∑k=0

Hu(uk,yk,pk+1)vk.

A necessary condition of local optimality is therefore (compare to (4.7)):

(4.31) Hu(uk,yk,pk+1) = 0, k = 0, . . . , N − 1.

Set tk = kh, k = 0 to N , and let the strong Legendre-Clebsch condition (4.8) be satisfied. If

|u(tk)− uk|+ |y(tk)− yk|is for all k small enough, we can (adding if necessary the time as a state variable) eliminate thecontrol as function of the state and costate :

(4.32) uk = Υ(yk,pk+1),

which allows, abbreviating Υ(yk,pk+1) into Υk, to reduce the discrete optimality system to thediscrete implicit dynamical system in (y,p), for k = 0, . . . , N − 1 :

(4.33)

yk+1 = yk + hf(Υk,yk),pk = pk+1 + h

(∇y `(Υk,yk) + fy(Υ

k,yk)†pk+1),

64

with the end conditions

(4.34) y0 = y0; pN = ∇ϕ(yN ).

Taking p0 = p0 as parameter and denoting by y[p0], p[p0] the solution of (4.33) with initialcondition (y0, p0), we obtain the discrete shooting function

(4.35) T N (p0) := ∇ϕ(yN [p0])− pN [p0].

When h is small enough, under the hypothesis (4.8), T N is defined and uniformly converges (aswell as its derivative) over a neighbourhood of p(0).

In the sequel we assume that

(4.36) p(0) is a regular zero of the shooting function T .We recall that this hypothesis means that T has an invertible Jacobian at the point p(0). Wecan then show the following:

Lemma 4.6. The shooting function T N has in the neighbourhood of p(0) a unique solution,with which is associated a solution (uh,yh,ph) of the discrete optimality conditions, and theorder of convergence is 1 in the sense that

(4.37) max0≤k≤N−1

(|u(tk)− ukh|+ |y(tk)− ykh|+ |p(tk)− pk+1

h |)

+ |y(T )− yNh | = O(h).

Proof. Let V be a closed ball of center p(0) over which the shooting function T N is well-defined and converges uniformly, as well as its Jacobian T N . Given ε > 0 we have the existenceof hε > 0 such that, for h < hε, we have

(4.38) |T N (p)− T (p)|+ |DT N (p)−DT (p)| ≤ ε, for all p ∈ V .

In fact, reducing V and hε if necessary and setting A := DT (p(0)) we may assume that

(4.39) |T N (p)|+ |DT N (p)−A| ≤ ε, for all p ∈ V .

Since we discretized by Euler’s method we have that for some c0 > 0

(4.40) |T N (p(0))| ≤ c0h.

Let pj , j ∈ N the sequence such that

(4.41) p0 := p(0); T N (pj) +A(pj+1 − pj) = 0, j ∈ N,

so that

(4.42) |pj+1 − pj | = |A−1T N (pj)| ≤ ‖A−1‖|T N (pi)|.On the other hand, by (4.41):

(4.43)T N (pj+1) = T N (pj) +

∫ 1

0DT N (pj + θ(pj+1 − pj))(pj+1 − pj)dθ

=

∫ 1

0D(T N (pj + θ(pj+1 − pj))−A

)(pj+1 − pj)dθ.

Let j0 be such that pj+1 ∈ V for j ≤ j0. Then for j ≤ j0, we have that pj + θ(pj+1 − pj) ∈ V .With (4.39) and (4.43) we get that

(4.44) |T N (pj+1)| ≤ maxθ∈[0,1]

‖T N (pj + θ(pj+1 − pj))−A‖|pj+1 − pj | ≤ ε|pj+1 − pj |.

Choosing ε such that ε‖A−1‖ ≤ 1/2, we deduce with (4.42) and the previous display that

(4.45) |pj+2 − pj+1| ≤ ε‖A−1‖|pj+1 − pj | ≤ 12 |p

j+1 − pj | ≤ 1

2j+1|p1 − p0|, for all j ≤ j0.

Since

(4.46) |p1 − p0| ≤ ‖A−1‖|T N (p0)| = O(h)

65

we may take j0 ≥ 1 for h small enough. By (4.45),

(4.47) |pj+2 − p0| ≤ |pj+2 − p1|+ |p1 − p0| ≤ 2|p1 − p0| = O(h),

so that all the sequence belongs to V and pj linearly converges to ph such that

(4.48) |ph − p0| ≤ 2|p1 − p0| ≤ 2‖A−1‖|T N (p0)|.

By(1.3), T N (ph) = limj T N (pj) = 0. Relation (4.37) follows. Finally if the shooting functionhas two zeros p and p′ then setting q = p′ − p:

(4.49) 0 = T N (p′)− T N (p) = A′q, where A′ :=∫ 1

0DT N (p+ θq)dθ.

Taking ε small enough we obtain that A′ is so close to A that it must be invertible and thereforeq = 0.

Remark 4.7. With higher order discretization scheme, the above proof also holds. It fol-lows from (4.48) that the error order for the state, costate and control is of the error order ofthe numerical integration scheme. In the case of Runge-Kutta schemes, it can be shown thatthe discretization of the optimal control problems provides a discretization of the state-costatedynamics that may have a smaller error order. See Hager [26], Bonnans and Laurent-Varin [12].

2. Second-order optimality conditions

In this section we connect the well-posedness of the shooting algorithm to the second-orderoptimality conditions for the optimal control problem.

2.1. Expansion of the reduced cost. We have seen in chapter 2 that the reduced costJR(·) : U → R defined in (4.4) is of class C1, but since the data are C2, we deduce by similararguments that it is of class C2. Denote by Q0(v) := J ′′R(u)(v,v) the quadratic form associatedwith its Hessian (second derivative). The second order Taylor expansion of JR can be writtenas:

(4.50) JR(u + v) = JR(u) + J ′R(u)v + 12Q0(v) +R2(v),

where the remainder of the Taylor expansion satisfies, by (2.34)-(2.35):

(4.51) R2(v) =

∫ 1

0(1− σ)(J ′′R(u + σv)− J ′′R(u))(v,v)dσ = o(‖v‖3∞).

A necessary condition for local optimality in U is therefore

(4.52) J ′R(u) = 0 and Q0(v) ≥ 0, for all v ∈ U .

Consider next the quadratic growth condition:

(4.53) For some α > 0: if ‖v‖∞ is small enough, then JR(u + v) ≥ JR(u) + 12α‖v‖

22,

and the second order sufficient condition:

(4.54) There exists α′ > 0 such that Q0(v) ≥ 2α′‖v‖22 for all v ∈ U .

Theorem 4.8. Let u ∈ U be a critical point of JR, i.e., J ′R(u) = 0. Then conditions (4.53)and (4.54) are equivalent.

Proof. If the quadratic growth condition (4.53) holds, then u is a local solution of theproblem of minimizing JR,α(u) := JR(u)− 1

2α‖u− u‖22. This is the reduced cost for the optimalcontrol problem with the same dynamics and final cost, and perturbed integrand of the integralcost: `α(t, u, y) := `(u, y) − 1

2α|u − u(t)|2, and J ′′R,α(u)(v,v) = J ′′R(u)(v,v) − α‖v‖22. So thenecessary condition J ′′R,α(u)(v,v) ≥ 0 for any v holds, which amounts to the second ordersufficient condition (4.54).

66

Conversely, let the second order sufficient condition (4.54) hold. In lemma 4.10 below, we checkthat

(4.55) |R2(v)| = O(‖v‖∞‖v‖22).

So, when ‖v‖∞ is small enough, |R2(v)| ≤ 12α′‖v‖22. In view of the Taylor expansion (4.50), we

deduce that

(4.56) JR(u + v) = JR(u) + 12α′‖v‖22,

so that the quadratic growth condition (4.53) holds for any α < α′.

Remember that we will complete the above proof by establishing (4.55) in lemma 4.10.

2.2. More on the Hessian of the reduced cost. The Lagrangian associated with theunconstrained problem (4.5) is, see (3.32):

(4.57)J(u,y) +

∫ T

0p(t) · (f(u(t),y(t))− y(t))dt

=

∫ T

0H(u(t),y(t),p(t))dt+ ϕ(y(T ))−

∫ T

0p(t) · y(t))dt.

Denote by D2H(t) the second derivative of the pre-Hamiltonian w.r.t. to (u,y), at the point(u(t), y(t), p(t)). The second derivative of the Lagrangian in the direction (v, z) ∈ U × Y is

Ω0(v, z) :=

∫ T

0D2H(t)(v(t), z(t))2dt+ ϕ′′(y(T ))(z(T ))2.

We can check that, denoting by z[v] the solution of (2.115) with z0 = 0:

Lemma 4.9. We have Q0(v) = Ω0(v, z[v]), for all v ∈ U .

Proof. We have shown in lemma 2.24 that the Hessian of a reduced cost in a given directionis equal to the Hessian of the ’reduction’ Lagrangian, provided the state and costate are thoseassociated with the control. The result follows.

Lemma 4.10. Relation (4.55) holds.

Proof. It is enough to get the result in the case of a zero final cost. Let v ∈ U and z = z[v].Set, for σ ∈ [0, 1],

(4.58) Hσ(t) := H(t, u(t) + σv(t), y(t) + σz(t), p(t)); δHσ(t) := Hσ(t)−H0(t),

and denote by δH ′′σ(t) the Hessian of δHσ(t) w.r.t. (u, y). The mapping u 7→ y[u] being LipschitzU → Y, we have that for some c > 0 not depending on t:

(4.59) |δH ′′σ(t)| ≤ c‖v‖.

By lemma 4.9 and the similar expression of the Hessian of Lagrangian at the point u + σv:

(4.60)∣∣J ′′R((u + σv)− J ′′R(u))(v,v)

∣∣ ≤ ∫ T

0|δH ′′σ(t)||v(t)|2dt ≤ c‖v‖‖v‖22.

We conclude then with (4.51).

Theorem 4.11. Let second order sufficient condition (4.54) hold. Then p(0) is a regularzero of the shooting function T .

Proof. We prove a more general statement in theorem 4.14 (since the qualification conditionautomatically holds when the initial state is given and there is no terminal constraints).

Remark 4.12. Conversely it can be proved that, if (4.54) does not hold, p(0) is not a regularzero of the shooting function T .

67

2.3. Case of initial-final equality constraints. We come back to problem (4.22), as-suming the qualification condition (4.23) to hold so that we skip the argument β = 1 and setλ := (p, Ψ). The Lagrangian (to be taken with β = 1) is stated in (3.32). Its Hessian at thepoint (u, y, p, Ψ), in the direction (v, z) ∈ U × Y is

(4.61) Ω0[λ](v, z) :=

∫ T

0D2H(t)(v(t), z(t))2dt+ D2LIF (y(0), y(T ), Ψ)(z(0), z(T ))2,

where by D2LIF we understand the second derivatives of LIF w.r.t. (y0, yT ). Set

Q[λ](v, z0) := Ω[λ](v, z[v, z0]).

Since this quadratic form is continuous in the norm of U2, we can extend it to (v, z0) ∈ U2×Rn.The second-order conditions involve the critical cone whose elements, called critical direc-

tions, are defined as follows. Set z = z[v, z0]

(4.62) C2(u, y) :=

(v, z0) ∈ U2 × Rn; ϕ′(y(0), y(T ))(z(0), z(T )) = 0Φ′(y(0), y(T ))(z(0), z(T )) = 0

.

Note that for equality constraints the critical cone is actually a closed subspace of U × Y.

Theorem 4.13. If (u, y) is a qualified local solution of (4.22), with unique associated La-grange multiplier λ, for any critical direction (v, z0), we have that Q[λ](v, z0) ≥ 0.

Proof. This follows from proposition 2.32.

2.4. Links with the shooting algorithm. We keep the framework of section 1.2 (theinitial-final constraints are equalities) with (u, y) local solution of the problem. The shootingfunction T IF is defined in (4.24). We have discussed in section 1.2 the interest of checking ifthe Jacobian of this function is invertible at the solution. This boils down to check if the kernelof the Jacobian is reduced to 0. We know that the qualification condition (4.23) ensures theuniqueness of the multiplier denoted by λ (with the normalisation β = 1), and that, by theorem4.8, (4.53) is equivalent then for some α > 0 to

(4.63) Q[λ](v, z0) ≥ α(‖v‖22 + |z0|2), for all (v, z0) ∈ C2(u, y).

In particular, the quadratic cost Q[λ](v, z0)−α(‖v‖22 + |z0|2) attains its minimum at (v, z0) = 0.Applying the PMP to this problem we deduce that the strong Legendre-Clebsch condition (4.8)holds.

Theorem 4.14. We assume satisfied the qualification condition (4.23) and the second ordercondition (4.63). Then the Jacobian of T IF at (y0, p0, Ψ) is invertible.

Proof. An element (z0, q0, δΨ) of KerDT IF (y0, p0, Ψ) is, by (4.24), characterized by

(4.64)

q0 + LIFy0y0(·)z0 + LIFy0y(T )(·)z(T ) + Φy0(y0, y(T ))†δΨ = 0,

−q(T ) + LIFyT y0(·)z0 + LIFyT yT (·)z(T ) + ΦyT (y0, y(T ))†δΨ = 0,

DΦ(y0, y(T ))(z0, z(T )) = 0,

with (v, z,q) solution of (4.18). We recognize the optimality conditions of the problem

(4.65) Minv,z0

Q[λ](v, z[v, z0]); DΦ(y0, y(T ))(z0, z[v, z0](T )) = 0.

By the second-order necessary condition (theorem 4.13), this problem has a convex cost overthe feasible domain (which is a vector space), and so (v, z0) minimizes Q[λ](·) over the criticalcone.

If (4.63) holds then Q[λ](·) is strictly convex over the critical cone, therefore its optimalitycondition is satisfied only at zero, i.e., (v, z0) = 0, and therefore z[v, z0] = 0. Then

(4.66) − q = f †yq.

68

Now for a(t) ∈ L∞(0, T ;Rn), let z = z[v, z0, a] be the solution of

(4.67) z(t) = Df(v(t), z(t)) + a(t).

We obtain that(4.68)

DΦ(z0, z(T )) = [q · z]T0 =

∫ T

0(q(t) · z(t)dt+ q(t) · z(t)) dt =

∫ T

0q(t) ·

(fu(t)v(t) + a(t)

)dt.

For any a(t) ∈ L∞(0, T ;Rn), by the qualification condition (4.23) , there exists (v, z0) ∈ U ×Rn

such that z = z[v, z0, a] satisfies DΦ(z0, z(T )). It follows that∫ T

0 q(t) · a(t)dt = 0 for any sucha, proving that q = 0 and therefore DΦ†Ψ = 0. But (4.23) implies that DΦ is onto, so thatDΦ† is injective, implying that Ψ = 0 as well, whence the injectivity of the linearized shootingfunction, as was to be shown.

3. Control constraints

We briefly show how to express the shooting function in the case of a scalar control (m = 1)subject to the bound constraint

(4.69) u(t) ≥ 0, t ∈ (0, T ),

the optimal control being continuous, and the constraint being active on an interval of the form(τ , T ) with τ ∈ (0, T ). The optimality system reads

(4.70) y(t) = f(u(t),y(t)) − p(t) = ∇yH(u(t),y(t),p(t)), t ∈ (0, τ),

(4.71) Hu(u,y,p) = 0, t ∈ (0, τ); u(t) = 0, t ∈ (τ, T ),

with initial-final conditions

(4.72) y(0) = y0; p(T ) = ∇ϕ(y(T )),

and the junction condition

(4.73) Hu(0,y(τ),p(τ)) = 0.

As in the study of variable horizon problems in section 2.2, we may rewrite an equivalent systemwith functions of the normalized time s ∈ [0, 1]. Indeed, set uN (s) := u(sτ) and

(4.74) y1N (s) := y(sτ); y2

N (s) := y(τ + s(T − τ)),

(4.75) p1N (s) := p(sτ); p2

N (s) := y(τ + s(T − τ)).

We easily see that the optimality system (4.70)-(4.73) is equivalent to, for s ∈ [0, 1]:

(4.76) y1N (s) = τf(uN (s),y1

N (s)),

(4.77) y2N (s) = (T − τ)f(0,y2

N (s)),

(4.78) − p1N (s) = τ∇yH(uN (s),y1

N (s),p1N (s)),

(4.79) − p2N (s) = (T − τ)∇yH(0,y2

N (s),p2N (s)),

(4.80) Hu(uN (s),y1N (s),p1

N (s) = 0,

(4.81) Hu(0,y2N (s),p2

N (s) = 0,

with initial-final conditions (matching those in (4.72), with in addition what corresponds to thecontinuity of state and costate at time τ):

(4.82) y1N (0) = y0; p2

N (1) = ∇ϕ(y2N (1)),

(4.83) y1N (1) = y2

N (0); p1N (1) = p2

N (0),

69

and the junction condition

(4.84) Hu(uN (s),y1N (s),p1

N (s)) = 0.

We integrate the ’normalized’ system (4.76)-(4.81) with initial condition (y0, y02) for the stateand (p01, p02) for the costate, the control being expressed as a function of the state and costate.The shooting equation reads

(4.85) T (y02, p01, p02, τ) =

y1N (1)− y02

p1N (1)− p02

p2N (1)−∇ϕ(y2

N (1))

Hu(0,y1N (1),p1

N (1))

.

4. Notes

Classical references on shooting methods are [20, 34, 41]. For problems with singular arcs,see [31]; there are typically more shooting equations than unknowns, see [2]. For the extensionto state constrained problems, see [32] and [10].

70

5State constrained problems

Contents

1. Pontryagin’s principle 711.1. Setting 711.2. Duality in spaces of continuous functions 721.3. Variation of the pre-Hamiltonian 741.4. Decision variables, variable horizon 762. Junction conditions 772.1. Representation of control constraints 772.2. Constraint order and junction conditions 772.3. Examples 813. Proof of Pontryagin’s principle 833.1. Some results in convex analysis 833.2. Renormalization and distance to a convex set 843.3. Diffuse perturbations. 853.4. Proof of theorem 5.5 86

1. Pontryagin’s principle

Consider now problems with additional inequality constraints on the state at each time.We will need an adapted version of Pontryagin’s principle in order to express the optimalityconditions of such problems.

1.1. Setting. We recall the following definitions: the cost function

(5.1) JIF (u,y) :=

∫ T

0`(t,u(t),y(t))dt+ ϕ(y(0),y(T )),

the state equation

(5.2) (i) y(t) = f(t,u(t),y(t)), for a.a. t ∈ [0, T ]; (ii) y(0) = y0,

the control constraint, where Uad is a closed subset of Rm,

(5.3) u(t) ∈ Uad, for a.a. t ∈ [0, T ],

and the two point constraints

(5.4) Φ(y(0),y(T )) ∈ KΦ.

71

In this chapter we add state constraints of the form

(5.5) gj(t,y(t)) ≤ 0, j = 1, . . . , ng, for all t ∈ [0, T ].

We consider the following optimal control problem:

(5.6) Min(u,y)∈U×Y

JIF (u,y) s.t. (5.2)-(5.5),

where we recall that

(5.7) U := L∞(0, T,Rm); Y := W 1,∞(0, T,Rn).

About the data we assume (here and for extensions to problems with optimization parameters)that

(5.8) We take the same hypotheses on data as in chapter 3 with in addition g of class C1.

1.2. Duality in spaces of continuous functions. Pontryagin’s principle involves multi-pliers associated with constraints. In the case of the state constraint we must first choose thefunction space in which this constraint is expressed, and see what are the properties of the dualspace, where the multipliers live. It appears that a convenient choice is Xg := C([0, T ])ng , thespace of continuous functions over [0, T ] with value in Rng since it is a Banach space whose dualhas a nice structure.

1.2.1. Bounded variation functions. We recall that the total variation of a function µ over[0, T ] is

(5.9) var(µ) := sup

npi∑i=0

|µ(ti+1)− µ(ti)|; where (ti) is a subdivision of [0, T ]

,

(subdivisions were defined in (3.107)). We say that µ has bounded variation if var(µ) < ∞,and denote by BV (0, T ) the set of functions with bounded variation. The Stieltjes integral ofh ∈ C([0, T ]) associated with the bounded variation function µ is

(5.10)∫ T

0h(t)dµ(t) := lim

npi∑i=0

h(τi)(µ(ti+1)− µ(ti)),

where τi ∈ [ti, ti+1], the limit being taken over subdivisions of vanishing maximal increment. Weendow the space C([0, T ]) of continuous functions over [0, T ] with the supremum norm

(5.11) ‖h‖∞ := max|h(t)|; t ∈ [0, T ].

We obviously have that

(5.12)∣∣∣∣∫ T

0h(t)dµ(t)

∣∣∣∣ ≤ var(µ)‖h‖∞.

Therefore, any Stieltjes integral defines a linear continuous operator over C([0, T ]). The conversestatement, due to Riesz, holds, see e.g. Malliavin [29]:

Lemma 5.1. Any continuous linear form on C([0, T ]) is of the type h 7→∫ T

0 h(t)dµ(t), whereµ is a bounded variation function.

In the sequel we may assume that µ belongs to the normed space

(5.13) BV0(0, T )ng := µ ∈ BV (0, T )ng ; µ(T ) = 0,

endowed with the norm

(5.14) ‖µ‖BV0 := var(µ).

72

1.2.2. Costate equation. We recall that, for optimal control problems in the above format,but without state constraints, the Lagrangian was defined in (3.32) as(5.15)

L(β,u,y,p,Ψ) := βJIF (u,y) +

∫ T

0p(t) · (f(t,u(t),y(t))− y(t))dt+ Ψ · Φ(y(0),y(T ))

=

∫ T

0(H(β, t,u(t),y(t),p(t))− p(t) · y(t)) dt+ LIF (β,y(0),y(T ),Ψ).

Here β ∈ 0, 1, u ∈ U , y ∈ Y, p belongs to the costate space P := BV (0, T )n (we explain whybelow) Ψ ∈ RnΦ . The Lagrangian say Lg of the state constrained problem (5.6) has a similarexpression, with the additional contribution of the state constraint:

(5.16) Lg(β,u,y,p,Ψ,µ) := L(β,u,y,p,Ψ) + ∆g(y,µ),

where µ ∈ BV0(0, T )ng and ∆g : Y ×BV0(0, T )ng → R, is defined by

(5.17) ∆g(y,µ) :=

ng∑j=1

∫ T

0gj(t,y(t))dµj(t).

Again, the costate equation is obtained by expressing the stationarity of the Lagrangian w.r.t.the state, i.e. the relation

(5.18) DyLg(β,u,y,p,Ψ,µ)z = 0, for all z ∈ Y.

As for problem (3.29), we obtain the same terms as in (3.33), except for the above linearizationof ∆g(y,µ), and for the integration by parts in the term involving z.

Lemma 5.2. Let p ∈ P and z ∈ Y. Then the following integration by parts formula is valid:

(5.19)∫ T

0p(t) · z(t)dt = p(T ) · z(T )− p(0) · z(0)−

n∑i=1

∫ T

0zi(t)dpi(t).

Proof. We may assume that n = 1. The sum similar to (5.10), with t0 = 0, tN+1 = T ,τi := ti, for i = 0 to N , reads

(5.20)

N∑i=0

z(ti)(p(ti+1)− p(ti)) =N+1∑i=1

z(ti−1)p(ti)−N∑i=0

z(ti)p(ti)

= −N∑i=1

p(ti)(z(ti)− z(ti−1)) + z(tn)p(T )− z(0)p(0)

= −N∑i=1

p(ti)

∫ ti

ti−1

z(t)dt+ z(tn)p(T )− z(0)p(0)

= −∫ tN

0p(τ(t))z(t)dt+ z(tN )p(T )− z(0)p(0),

where τ(t) :=∑N

i=1 ti1t∈(ti−1,ti)(t). Denote by hN := maxti+1− ti; i = 0, . . . , N the supremumof the stepsizes of the subdivision. When N ↑ ∞ (by this we understand that also hN → 0),then τ(t) → t, for all t ∈ [0, T ]. Having in mind that a bounded variation function is a.e.continuous, it follows that p(τ(t)) → t a.e. By the dominated convergence theorem, the aboveintegral converges to the l.h.s. of (5.19). So, since z is continuous and therefore z(tN )→ z(T ),∫ T

0z(t)dp(t) = lim

N

N∑i=0

z(ti)(p(ti+1)− p(ti)) = −∫ T

0p(t) · z(t)dt+ p(T ) · z(T )− p(0) · z(0)

as was to be proved.

73

Let β ∈ 0, 1, u ∈ U , y ∈ Y, p ∈ P, Ψ ∈ RnΦ , µ ∈ BV0(0, T )ng ; Thanks to (5.19), one caneasily express the directional derivative of the Lagrangian w.r.t. the state in a direction z ∈ Y,and deduce that the costate equation is, the pre-Hamiltonian H(·) and end points LagrangianLIF (·) being defined in (3.30)-(3.31):

(5.21)

−dp(t) = ∇yH(β, t, u(t), y(t), p(t))dt+

ng∑j=1

∇ygj(t, y(t))dµj(t), for t ∈ [0, T ],

−p(0) = ∇y0LIF (β, y(0), y(T ), Ψ),p(T ) = ∇yTLIF (β, y(0), y(T ), Ψ).

Remark 5.3. Let e(t) := ∇yH(β, t, u(t), y(t), p(t)). The first equality in (5.21), being anequality between measures, means that, for any ϕ ∈ C([0, T ])n:

(5.22)n∑i=1

∫ T

0ϕi(t)dpi(t) +

∫ T

0ϕ(t) · e(t)dt+

ng∑j=1

∫ T

0ϕ(t) · ∇ygj(t, y(t))dµj(t) = 0.

We call Pontryagin multiplier, associated with the nominal trajectory (u, y), any λ =(β, Ψ, µ, p) satisfying (5.21), the relations

(5.23) Ψ ∈ NKΦ(Φ(y(0), y(T ))); β ∈ 0, 1,

the Hamiltonian inequality similar to (3.38), i.e.,

(5.24) H(β, u(t), y(t), p(t)) = infu∈Uad

H(β, u, y(t), p(t)) for a.a. t ∈ (0, T ).

and the complementarity relations

(5.25) dµ ≥ 0;

∫ T

0gj(t, y(t))dµj(t) = 0, j = 1, . . . , ng, µ(T ) = 0,

and of non triviality

(5.26) β + |Ψ|+ ‖µ‖BV0 > 0.

Remark 5.4. By dµ ≥ 0 we mean that∫ T

0 ϕ(t)dµ ≥ 0 whenever ϕ ∈ Xg has nonnegativevalues. This is equivalent to say that the function µ is nondecreasing.

Theorem 5.5. Any (u, y) solution of (2.141), is a Pontryagin extremal.

Proof. We postpone to section 3 the proof of this difficult result.

1.2.3. Link with Lagrange multipliers; convex problems. We have a natural extension of theanalysis of section 1.2.3 in chapter 3 to the state constrained case. Assuming Uad to be convex,we say that λ := (β, Ψ, p, µ) is a Lagrange multiplier if it satisfies the same relations as forPontryagin multipliers, but with (3.46) instead of the Hamiltonian inequality. Then obviouslyΛP (u, y) ⊂ ΛL(u, y).

Lemma 5.6. Let λ := (β, Ψ, p, µ) be a regular (β = 1) Lagrange multiplier, such that theassociated Lagrangian defined in (3.32) is a convex function of (u,y). Then (u, y) is solution ofproblem (3.29).

Proof. Adapt the arguments in the proof of lemma 3.20.

1.3. Variation of the pre-Hamiltonian.

74

1.3.1. Constant pre-Hamiltonian for autonomous systems. We say that the optimal controlproblem is autonomous if the dynamics and running state and control constraints do not dependon time. For autonomous state-constrained problems, we still can prove that the pre-Hamiltonianis constant. The proof is based on a time dilation approach.

Lemma 5.7. If (u, y) is solution of the state constrained optimal control problem, the databeing autonomous, then it has an associated Pontryagin multiplier such that the pre-Hamiltonianis constant.

Proof. We may assume that there is no integral cost. Consider the change of time t =∫ τ0 w(s)ds, with w(s) ∈ [1

2 , 2] a.e. and τ ∈ [0, T ], for t ∈ (0, T ). We have that t = θ(τ), withθ(τ) = w(τ). Set

(5.27) u(τ) := u(θ(τ )); y(τ) := y(θ(τ )).

We easily check that for τ ∈ [0, T ]:

(5.28)

d

dτy(τ) = w(τ)f(u(τ), y(τ)),

d

dτθ(τ) = w(τ).

So, we may consider an optimal control problem with state (y, θ), control variable (u,w), whoseexpression is

(5.29)

Minϕ(y(0), y(T )); subject to (5.28), Φ(y(0), y(T )) ∈ KΦ andu(τ) ∈ Uad a.e.; gj(τ, y(τ)) ≤ 0, τ ∈ [0, T ]; θ(T ) = T.

By (5.27) any feasible trajectory of (5.29) is associated with a feasible trajectory of the originalproblem with same cost (and conversely). It follows that a solution of (5.29) is obtained whenw(τ) = 1 for a.a. τ , so that θ(τ) = τ , and (u, y) = (u, y). The costate of (5.29) can be denotedas (p, q), where p, q are associated resp. with y and θ; having zero derivative, q is constant(equal to the multiplier associated with the terminal constraint θ(T ) = T ). The pre-Hamiltonianand its value for the considered trajectory are resp.

(5.30) w(p · f(u, y) + q); w(t)(H(t) + q(t)).

By the Hamiltonian inequality, for a.a. t ∈ (0, T ), its minimum over [1/2, 2] is attained atw(t) = 1, which implies that H(t) = −q(t) is constant. In addition, since w(t) = 1 and τ = ta.e., the costates p and p coincide through the considered change of time. In this way, anyPontryagin multiplier for (5.29) corresponds in an obvious way to a Pontryagin multiplier forthe original one, with constant Hamiltonian. The conclusion follows.

1.3.2. Non autonomous systems. We can rewrite the optimal control problem (5.6) as anautonomous one, viewing the time as an additional state variable τ , with dynamics τ = 1 andinitial value τ(0) = 0.

The corresponding pre-Hamiltonian is Ha := H + q, and the additional costate q satisfies

(5.31) − dq(t) = Ht(t)dt+

ng∑j=1

Dtgj(t, y(t))dµj(t), for t ∈ [0, T ]; q(T ) = 0.

Set

(5.32) h(t) := infu∈Uad

H(β, t, u, y(t), p(t)).

Since Ha(t) = H(t) + q(t) is constant by lemma 5.7, this means that h(t) (equal to H(t) a.e.)has bounded variation and satisfies dh(t) = −dq(t). We obtain:

75

Lemma 5.8. We have that

(5.33) dh(t) = −dq(t) = Ht(t)dt+

ng∑j=1

Dtgj(t, y(t))dµj(t), for t ∈ [0, T ].

In particular:

(5.34)d

dth(t) = Ht(t), for a.a. t ∈ (0, T ), if the state constraints are autonomous.

1.4. Decision variables, variable horizon.1.4.1. Decision variables. As in chapter 3 we can extend Pontryagin’s principle to the case

when decision variables (not depending on time) are involved, by viewing them as additionalstate variables, with zero dynamics.

Lemma 5.9. Consider the state constrained optimal control problem similar to (5.6), but withdecision variables π, entering as additional arguments in functions f , `, Φ, g. Assume that thedata satisfy (3.64) and, in addition:

(5.35)g is a C1 function, the mappings g, gy, gπ beinguniformly locally Lipschitz w.r.t. (y, π).

Then the PMP holds with the same expression as for the corresponding problem with decisionvariables set at their nominal value, with in addition the condition of stationarity of the La-grangian w.r.t the decision variables, i.e. (compare to (3.71)):

(5.36)

∫ T

0∇πH(β, t, u(t), y(t), π, p(t))dt+∇πLIF (β, y(0), y(T ), π, Ψ)

+

∫ T

0∇πg(t, y(t), π))dµ(t) = 0.

Proof. The proof is similar to the one of the corresponding result for problems withoutstate constraints, i.e., lemma 3.26.

1.4.2. Variable horizon. We can then deal with variable horizon problems, again as in chapter3, by formulating a ’normalized time’ problem with a normalized time τ ∈ [0, 1], where thehorizon T enters in the dynamics and integral cost. We obtain the following extension of lemma3.28:

Lemma 5.10. If the initial time is fixed and the final time is free, the final cost not dependingon the final time, the PMP has the same expression as in the case of a fixed final time, with inaddition (assuming that the final time does not enter in the constraints) the condition

(5.37) h(T ) = 0.

Proof. Let LN be defined as in (3.87). The Lagrangian function of the normalized timeproblem is

(5.38) LN := LN +

ng∑j=1

∫ 1

0gj(Tτ,y

N (τ))dµNj (τ).

A s in (3.88), set H(τ) := H(β, Tτ,uN (τ),yN (τ),pN (τ)), which is a.e. equal to

(5.39) hN (τ) := infu∈Uad

H(β, Tτ, u,yN (τ),pN (τ)).

The optimality condition is, using (5.33):

(5.40)0 =

∂LN

∂T=

∫ 1

0

(H(τ) + TτHt(τ)

)dτ + T

∫ 1

0

ng∑j=1

τDtgj(Tτ, y(τ))dµNj (τ)

=

∫ 1

0

(hN (τ)dτ + τdhN (τ)

)=

∫ 1

0d(τ hN (τ)) = hN (1).

76

The conclusion follows.

Remark 5.11. The costate equation for the normalized time problem is

(5.41) − dpN (τ)) = T∇yH(τ)dτ +

ng∑j=1

∇ygj(Tτ, y(τ))dµNj (τ).

Since pN (τ) = p(Tτ), dpN (τ) = Tdp(Tτ) and therefore dµN (τ) = Tdµ(Tτ). Since both sidesvanish when τ = 1 if follows that

(5.42) µN (τ) = µ(Tτ), τ ∈ [0, 1].

Remark 5.12. (i) Although h(t) may be discontinuous, its value at time T makes sensesince this is a bounded variation function.(ii) In the more general case of varying initial and final time and of initial-final cost and con-stratints depending on the initial and final time, we have a natural extension of lemma 3.32 forwhich (3.104) still holds.

2. Junction conditions

In this section we assume that the functions involved in the expression of dynamics andconstraints are of class C∞.

2.1. Representation of control constraints. Consider the case when

(5.43) Uad = u ∈ Rm; ci(u) ≤ 0, i = 1, . . . , ncu,where the functions ci(u) are C2 with Lipschitz second derivatives. Set

(5.44) h(t, u) := H(β, t, u, y(t), p(t)).

By the Hamiltonian inequality, for a.a. t, u(t) is solution of the problem

(5.45) Minu∈Rm

h(t, u); ci(u) ≤ 0, i = 1, . . . , ncu.

Denote the set of active control constraints at time t by

(5.46) I0(t) := 1 ≤ i ≤ ncu; ci(u(t)) = 0.We assume in the sequel that the following qualification condition for the control constraintsholds: ∇uci((u(t)), i ∈ I0(t) is uniformly linearly independent, i.e.,(5.47)

There exists α > 0 such that for all t: α|λ| ≤

∣∣∣∣∣∣∑i∈I0(t)

λi∇uci(u(t))

∣∣∣∣∣∣ if λi = 0, i 6∈ I0(t) .

From the Lagrange multiplier theory (see [15, Section 5.2]) we deduce that there exists a uniqueLagrange multiplier λ(t) ∈ Rncu such that

(5.48) ∇uh(t, u(t)) +∑i∈I0(t)

λi(t)∇uci(u(t)) = 0; λ(t) ≥ 0; λ(t) · c(u(t)) = 0.

It can easily be checked that λ is measurable and belongs to L∞(0, T ;Rncu). Defining theaugmented Hamiltonian as

(5.49) Ha(β, t, u, y, p, λ) := H(β, t, u, y, p) + λ · c(u)

we observe that (5.48) can be rewritten in the form

(5.50) ∇uHa(β, t, u(t), y(t), p(t), λ(t)) = 0;

λ(t) · c(u(t)) = 0, for a.a. t.

2.2. Constraint order and junction conditions.

77

2.2.1. Order of a state constraint. Let the total derivative of gj (along the dynamics f) bedefined by

(5.51) g(1)j (t, u, y) := Dygj(t, y)f(t, u, y) +Dtgj(t, y).

Note that

(5.52) Dug(1)j (t, u, y) = Dygj(t, y)fu(t, u, y).

Along the trajectory (u,y), g(1)j (t,u(t),y(t)) is the time derivative of gj(t,y(t)). If g(1)

j (u, y)

depends on u, i.e., if Dug(1)j (t, u, y) does not identically vanishes, we say that gj is a first order

state constraint. Otherwise, writing g(1)j as function of t and y only, we may compute the total

derivative of the latter, i.e., skipping some arguments:

(5.53)g

(2)j (t, u, y) := Dyg

(1)j (t, y)f(t, u, y) +Dtg

(1)j (t, y)

= D2yygj(t, y)(f, f) +Dygj(t, y)fyf +D2

tygj(t, y)f +Dtg(1)j (t, y).

If g(2)j (t, u, y) depend on u, we say that gj is a second order state constraint, etc. We can interpret

the control constraints as zero order state constraints.In the sequel we denote by qj the order, assumed to be well-defined, of gj , by Ir ⊂ 1, . . . , ng

the set of constraints of order r, and Ir(t) those active at time t.2.2.2. Costate jumps. We denote the jumps of function of time having left and right limits

at τ ∈ [0, T ] (this is the case for functions with bounded variation) by a bracket, e.g.

(5.54) [p(τ)] := p(τ+)− p(τ−),

where we set p(0−) := p(0) and p(T+) := p(T ). We also set νj(τ) := [µj(τ)]. In view of (5.21),we have, for all τ ∈ [0, T ]:

(5.55) − [p(τ)] =

ng∑j=1

νj(τ)∇ygj(τ, y(τ)).

2.2.3. Continuity of the multipliers.

Lemma 5.13. Let ¯u and fu be continuous at time τ ∈ (0, T ) (this holds in particular if u is

continuous at time τ). Then

(5.56) [∇uH(τ)] = −∑

j∈I1(τ)

νj(τ)∇ug(1)j (τ).

Proof. In view of (5.55), we have that

(5.57) [∇uH(τ)] = fu(τ)†[p(τ)] = −ng∑j=1

νj(τ)fu(τ)†∇ygj(τ, y(τ)).

Note that

(5.58) fu(τ)†∇ygj(τ, y(τ)) = (Dygj(τ, y(τ))fu(τ))† =(Dug

(1)j

)†= ∇ug(1)

j (τ).

if j 6∈ I1, by the definition of the order of a state constraint, this is equal to zero. On the otherhand, if gj(τ, y(τ)) < 0 then νj(τ) = 0. So, in (5.57) we may reduce the sum over j to I1(τ)and the conclusion follows using again (5.58).

Proposition 5.14. (i) Assume that

(5.59) ¯u, fu and ∇ci(u(τ)), j ∈ I0(τ) are continuous at time τ ∈ (0, T ).

Then

(5.60)∑

j∈I0(τ)

[λj(τ)]∇cj(u(τ))−∑

j∈I1(τ)

νj(τ)∇ug(1)j (τ) = 0.

78

(ii) Assume in addition that

(5.61) ∇ci(u(τ)), j ∈ I0(τ) ∪ ∇ug(1)j (t, u(τ), y(τ)), j ∈ I1(τ) is linearly independent.

Then λ and µj , j ∈ I1 are continuous at time τ .

Proof. (i) If cj(u(τ)) < 0 for some 1 ≤ j ≤ ncu, then λj(τ±) = 0 = [λj(τ)]. Combining

with (5.50) we get that

(5.62) 0 = [∇uHa(τ)] = [∇uH(τ)] +∑

j∈I0(τ)

[λj(τ)]∇cj(u(τ)).

We conclude using lemma 5.13.(ii) Immediate consequence of point (i), having in mind that if a state constraint is not active,the corresponding component is constant near time τ .

Remark 5.15. An obvious sufficient condition for (5.59) is that the control is continuous attime τ . Another case is when `, f and c are affine functions of the control. This is also the casewhen u = (u′, u′′) ∈ Rm′ × Rm′′ with u′ continuous at time τ and for some functions `0, `i, f0,f i, c0, and constant c ∈ Rm′′ :

(5.63)

`(t, u, y) = `0(t, u′, y) +

∑m′′

i=1 u′′i `i(t, y),

f(t, u, y) = f0(t, u′, y) +∑m′′

i=1 u′′i f

i(t, y),

c(u) = c0(u′) +∑m′′

i=1 u′′i ci.

2.2.4. Continuity of the control. Let (u, y) be a Pontryagin extremal, (β, p, µ) be an asso-ciated Pontryagin multiplier, and let τ ∈ (0, T ). For τ ∈ (0, T ), set

(5.64) p− := p(τ−); p+ := p(τ+); pσ := (1− σ)p− + σp+, σ ∈ [0, 1].

We may define in the same way u±, uσ, λ±, λσ, assuming that u and λ have right and leftlimits at time τ .

Definition 5.16. We say thay Ha(β, t, ·, y(τ),pσ,λσ) is uniformly strongly convex in av-erage between τ− and τ+, in the direction [u(τ)], if there exists α > 0 such that

(5.65) α|[u(τ)]|2 ≤[∫ 1

0DuuH

a(β, t,uσ, y(τ),pσ,λσ)dσ

]([u(τ)], [u(τ)]), for all σ ∈ [0, 1].

Lemma 5.17. Let Ha(β, τ, ·, y(τ),pσ,λσ) be uniformly strongly convex in average betweenτ− and τ+, in the direction [u(τ)]. Then u is continuous at time τ .

Proof. Use F (1)− F (0) =∫ 1

0 F′(σ)dσ, with

(5.66) F (σ) := ∇uHa(β, τ,uσ, y(τ),pσ,λσ).

We get using (5.50):(5.67)

0 = [∇uHa(τ)]

=

∫ 1

0

(Hauu(β, τ,uσ, y(τ),pσ,λσ)[u(τ)] + cu(uσ)†[λ(τ)] + fu(t,uσ, y(τ))†[p(τ)]

)dσ.

With (5.65) and (5.55), and observing that gi,yfu = g(1)i,u = 0 if qi > 1, we obtain that

(5.68)

α|[u]|2 ≤[∫ 1

0Hauu(β, τ,uσ, y(τ),pσ,λσ)dσ

]([u(τ)], [u(τ)])

=

∫ 1

0

− ∑i∈I0(t)

[λi(τ)]∇ci(uσ) +∑i∈I1(t)

νi

∫ 1

0∇ug(1)

i (τ,uσ, y(τ))

[u(τ)]dσ.

79

Next observe that

(5.69)∫ 1

0∇ci(uσ)[u(τ)]dσ = [ci(τ)];

∫ 1

0∇ug(1)

i (τ,uσ, y(τ))[u]dσ = [g(1)i (τ)].

So, we have proved that

(5.70) α|[u(τ)]|2 ≤ −∑i∈I0

[λi(τ)][ci(τ)] +∑i∈I1(t)

νi[g(1)i (τ)].

If i ∈ I0, by the complementarity conditions we have that

(5.71) [λi(τ)][ci(τ)] = −λi(τ+)[ci(τ−)]− λi(τ−)[ci(τ+)] ≥ 0.

Also, for i ∈ I1(τ), t 7→ gi(t, y(t)) attains its maximum at time τ , so that [g(1)i (τ)] ≤ 0. Since

νi ≥ 0, we have proved that the r.h.s. of (5.70) is nonpositive, implying [u(τ)] = 0. Theconclusion follows.

Remark 5.18. Sometimes on can deduce from the Hamiltonian inequality that u is a Lips-chitz function of (t, y(t), p(t)). and then it has left and right limits at time τ . Then so has λ ifin addition the qualification condition (5.47) holds, in view of (5.50).

2.2.5. Jumps of the derivative of the control. We next show the link between a jump of thederivative of the control and those of µ or of its derivative. Below τ ∈ (0, T ) is such that forsome small ε > 0, y, u, p, λ and µ are continuously differentiable over (τ − ε, τ) ∪ (τ, τ + ε).

Lemma 5.19. Let the control be continuous at time τ ∈ (0, T ), and the linear independencehypothesis (5.61) hold. Then(i) Huu(t) and Ha

uu(t) are continuous, and(ii) if the jumps below are well defined, then, for all τ ∈]0, T [:(5.72)Hauu(τ)[ ˙u(τ)] = −

∑j∈I0(τ)

[ ˙λi(τ)]∇ci(u(τ)) +∑

j∈I1(τ)

[ ˙µj(τ)]∇ug(1)j (τ)−

∑j∈I2(τ)

νj(τ)∇ug(2)j (τ).

Proof. (i) Note first that, for 1 ≤ i, k ≤ m:

−[Huiuk(τ)] = −[p(τ)] · fuiuk(τ) =

ng∑j=1

νj(τ)g′j(τ)fuiuk(τ) =

ng∑j=1

νj(τ)D2uiuk

g(1)j (τ).

If j ∈ I1(τ), then νj(τ) = 0 by proposition 5.14(ii). Otherwise, g(1)j (t, u, y) does not depend on

u and so D2uiuk

g(1)j (τ) = 0. Therefore, Huu(t) is continuous. Since λ is continuous at time τ by

proposition 5.14(ii), so is Hauu(t).

(ii) It is enough to give the proof for an autonomous problem, since for a non autonomous one,we may add the time as a state variable with derivative 1, which leaves invariant the expressionof Hu. Let i ∈ 1, . . . ,m. Skipping time arguments, we get for any time close to τ but differentfrom it:

(5.73)d

dtHui =

d

dt

(¯ui + p · fui

)= ¯

uiyf + Huiu˙u + p†fuiyf + ˙p · fui .

Set η := ¯uiyf −∇u ¯· fui . Since g′j fu = 0 if j 6∈ I1(τ), using the costate equation, we get

(5.74)d

dtHui = η(t) + Huiu

˙u + p · (fuiyf − fyfui)−∑j∈I1

˙µj g′j fui .

On the other hand

(5.75)d

dt

(λ · cui(u)

)= ˙λ · cui(u) +

∑j∈I0(t)

λj∂2cj∂ui∂u

˙u.

80

Adding both relations and using (5.50), we obtain that

(5.76) 0 =d

dtHaui = η(t) + Ha

uiu˙u + ˙λ · cui(u) + p · (fuiyf − fyfui)−

∑j∈I1

˙µj g′j fui .

Consequently, using (i) and the continuity of η:

(5.77) Hauiu[ ˙u] = −[ ˙λ] · cui(u) +

∑j∈I1

[ ˙µj ]g′j fui − [p] · (fuiyf − fyfui).

If j 6∈ I1(τ), we have that g′j fui = 0, and therefore

(5.78) 0 =(Dy(g

′j fui)

)f = g′′j (f , fui) + g′j fuiyf ,

and so, with (5.53):

(5.79)g

(2)j,ui

= Dui(g′′j (f , f) + g′j fyf))

= 2g′′j (f , fui) + g′j fuiyf + g′j fyfui

= g′j(fyfui − fuiyf).

Therefore,

(5.80) − [p] · (fuiyf − fyfui) =

ng∑j=1

νj g′j(fuiyf − fyfui) = −

ng∑j=1

νj g(2)j,ui

.

We conclude with (5.77), noting that νj = 0 when qj = 1, and g(2)j,ui

= 0 if qj > 2.

2.3. Examples.

Example 5.20. Let the control be continuous at time τ , (5.61) hold, Huu(τ) be invertible,no control constraints, the state constraint being scalar and of first order. Then

(5.81)

Huu(τ)[ ˙u(τ)] = [ ˙µ(τ)]∇ug(1)(τ);

If g(1)u (τ) 6= 0, then ˙u(τ) is discontinuous iff ˙µ is.

With the same hypotheses, but a second (instead of first) order state constraint, we have that

(5.82)

Huu(τ)[ ˙u(τ)] = −ν(τ)∇ug(2)(τ);

If g(2)u (τ) 6= 0, then ˙u(τ) is discontinuous iff µ is.

With the same hypotheses, but a state constraint of order greater than two, ˙u(τ) is continuous.This illustrates the fact that the behavior of the solution depends on an essential way on theorder of the state constraint.

Example 5.21. For T = 2, minimize∫ T

0 y(t)dt + y(T ) s.t. y(t) = u(t) a.e. over (−1, 1),initial condition y(0) = 1, and state constraint y(t) ≥ 0. The solution is obviously u(t) = −1 andy(t) = 1− t for t ∈ (0, 1), u(t) = y(t) = 0 over (1, 2). The pre-Hamiltonian being H = βy+ pu,the costate equation is

(5.83) − dp = βdt− dµ; p(T ) = β.

Over the open constrained arc C = (1, 2), since u = 0, by the Hamiltonian inequality, p = 0 sothat ˙µ = β. In addition p, and therefore µ, has a final jump of value β.

We claim that β = 0 is impossible. Indeed, we would have p = µ in view of (5.83), so thatp(1+) = µ(1+) = 0. Since µ cannot be zero and is constant over [0, 1) it must have a positivejums at t = 1, so that it is negative as well a p over [0, 1). But then the control cannot havenegative values over [0, 1) in view of the Hamiltonian inequality.

So, necessarily β = 1, p and µ have a jump of 1 at the final time, and over (1, 2), µ(t) = t−3.A nonzero jump at time 1 would imply negative values of p(t) for t < 1 close to 1, contradictingthe Hamiltonian inequality. So, over (0, 1), ˙p(t) = −1 so that p(t) = 1 − t is positive, and theHamiltonian inequality implies as expected that u(t) = −1 a.e.

81

Example 5.22. Same state equation, cost function and control constraint, but now T = 3and the state constraint is

(5.84) x(t) ≥ −12(t− 1)2.

The optimal control u(t) has value -1 over (0, 1), 1 − t over (1, 2), and −1 over (2, 3), so it isdiscontinuous at time 1, and continuous at time 2. The state constraint is active over [1, 2].Consider only the case β = 1. Over (2, 3], since the state constraint is not active and the finalcost is y(T ), p(t) = 1 and µ is zero. Over (1, 2), p(t) = 0 since u is out of bounds. So, attime 2, there is a jump of p and µ of size 1. The analysis over (0, 1) is similar to the one of theprevious example. Observe that, at time 2, there is a jump despite the fact that the control iscontinuous.

Example 5.23. Consider a problem of equilibrium of an elastic string with length T , fixedat endpoints, with obstacle. Specifically, we minimize the sum of elastic and gravity energies

(5.85) E(y) :=

∫ T

0(1

2 y(t)2 + y(t))dt,

where y(t) represents the vertical deformation, with constraints

(5.86) y(0) = y(T ) = 1; y(t) ≥ 0 for all t ∈ [0, T ].

We can reformulate this problem as an optimal control one, with state equation, integrand ofintegral cost, state constraint and initial-final constraints given by

(5.87)

y(t) = u(t), `(u, y) := 1

2u2 + y; g(y) := −y;

Φ(y0, yT ) := (y0 − 1, yT − 1); KΦ := 0R2 .

For large enough T , the optimal state is, for some t0 ∈ (0, 12T ), equal to zero over [t0, t1] with

t1 = T − t0, and positive outside. The pre-Hamiltonian and costate dynamics are

(5.88)

H(u, y, p) = 1

2u2 + y + pu,

−dp(t) = dt− dµ(t).

Since µ(T ) = 0, it follows that

(5.89) p(t) = T − t+ µ(t) + p(T ), t ∈ [0, T ].

By the Hamiltonian inequality, u(t) = −p(t). Over ]t0, t1[, we have therefore

(5.90) 0 = ˙y(t) = u(t) = −p(t),

which implies ˙µ = 1. Over each unconstrained arc, we have that p(t) = c − t for some c ∈ R,and hence, u(t) = t − c. By lemma 5.17, both the control and state constraint multiplier arecontinuous at times t0 and t1, and so, the control is equal to t − t0 over [0, t0[ and t − t1 over[t1, T [. By proposition 5.14, µ is continuous. The relation (5.72) gives [ ˙u(τ)] = −[ ˙µ(τ)]. While˙u has a jump of -1 at t0 and 1 at t1, since ˙µ = 0 out of [t0, t1], the jump of ˙µ is 1 at t0 and -1at t1, so that (5.72) holds, as expected.

Example 5.24. Consider a problem of equilibrium of an elastic beam with length T , fixedat endpoints and with obstacle. We minimize the sum of elastic and gravity energies: E(x) :=∫ T

0 (12 x(t)2 +x(t))dt, where x(t) represents the vertical deformation, subject to x(0) = x(T ) = 1

and x(t) ≥ 0, for all t. The equivalent optimal control problem, in which y1(t) = x(t), has stateequation y1(t) = y2(t), y2(t) = u(t), an integral cost function with integrand `(u, y) = 1

2u2 +y1,

the state constraint g(y) = −y1, and the initial-final constraints y1(0) = y1(T ) = 1. For largeenough T the optimal state is, for some t0 ∈ (0, 1

2T ), equal to zero over [t0, t1] with t1 = T − t0,and positive outside. The pre-Hamiltonian is H(u, y, p) = 1

2u2 +y1 +p1y2 +p2u, and the costate

has dynamics

(5.91)−dp1(t) = dt− dµ(t),− ˙p2(t) = p1(t).

82

Since µ(T ) = 0, it follows that p1(t) = T − t + µ(t) + p1(T ). By the Hamiltonian inequality,u(t) = −p2(t). Over (t0, t1), we have that

0 = ˙y2(t) = u(t) = −p2(t) = − ˙p2(t) = p1(t),

implying ˙µ = 1. Over each unconstrained arc, we have that p1(t) = c1 − t for some c1, andhence, p2(t) = 1

2 t2 − c1t − c2 for some c2, so that u(t) = c1t + c2 − 1

2 t2. By lemma 5.17, the

control is continuous at times t0 and t1, and is therefore of the form −c′1(t− t0)− 12(t− t0)2 over

[0, t0[ and c′′1(t− t1)− 12(t− t1)2 over [t1, T [. The curvature of the beam (i.e. the optimal control)

being nonnegative and continuous with value zero on [t0, t1], we have that [ ˙u(t0)] = c′1 ≥ 0 and[ ˙u(t1)] = c′′1 ≥ 0. By (5.72), the state constraint being of second order, since g(2)

j,u(t) = −1, wehave that [ ˙u(ti)] = ν(ti) ≥ 0, i = 1, 2, which is compatible with the nonnegativity of the jumpsof ˙u, established just before.

3. Proof of Pontryagin’s principle

This section is devoted to the proof of theorem 5.5. We need some preparatory lemmas.

3.1. Some results in convex analysis. In what follows X, Y are Banach spaces. Givenf : X → R we define the subdifferential (in the sense of convex analysis) of f at x ∈ X by

(5.92) ∂f(x) := x∗ ∈ X∗; f(x′) ≥ f(x) + 〈x∗, x′ − x〉, for all x′ ∈ X.

Lemma 5.25. Let A ∈ L(X,Y ) and F (x) := f(x) + g(Ax) with f : X → R and g : Y → R,both convex and continuous. Then

(5.93) ∂F (x) = ∂f(x) +A†∂g(Ax), for all x ∈ X.

Proof. See e.g. [22, Ch. 1, prop. 5.6 and 5.7].

We say that f : X → R is Gâteaux differentiable at x ∈ X iff there exists x∗ ∈ X∗ (calledthe Gâteaux derivative of f at x) such that

(5.94) limt↓0

(f(x+ ty)− f(x))/t = 〈x∗, y〉, for all y ∈ X.

We say that f is Hadamard differentiable at x ∈ X if

(5.95) limt↓0

(f(x+ ty(t))− f(x))/t = 〈x∗, y〉, for all y ∈ X, whenever y(t)→ y when t ↓ 0.

We may write in short G, or H differentiable. Clearly, the Hadamard differentiability impliesthe Gâteaux differentiability; it is easily shown that the converse holds if f is Lipschitz near x.

Lemma 5.26. Let f : X → R be convex and continuous. Then f is H-differentiable at x withG-derivative x∗ ∈ X∗ iff ∂f(x) = x∗.

Proof. By [22, Ch. 1, prop. 5.3], a convex fuction, continuous at x, whose subdifferentialis a singleton, is G-differentiable at x. On the other hand, by [22, Cor. 2.4], being continuous,f is Lipschitz near x. So, as observed above, f is also H-differentiable at x.

We say that a sequence x∗k in X∗ ∗weakly converges to x∗ ∈ X∗ if

(5.96) 〈x∗k, x〉 → 〈x∗, x〉, for all x ∈ X.

We recall that a Banach space X is said to be separable if it contains a dense sequence. ByX∗ we denote its topological dual, i.e., the set of continuous linear forms over X.

Lemma 5.27. Let X be a separable Banach space. Then any bounded sequence on X∗ has a∗weakly convergent subsequence.

83

Proof. (a) Let xk, k ∈ N, be a dense sequence in X, and x∗j be a bounded sequence in X∗.We claim that x∗j ∗weakly converges to x∗ in X∗ iff

(5.97) 〈x∗j , xk〉 → 〈x∗, xk〉, for all k ∈ N.The condition is obviously necessary. Conversely, if it holds, then given x ∈ X and ε > 0, thereexists n ∈ N such that ‖x− xn‖ < ε, and then

(5.98)

lim supj |〈x∗j − x∗, x〉| ≤ lim supj |〈x∗j − x∗, x− xn〉|+ lim supj |〈x∗j − x∗, xn〉|= lim supj |〈x∗j − x∗, x− xn〉|≤ ε

(supj ‖x∗j‖+ ‖x∗‖

)can be made arbitrarily small. Our claim follows.(b) Let now y∗i be a bounded sequence in X∗. By a standard diagonal argument we have thatsome subsequence denoted by x∗j satisfies (5.97). The conclusion follows.

Remark 5.28. See another type of proof in [18, Cor. 3.30].

Lemma 5.29. Let X be a Banach space, K a convex subset of X with nonempty interior,xk ∈ K converge to x, x∗k ∗weakly converge to x∗ in X∗, be such that

(5.99) 〈x∗k, x− xk〉 ≤ 0 for all x ∈ K.If lim infk ‖x∗k‖ > 0, then x∗ 6= 0.

Proof. We may assume that B(0, ε) ⊂ int(K), for some ε > 0. By (5.99):

(5.100) ε‖x∗k‖ = supx∈B(0,ε)

〈x∗k, x〉 ≤ supx∈K〈x∗k, x〉 ≤ 〈x∗k, xk〉,

so that

(5.101) 0 < limk〈x∗k, xk〉 = lim

k〈x∗k, x〉 = 〈x∗, x〉,

proving that 〈x∗k, x〉 > 0. The result follows.

3.2. Renormalization and distance to a convex set. We need the renormalizationlemma 5.30 below. It is a particular case of Day [19, Ch. VII, Sec. 4, Thm 1, p. 160 of thirded.]. For other results on renormalization see Diestel [21, Ch. 4].

Lemma 5.30. Let X be a separable Banach space. Then it has equivalent norm, say N , suchthat the corresponding dual norm is strictly convex.

Proof. Let xk, k ∈ N, be a dense sequence in X. Set yk := xk/‖xk‖ (we may assume thexk to be nonzero). Consider the following function on X∗:

(5.102) N∗(x∗) := ‖x∗‖+N∗1 (x∗), N∗1 (x∗) :=

(∑k∈N

2−k〈x∗, yk〉2)1/2

.

Since 〈x∗, yk〉 ≤ ‖x∗‖, we have that N∗1 (x∗) ≤√

2‖x∗‖, and we easily check that N∗(·) is a norm,N∗ is equivalent to the dual norm, since

(5.103) ‖x∗‖ ≤ N∗(x∗) ≤ γ‖x∗‖; γ := 1 +√

2,

The sum of two convex functions, one of them being is strictly convex, is strictly convex. So,the strict convexity of N∗ will follow from the strict convexity of N∗1 . We can write N∗1 (x∗) =N∗2 (Ax∗) where N∗2 (·) is the strictly convex norm of `2 (the Hilbert space of square summablesequences) and A ∈ L(X∗, `2), defined by (Ax∗)k := 2−k〈x∗, yk〉 for all k ∈ N, is obviouslycontinuous and injective. It is easily checked that the composition of an injective linear mappingby a strictly convex function is strictly convex. So, N∗ is a strictly convex norm over X∗. LetB∗N denote its unit ball:

(5.104) B∗N = x∗ ∈ X∗; ‖x∗‖+N∗1 (x∗) ≤ 1.84

Denoting by B the unit ball of the primal space X, we have that

(5.105) B∗N = ∩x∈BB∗N (x), where B∗N (x) := x∗ ∈ X∗; 〈x∗, x〉+N∗1 (x∗) ≤ 1.

The continuous convex function N∗1 (x∗) being positively homogeneous, it can be expressed atthe support function of its subdifferential at zero (see e.g. [15, equation (2.240)]), that is,

(5.106) N∗1 (x∗) = sup〈x∗∗, x∗〉; x∗∗ ∈ ∂N∗1 (0).

Here x∗∗ is an element of the bidual space X∗∗. Since A ∈ L(X∗, `2), the subdifferential calculusrule in lemma 5.25 applies. It follows that

(5.107) ∂N∗1 (0) = A†B2,

where B2 is the closed unit ball of `2. For z ∈ `2 we have that

(5.108) A†z =∑k∈N

2−k zkyk ∈ X.

So, setting K := B +A†B2, we have that K ⊂ X and

(5.109) B∗N = x∗ ∈ X∗; 〈x∗, x〉 ≤ 1, for all x ∈ K

is an intersection of closed half spaces corresponding to elements of X. The associated primalnorm

(5.110) N(x) := infλ > 0; λ−1x ∈ K

is equivalent to the original one since, by (5.103), K contains B and is contained in γB. Theclosed unit ball of this new norm is the closure of K. Therefore, the associated dual norm isnothing but N∗. The conclusion follows.

Remark 5.31. For a characterization of equivalent dual norms that are the dual of anequivalent primal norm, see Penot [35, Lemma 3.94, p. 251].

Lemma 5.32. Let X be a separable Banach space endowed with the above norm N , and letK be a closed convex subset of X. Then the distance function dK is H-differentiable over X \K.

Proof. Obviously, dK is convex and nonexpansive. Let x ∈ X \ K, x∗ ∈ ∂dK(x), andy ∈ K. Then

(5.111) 0 = dK(y) ≥ dK(x) + 〈x∗, y − x〉.

Let yk ∈ K be such that ‖yk − x‖ → dK(x). Then

(5.112) dK(x) ≤ −〈x∗, yk − x〉 ≤ ‖x∗‖‖yk − x‖ → ‖x∗‖dK(x),

proving that ‖x∗‖ = 1. Therefore every element of the subdifferential has a unit norm. Sincethe dual norm is strictly convex it follows that ∂dK(x) is a singleton. The conclusion followswith lemma 5.26.

3.3. Diffuse perturbations. We next consider the diffuse perturbation theory; our resultsare a variant of Li and Yong [28, p. 143 and 175].

Lemma 5.33. Let w ∈ L1(0, T )m, ε > 0 and ρ ∈ (0, 1). Then there exists a measurablesubset E of (0, T ), of measure Tρ, such that

(5.113)∣∣∣∣∫ T

0(1− 1

ρ1E)w(t)dt

∣∣∣∣ ≤ ε.Proof. The set of simple functions being dense in L1(0, T )m, for any ε > 0, there exists

βk ∈ Rm and disjoint measurable subsets Ak of (0, T ), k = 1 to nε such that

(5.114) ‖w − wε‖1 ≤ ρε, where wε :=∑k≤nε

βk1Ak .

85

For any k ≤ nε and ρ ∈ (0, 1), there exists a measurable subset A′k of Ak such that |A′k| = ρ|Ak|(where by | · | we denote the Lebesgue measure). Then Eε := ∪k≤nεA′k satisfies

(5.115)∣∣∣∣∫ T

0(1− 1

ρ1Eε)w(t)dt

∣∣∣∣ ≤ ∣∣∣∣∫ T

0(1− 1

ρ1Eε)wεdt

∣∣∣∣+ ‖1− 1

ρ1Eε‖∞‖w − wε‖1 ≤ ε,

since the above integral on the r.h.s. is equal to 0, and ‖1 − 1ρ1Eε‖∞ ≤ 1/ρ. The result

follows.

We next need a variant of the previous lemma where we consider now integrals from 0 to t,for all t ∈ [0, T ].

Lemma 5.34. Let w ∈ L∞(0, T )m, and ρk ↓ 0 with values in (0, 1). Then there exists asequence Ek of subsets of (0, T ) such that |Ek| = Tρk, and

(5.116)∣∣∣∣∫ t

0w(s)ds− 1

ρk

∫ t

01Ekw(s)ds

∣∣∣∣ ≤ ρk, for all t ∈ (0, T ).

Proof. For each k in N, let N(k) ∈ N, N(k) 6= 0. Set G(j) := [0, jT/N(k)], for j = 1 toN(k), and (skipping the time argument)

(5.117) ϕk := (1G(1)w, . . . ,1G(N(k))w).

Let εk ↓ 0. By lemma 5.33, there exists a measurable subset Ek of (0, T ) such that |Ek| = Tρk,and

(5.118)∣∣∣∣∫ T

0(1− 1

ρk1Ek)1G(j)w(t)dt

∣∣∣∣ ≤ εk, for all 1 ≤ j ≤ N(k).

Equivalently

(5.119)

∣∣∣∣∣∫ jT/N(k)

0w(s)ds− 1

ρk

∫ jT/N(k)

01Ekw(s)ds

∣∣∣∣∣ ≤ εk, for all 1 ≤ j ≤ N(k).

Let t ∈ (0, T ) and j ∈ N, with Tj/N(k) ≤ t < T (j + 1)/N(k). Then

(5.120)

∆ :=

∣∣∣∣∫ t

0w(s)ds− 1

ρk

∫ t

01Ekw(s)ds

∣∣∣∣≤

∣∣∣∣∣∫ jT/N(k)

0w(s)ds− 1

ρk

∫ jT/N(k)

01Ekw(s)ds

∣∣∣∣∣+

∣∣∣∣∣∫ t

jT/N(k)w(s)ds− 1

ρk

∫ t

jT/N(k)1Ekw(s)ds

∣∣∣∣∣≤ εk +

T

N(k)

(1 +

1

ρk

)‖w‖∞.

By a proper choice of εk and N(k), the conclusion follows.

3.4. Proof of theorem 5.5. It suffices to discuss the case when the problem is autonomous,the inital state is fixed, and the final state is free (since the essential diffficulties remain in thatcase). Set K := C([0, T ])ng . In the spirit of the proof of theorem 3.13 we introduce, for ε > 0,the ’penalized’ cost function (compare to (3.179)):

(5.121) JεR(u) :=([JR(u)− J + ε2]2+ + d2

K([g(y[u]))1/2

.

The distance function dK is computed after renormalization of the separable space C([0, T ])ng ,so that it is Hadamard differentiable out of K. For R ≥ ‖u‖, set

(5.122) UR := L∞(0, T, Uad ∩ B(0, R)).

86

Since JεR is continuous over this set endowed with Ekeland’s metric (3.177), and u is an ε2

minimizer of JεR over UR, by Ekeland’s principle (theorem 3.57), there exists uε ∈ UR such that

(5.123)ρE(uε, u) ≤ ε;JεR(uε) ≤ JεR(u) + ερE(u,uε), for all u ∈ U .

We have that JεR(uε) > 0 (otherwise this would contradict the fact that u is a solution of theproblem). Denote by yε the associated state. Set

(5.124)

βε :=

(J(uε)− J(u) + ε2)+

JεR(uε);

ηε :=dK(g(yε))DdK(g(yε))

JεR(uε)if g(yε) 6∈ K, 0 sinon.

Since ‖DdK(g(yε))‖ = 1, we have that β2ε + ‖ηε‖2 = 1. In addition, since dK(·) is a convex

function, we have, for all w ∈ C([0, T ])−:

(5.125) 〈ηε, w − g(yε)〉 ≤ dK(w)− dK(g(yε)〉 ≤ 0.

Expanding the square root function as in (3.184) we obtain that

(5.126) JεR(u) = JεR(uε) + βε(J(u)− J(uε)) + 12

d2K(g(y))− d2

K(g(yε))

JεR(uε)+ o(‖u− uε‖1).

We next analyze how to get an expansion of the term involving the distance dK . Set f εy [t] :=fy(u

ε(t),yε(t)), and let ξε[u] be defined as in (3.19), but with uε instead of u, i.e.,

(5.127) ξε(t) = f εy [t]ξε(t) + f(u(t),yε(t))− f(uε(t),yε(t)), for a.a. t ∈ (0, T ),

with u ∈ U , and initial condition ξ(0) = 0. This equation has a unique solution in Y, denotedby ξε[u]. Next, fix u ∈ U . Let Ek, ρk be as in lemma 5.34, where

(5.128) w(t) := f(u(t),yε(t))− f(uε(t),yε(t)), t ∈ (0, T ).

Define uk by

(5.129) uk(t) =

u(t) if t ∈ Ek,uε otherwise,

with associated state yk. Set ξk := ξε[uk], and ξ := ξε[u].

Lemma 5.35. Given u, Ek and uk, yk as above, we have that

(5.130) ‖yk − yε − ρkξ‖∞ = o(ρk).

Proof. (i) By lemma 3.9, ‖yk − yε − ξk‖∞ = O(‖uk − uε‖21) = O(ρ2k). So, it suffices to

prove that ‖ξk − ρkξ‖∞ = o(ρk), or equivalently:

(5.131) ‖ξ − ξk/ρk‖∞ = o(1).

(ii) We have that ξ(t) = f εy [t]ξ(t) + w(t), and ξ(0) = 0. The solution of this linear ODE is ofthe form

(5.132) ξ(t) =

∫ t

0A(t, s)w(s)ds,

where the n×n matrix A(t, s) is a Lipschitz function of (t, s), and A(t, t) is equal to the identity.Let W (t) :=

∫ t0 w(s)ds. Integrating by parts in (5.132) (which is valid for a product of Lipschitz

functions) we find that

(5.133) ξ(t) = [A(t, s)W (s)]t0 −∫ t

0

∂A(t, s)

∂sW (s)ds = W (t)−

∫ t

0

∂A(t, s)

∂sW (s)ds.

Next, set, for t ∈ (0, T ):

(5.134) wk(t) := f(uk(t),yε(t))− f(uε(t),yε(t)); Wk(t) :=

∫ t

0wk(s)ds.

87

By the above arguments we have that ξk(t) = Wk(t)−∫ t

0∂A(t,s)∂s Wk(s)ds, and therefore

(5.135) ξ(t)− ξk(t)/ρk = W (t)−Wk(t)/ρk −∫ t

0

∂A(t, s)

∂s(W (s)−Wk(s)/ρk)ds.

By lemma 5.34, Wk/ρk →W uniformly. Therefore, (5.131) follows from (5.135).

In the sequel, u, Ek and uk are defined as before the above lemma. Let ηε ∈ BV (0, T )r besuch that dηε = ηε and ηε(T ) = 0. Let pε ∈ BV ([0, T ])n be solution of (note that, since ` = 0,the pre-Hamiltonian does not depend on β):(5.136)

−dpε(t) = ∇yH(uε(t),yε(t),pε(t))dt+∑r

i=1∇ygi(yε(t))dηεi (t), for a.a. t ∈ [0, T ],pε(T ) = βε∇φ(yε(T )).

Corollary 5.36. We have the following expansion:

(5.137) JεR(uk) = JεR(u) + ρk

∫ T

0(H(t,u(t),yε(t),pε(t))−H(t,uε(t),yε(t),pε(t))) dt+ o(ρk).

Proof. i) By lemma 3.4, since

(5.138) ‖uk − uε‖21 = O(ρk),

we have that

(5.139) J(uk) = J(uε) +

∫Ek

(H(t,u(t),yε(t), pε(t))−H(t,uε(t),yε(t), pε(t)))dt+ o(ρk),

where pε is solution of

(5.140)−dpε(t) = ∇yH(uε(t),yε(t), pε(t)), for a.a. t ∈ (0, T ),pε(T ) = ∇φ(yε(T )).

(ii) Since 12d

2K is H-differentiable with derivative ηε (see (5.124)), and a composition of H-

differentiable functions is H-differentiable, using lemma 5.35, we get that

(5.141) 12d

2K(yk) = 1

2d2K(yε) + 〈ηε,yk − yε〉+ o(ρk).

Extending lemma 3.4 and using again (5.138), we can check that

(5.142) 〈ηε,yk − yε〉 =

∫Ek

(H(t,u(t),yε(t), pε(t))−H(t,uε(t),yε(t), pε(t)))dt+ o(ρk),

where pε is solution of

(5.143)

−dpε(t) = ∇yH(uε(t),yε(t), pε(t))dt+ JR(uε)∑r

i=1∇ygi(yε(t))dηεi (t),for a.a. t ∈ (0, T ),

pε(T ) = 0.

(iii) Observe that pε solution of (5.136) satisfies

(5.144) pε = JR(uε)pε + βεpε.

Since a composition of H-differentiable mappings is H-differentiable, combining the previoussteps, we get the conclusion.

Then, by arguments similar to those in the proof of theorem 3.13 (taking advantage of theintegration by parts formula (5.19)) we deduce that

(5.145)H(βε,u

ε(t),yε(t),pε(t)) ≤ H(βε, u,yε(t),pε(t)) + ε,

for all u ∈ UR, and a.a. t ∈ (0, T ).

Let us take R = Rε = max(‖u‖∞, 1/√ε). Since ρE(uε, u) ≤ ε, we have that, for small enough

ε, ‖uε − u‖1 ≤ 2εRε = 2√ε proving the uniform convergence of yε towards y. Next since

88

β2ε + ‖ηε‖2 = 1, in view of lemma 5.27 and (5.125) we can extract a subsequence εk ↓ 0, such

that (denoting by ∗ the ∗weak convergence)

(5.146)βεk → β ∈ [0, 1]; ηεk

∗ η;

〈η,y − g(y)〉 ≤ 0, for all y ∈ K.We deduce from the costate equation that pεk converges in the weak ∗ topology ofM([0, T ],Rn∗)to p solution of the costate equation (5.21). This implies pεk → p in L1(0, T )n and (up to theextraction of a subsequence) also a.e., see [1, Thm. 3.23]. Passing to the limit in (5.145), weobtain the Hamiltonian inequality. Passing to the limit in the last relation of (5.146) we obtain(5.25). If β > 0, dividing (η, p) by β > 0, we obtain the conclusion with β = 1. Finally, if β = 0,then ‖ηk‖ → 1, and then by lemma 5.29, the non nullity relation (5.26) follows.

89

Bibliography

[1] L. Ambrosio, N. Fusco, and D. Pallara. Functions of bounded variation and free discontinuity problems.Oxford Mathematical Monographs. The Clarendon Press Oxford University Press, New York, 2000. 89

[2] M.S. Aronna, J.F. Bonnans, and P. Martinon. A shooting algorithm for optimal control problems withsingular arcs. J. Optim. Theory Appl., 158(2):419–459, 2013. 3, 70

[3] T. Bayen, F. Mairet, P. Martinon, and M. Sebbah. Analysis of a periodic optimal control problem connectedto microalgae anaerobic digestion. Optimal Control Appl. Methods, 36(6):750–773, 2015. 13

[4] T. Bayen, A. Rapaport, and M. Sebbah. Minimal time control of the two tanks gradostat model under acascade inputs constraint. SIAM J. Control Optim, 52(4):2568–2594, 2014. 3

[5] J.T. Betts. Practical methods for optimal control and estimation using nonlinear programming. SIAM,Philadelphia, PA, second edition, 2010. 3

[6] L.T. Biegler. Nonlinear programming. SIAM, Philadelphia, PA, 2010. 3[7] P. Billingsley. Convergence of probability measures. Wiley Series in Probability and Statistics: Probability

and Statistics. John Wiley & Sons Inc., New York, second edition, 1999. A Wiley-Interscience Publication.28

[8] J. Bonnans, J.F., D. Giorgi, V. Grélard, B. Heymann, S. Maindrault, P. Martinon, O. Tissot, and J. Liu.Bocop – A collection of examples. Technical report, INRIA, 2017. 9

[9] J.F. Bonnans. Optimisation Continue. Dunod, Paris, 2006. 21, 24, 58[10] J.F. Bonnans and A. Hermant. Well-posedness of the shooting algorithm for state constrained optimal control

problems with a single constraint and control. SIAM J. Control Optim., 46(4):1398–1430, 2007. 70[11] J.F. Bonnans and G. Launay. Large scale direct optimal control applied to a re-entry problem. AIAA J. of

Guidance, Control and Dynamics, 21:996–1000, 1998. 13[12] J.F. Bonnans and J. Laurent-Varin. Computation of order conditions for symplectic partitioned Runge-Kutta

schemes with application to optimal control. Numer. Math., 103(1):1–10, 2006. 66[13] J.F. Bonnans, P. Martinon, and E. Trélat. Singular arcs in the generalized Goddard’s problem. J. Optim.

Theory Appl., 139(2):439–461, 2008. 13[14] J.F. Bonnans and P. Rouchon. Commande et optimisation de systèmes dynamiques. Editions de l’Ecole

Polytechnique, Palaiseau, 2005. 3, 52[15] J.F. Bonnans and A. Shapiro. Perturbation analysis of optimization problems. Springer-Verlag, New York,

2000. 3, 77, 85[16] B. Bonnard, L. Faubourg, and E. Trélat.Mécanique céleste et contrôle des véhicules spatiaux. Springer-Verlag,

Berlin, 2006. 3[17] B. Bonnard and D. Sugny. Optimal control with applications in space and quantum dynamics. American

Institute of Mathematical Sciences (AIMS), Springfield, MO, 2012. 3[18] H. Brézis. Functional analysis, Sobolev spaces and partial differential equations. Springer, New York, 2011.

18, 20, 25, 84[19] M.M. Day. Normed linear spaces. Springer-Verlag, New York-Heidelberg, third edition, 1973. Ergebnisse der

Mathematik und ihrer Grenzgebiete, Band 21. 84[20] H.-J. Diekhoff, P. Lory, H.J. Oberle, H.-J. Pesch, P. Rentrop, and R. Seydel. Comparing routines for the nu-

merical solution of initial value problems of ordinary differential equations in multiple shooting. NumerischeMathematik, 27:449–469, 1977. 70

[21] J. Diestel. Geometry of Banach spaces—selected topics. Springer-Verlag, Berlin, 1975. Lecture Notes inMathematics, Vol. 485. 84

[22] I. Ekeland and R. Temam. Convex analysis and variational problems, volume 1 of Studies in Mathemat-ics and its Applications. North-Holland, Amsterdam, 1976. French edition: Analyse convexe et problèmesvariationnels, Dunod, Paris, 1974. 83

[23] A.T. Fuller. Relay control systems optimized for various performance criteria. In Proc. IFAC Congress,Moscow, pages 510–519. Butterworth, London, 1961. 10

[24] B.S. Goh. Management and analysis of biological populations. Elsevier, 1980. 3, 11, 12[25] B.S. Goh, G. Leitmann, and T.L. Vincent. Optimal control of a prey-predator system. Math. Biosci., 19:263–

286, 1974. 12[26] W.W. Hager. Runge-Kutta methods in optimal control and the transformed adjoint system. Numer. Math.,

87(2):247–282, 2000. 66

91

[27] U. Ledzewicz, H. Schättler, A. Friedman, and E. Kashdan, editors. Mathematical methods and models inbiomedicine. Springer, New York, 2013. 3

[28] X. Li and J. Yong. Optimal control theory for infinite-dimensional systems. Systems & Control: Foundations& Applications. Birkhäuser Boston, Inc., Boston, MA, 1995. 85

[29] P. Malliavin. Integration and probability. Springer-Verlag, New York, 1995. French edition: Masson, Paris,1982. 72

[30] P. Martinon, J.F. Bonnans, J. Laurent-Varin, and E. Trélat. Numerical study of optimal trajectories withsingular arcs for an Ariane 5 launcher. J. Guidance, Control, and Dynamics, 32(1):51–55, 2009. 13

[31] H. Maurer. Numerical solution of singular control problems using multiple shooting techniques. J. Optimiza-tion Theory Appl., 18(2):235–257, 1976. 70

[32] H. Maurer and W. Gillessen. Application of multiple shooting to the numerical solution of optimal controlproblems with bounded state variables. Computing, 15(2):105–126, 1975. 70

[33] J. Moreno. Optimal time control of bioreactors for the wastewater treatment. Optimal Control Applications& Methods, 20:145–164, 1999. 12

[34] D. D. Morrison, J.D. Riley, and J.F. Zancanaro. Multiple shooting method for two-point boundary valueproblems. Comm. ACM, 5:613–614, 1962. 70

[35] J.-P. Penot. Calculus without derivatives. Springer, New York, 2013. 85[36] R.T. Rockafellar. Convex Analysis. Princeton University Press, Princeton, New Jersey, 1970. 18[37] H. L. Royden. Real analysis. Macmillan Publishing Company, New York, third edition, 1988. 28[38] H. Schättler and U. Ledzewicz. Geometric optimal control. Springer, New York, 2012. 3[39] Heinz Schättler and Urszula Ledzewicz. Optimal control for mathematical models of cancer therapies, vol-

ume 42 of Interdisciplinary Applied Mathematics. Springer, New York, 2015. An application of geometricmethods. 11

[40] H. Seywald and E.M. Cliff. Goddard problem in presence of a dynamic pressure limit. Journal of Guidance,Control, and Dynamics, 16(4):776–781, 1993. 13

[41] J. Stoer and R. Bulirsch. Introduction to Numerical Analysis. Springer-Verlag, New-York, 1993. 70

92

Index

augmented metric, 57autonomous, 42

batch process, 12biextremalPontryagin, 36

boundary, 16bounded variation, 72

cell population, 11closure, 16composite cost function, 58conditionsufficient, 66

cone, 17normal, 17polar, 18positive, 17

constraintinterior point, 48

continuitymodulus, 20quasi uniform, 27uniform, 20

continuous extension, 29control, 28controlled dynamical systems, 28convergencePontryagin, 37weak, 83

costfinal, 29integral, 29reduced, 29

costate, 30costate equation, 73

derivative, 18partial, 19weak, 27

derivativesdirectional, 18

diameter, 57differentiableFréchet, 18

directional derivatives, 31

extremalPontryagin, 36

fish harvesting, 10Fuller, 10

Gâteaux differentiable, 83geodesics, 53gradient, 19

Hadamard differentiable, 83Hessian, 23

Implicit function theorem, 20inequalityHamiltonian, 36

integralStieltjes, 72

integrand, 29integration by parts, 73interior, 16invertible linear operator, 20

Lagrange multipliers, 41Lagrangian, 23reduction, 23

linearform, 16

Lokta-Volterra, 12

matrixJacobian, 19

Mayer form, 10minimumlocal, 17Pontryagin, 37weak, 31

Minkowski sum, 16multiplierabnormal, 39cost, 39normal, 39singular, 39

Nemitskii mapping, 26Newton’s method, 21normdual, 16

normalcone, 17direction, 17

normalized time, 45

orthogonal space, 17

93

pharmacodynamics, 11pharmacokinetics, 11polar cone, 18

quadratic growth, 66

reducedcost, 22problem, 22

reflexivemultifunction, 57

segment, 17separation, 18shooting, 61discrete, 65equation, 62function, 62parameter, 62

spaceBanach, 15Hilbert, 16

state, 28constraints, 72equation, 29

stationary point, 23steady-state, 11subdifferential, 83

Taylor expansion, 19theoremFréchet-Riesz, 16

time dilation, 75TPBVP, 62trajectory, 29transitivemultifunction, 57

transpose operator, 16transversality conditions, 39two point boundary value problem, 31, 62

94