École des Hautes Études Commerciales ... - École de gestion
Transcript of École des Hautes Études Commerciales ... - École de gestion
École des Hautes Études Commerciales
affiliée à
l'Université de Montréal
The Two-Level Decomposition Principle Via Dual Central Pricing
for Multi-Stage Stochastic Programming
par
Lila Rasekh
HEC Montréal
Thèse présentée à la . Faculté des études supérieures
en vue de l'obtention du grade de
en administration
Décembre 2008
C) Lila Rasekli, 2008
École des Hautes Études Commerciales
affiliée à
l'Université de Montréal
Cette thèse intitulée :
The Two-Level Decomposition Principle Via Dual Central Pricing
for Multi-Stage Stochastic Programming
Présentée par
Lila Rasekh
a été évaluée par un jury composé des personnes suivantes :
Patrick Soriano, HEC Montréal, président-rapporteur
Jacques Desrosiers, HEC Montréal, directeur de recherche
Olivier Bahr', HEC Montréal, membre du jury
Shanling Li, McGill University, membre du jury
Samir Elhedhli, University of Waterloo, examinateur externe
Résumé
La programmation stochastique multi-périodes est tout à fait adéquate pour
résoudre des applications en tenant compte d'incertitudes dans le temps.
Diverses méthodes exitent pour résoudre les problèmes de programmation
stochastiques parmi lesquelles on retrouve : les approches directes comme la
méthode du simplexe et les méthodes de points intérieurs, et les approches
par décomposition comme la méthode de coupes par centres analytiques,
les décompositions de Dantzig-Wolfe et de Benders, et la méthode de forme
en L. On peut de plus aussi utiliser dans certains cas des méthodes approxi-
matives comme la simulation de Monte Carlo et les approches combinatoires
heuristiques.
En ce qui concerne la programmation stochastique à deux périodes en di-
mension finie et avec fonction de recours, les méthodes de résolution ont bien
été étudiées théoriquement et il existe des approches efficaces pour résoudre
cette classe de problèmes. La programmation stochastique à deux périodes
peut être généralisée au cas multi-périodes lorsqu'un ensemble de décisions re-
quiert une certaine évolution au cours du temps. Comme approches efficaces
de solution des problèmes de programmation stochastique multi-périodes, on
peut utiliser les méthodes de décomposition emboîtée.
Dans cette thèse, nous étudions deux aspects de la planification de la
production en présence d'une demande aléatoire : d'une part, la gestion
de la capacité, d'autre part, la production locale versus l'externalisation.
Le premier problème se formule comme une problème de programmation
stochastique multi-périodes de grande taille de façon à prendre en compte
la demande aléaloire. Pour résoudre ce type de problèmes, nous avons
implémenté une méthode de résolution par décomposition à deux niveaux
basée sur la méthode de points intérieurs ACCPM (Analytic Center Cut-
ting Plane Method), un algorithme de plans coupants calculant des centres
analytiques. La décomposition est appliquée sur le dual de la formulation
afin d'exploiter sa structure particulière. Les valeurs centrales des variables
duales obtenues par ACCPM procurent une convergence rapide en terme
du nombre d'itérations. Les résultats de cette approche sont comparés à la
méthode de génération de colonnes à deux niveaux basée sur la décomposition
de Dantzig-Wolfe.
Le second problème considère une décision stratégique au niveau de la
planification de la capacité de production d'une entreprise. La production par
des fournisseurs étrangers est souvent associé à un risque en ce qui concerne
la qualité des produits. Nous avons modélisé cet aspect à l'aide d'une for-
mulation par programmation stochastique multi-périodes de manière à pren-
dre en compte la variation de la qualité des produits fabriqués à l'étranger.
L'objectif est de déterminer le choix optimal entre, d'une part, le contrôle
de la qualité par la production interne et la sous-traitance à des entreprises
locales, d'autre part, la variation dans la qualité des produits par l'achat à
moindre coût à des fournisseurs étrangers. Nous proposons trois méthodes
de décompositon à deux niveaux pour résoudre ce type de problèmes : la
génération de colonnes appliquée à la formulation duale en utilisant AC-
CPM, la décomposition de Benders appliquée à la formulation primale en
utilisant ACCPM, et enfin, une méthode de décomposition appliquée à la
formulation duale où le premier niveau est résolu par la décomposition de
Dantzig-Wolfe alors que le second utilise ACCPM.
Abstract
Multi-stage Stochastic Programming (MSP) has many practical applications
for solving problems whose current resolution must be made while taking into
account future uncertainty. A variety of methods exist for solving stochastic
programming problems, among them are: direct methods such as simplex
and interior point methods, and decomposition methods such as ACCPM
(Analytic Center Cutting Plane Method), Dantzig-Wolfe and Benders decom-
positions, and L-shaped method. Moreover, approximation methods such as
Monte Carlo simulation, as well as combinatorial heuristics are employed to
solve this class of problems.
In terms of the solution methods, two-stage stochastic programming with
recourse and finite support is well developed theoretically and there exist
efficient algorithms to solve it. The two-stage stochastic programming model
can be extended to the multi-stage case, when a set of decisions need to evolve
over time. As efficient solution approaches, nested decomposition methods
can be implemented to solve the multi-stage case of stochastic programming.
This thesis addresses two important production planning challenges —
namely, stochastic capacity planning; and stochastic in-house production and
outsourcing planning.
The former problem formulated as a large-scale, Multi-stage Stochastic
Program (MSP) to capture the product demand's uncertainty over time.
To solve this problem we implement an innovative two-level, interior-point
decomposition algorithm based on ACCPM. This decomposition is done on
the dual of the formulation to take advantage of the special structure of the
MSP. The central prices obtained by ACCPM provides a fast convergence
and promising computational results in terms of the number of iterations.
Finally, the resulting algorithm is compared against the two-level, classical
Dantzig-Wolfe reformulation with a dynamic column generation.
The later problem addresses an important strategic decision to plan the
production capacity in business organizations. Outsourcing to overseas sup-
pliers is often associated with some risk on the quality of the products. Hence
we developed a multi-stage stochastic programming model which takes into
account the uncertainty involved with the quality of outsourced products in
the face of stochastic demand. The goal is to find the optimal choice between
in-house capacity expansion and buying from the local suppliers with assured
quality, versus that of buying from overseas suppliers. We propose three al-
ternative algorithms for solving this problem: a two-level column generation
via the ACCPM on the dual formulation, a two-level Benders' decomposition
via ACCPM on the primal formulation, and a two-level decomposition on the
dual formulation where the fist level is solved via the classical Dantzig-Wolfe
decomposition approach while the second level is solved by using the AC-
CPM.
Table of Contents
Résumé
Abstract
Table of Contents
List of Figures
List of Tables
Abbreviations
Dedication
Acknowledgements
1 Basic Definitions and Modeling Uncertainty
1.1 Introduction
1.2 Probability Spaces and Distributions
1.2.1 Decision Variables and Stochastic Parameters . 5
1.3 Modeling of a Stochastic Program 6
V
TABLE OF CONTENTS vï
1.3.1 Expected Value Program
1.3.2 Scenario Tree Analysis
1.3.3 Chance Constrained Program
1.3.4 Recourse Programs
7
8
9
10
1.4 Feasibility Sets 13
1.5 Properties of a Recourse Function 16
1.5.1 Special Recourse Structures 18 1.6 Subdifferential Characterization 20
1.6.1 The Subdifferential of a Stochastic Program 21
2 Optimization Methods for Stochastic Programming 23
2.1 Introduction 23
2.2 Simplex-based Decomposition Algorithms 24
2.2.1 The Simplex Method 24
2.2.2 The Dantzig-Wolfe Decomposition Principle 25
2.2.3 The Lagrangian Relaxation 29
2.2.4 The Benders' Decomposition Principle 31
2.3 L-shaped methods 33
2.3.1 The Single-cut L-Shaped Method 33
2.3.2 The Multi-cut L-shaped Method 38
2.4 Sampled Based L-Shaped Methods 39
2.4.1 The Stochastic Decomposition Algorithm 39
2.4.2 The Importance Sampling 42
2.5 Multi-stage Stochastic Programming 43
2.5.1 The Nested Decomposition Method 47
TABLE OF CONTENTS vii
3 A Decomposition Based on an Interior Point Algorithm
3.1 Introduction
3.2 The Analytic Center Cutting Plane Method
49
50
3.2.1 The Oracle 51
3.2.2 The Localization Set 52
3.2.3 The Analytic Center 54
3.2.4 The ACCPM and Interior Point Algorithms 57
3.2.5 Multiple Cuts 58
3.2.6 Adding and Dropping Cuts 58
3.3 The ACCPM for Stochastic Programming with Recourse . 65
3.3.1 Stochastic Programming with Recourse 66
3.3.2 The Potential Function and the Analytic Center . 68
3.3.3 ACCPM on Two-Stage Stochastic Programming . • 70
3.4 Two-level ACCPM for Three-Stage Stocha,stic Programming 71
3.4.1 Column Generation on the Dual 71
3.4.2 Bender& decomposition of the Primal 76
3.5 Conclusion 81
4 A Two-Level Interior-Point Decomposition Algorithm for Multi-
Stage Stochastic Capacity Planning and Technology Acqui-
sition 83
4.1 Introduction 83
4.2 The Technology Acquisition and Capacity Expansion Model 88
4.3 A Two-level Decomposition Algorithm 91
4.4 ACCPM 96
4.4.1 Solving the Master Problems 96
TABLE OF CONTENTS viii
4.4.2 The Two-level ACCPM 99
4.5 Computational Experiments 101
4.6 Conclusion 104
5 Solving Multi-Stage Stochastic, In-bouse Production and Out-
sourcing Planning by Two-level Decomposition 105
5.1 Introduction 105
5.2 In-buse Production/Outsourcing Planning Model 109
5.3 Solution Approaches 116
5.3.1 A Two-level Column Generation Approach 117
5.3.2 A Two-level Benders' Decomposition 123
5.3.3 The Analytic Center Cutting Plane Method 130
5.4 Computational Results 131
5.5 Conclusion 135
6 Conclusions 137
Bibliography 141
List of Figures
2.1 Scenario Tree 45
3.1 Two-level decomposition along the scenarios for a three-stage
SP 74
4.1 Two-level decomposition along the scenarios 93
5.1 In-bouse production and outsourcing planning, decision tree . 110
ix
List of Tables
4.1 Computational Results 103
5.1 Problem Sizes in the Column Generation Approach 132
5.2 Computational Results for C:AC-AC 133
5.3 Computational Results for C:DW-AC 134
5.4 Computational Results for B:AC-AC 135
X
List of Abbreviations
ACCPM: Analytic Center Cutting Plane Method. DEP: Deterministic Equivalent Program. DW: Dantzig-Wolfe Decomposition. MSP: Multi-stage Stochastic Programming. OEM: Original Equipment Manufacturing. SD: Stochastic Decomposition. SP: Stochastic Programming. SLP: Stochastic Linear Program. VOA: Vendor Operations in Asia.
xi
To the memory of my late father
and
To my beloved mother
Acknowledgements
First of ail, I would like to thank my supervisor Dr. Jacques Desrosiers for
his guidance, his confidence in my abilities and his encouragements during
my study and research. My gratitude extend to my advisory committee, Dr.
Shanling Li and Dr. Olivier Bahn for their helpful comments on my thesis.
I would also like to thank Dr. Jean-Louis Goffin for his ideas and support in
the early years of my doctoral studies.
I would like to thank my mother and my brothers for their patience,
unconditional love and encouragements. This work was not possible without
their presence. My special thanks is toward my friends Leila and Abhijit for
their support.
Chapter 1
Basic Definitions and Modeling
Uncertainty
1.1 Introduction
Stochastic programming is mathematical (i.e., linear, integer, mixed-integer
or nonlinear) programming that involves uncertainty, whereas deterministic
optimization problems are formulated with known par ameters. In most real-
world problems data cannot be collected accurately for a variety of reasons.
The first reason is due to measurement error, and the second and a more
fundamental cause is that sonne data forecast informations about the future
(for example, pro(luct demand or prices) and cannot be known with certainty.
Stochastic programming (SP) models take advantage of the fact that
probability distributions of stochastic data are known or cari be estimatecl.
The goal is to find a solution that is feasible for all (or almost ail) of the pos-
sible data instances and that minimizes the expectation of some function of
Basic Definitions and Modeling Uncertainty 9
the decisions and the random variables. Such models are formulated, solved
analytically or numerically, and analyzed to facilitate the decision-making
process. In later chapters, this dissertation discusses the ways of taking this
uncertainty into account and, specifically, illustrates how stochastic program-
ming can be used to make better decisions.
The first chapter discusses different approaches to modeling and gives
basic definitions of SP, as well as discussing two-stage stochastic program-
ming, feasibility sets and relate(' theorems. lt is concerne(' with convexity,
continuity, differentiability, and optimality conditions for the SP models and
it is concluded with a subdifferential characterization of SP models.
Chapters 2 and 3 are devoted to solution methods for SP. We catego-
rized the solution methods to SP into two types. The first type winch is
discussed in Chapter 2 is the Sintplex-based (h:composition methods, such as
the L-shaped and stochastic decomposition methods. This chapter also dis-
eusses nested decomposition as a solution method to multi-stage SP. Chanter
3 is devoted to the decomposition principles based on the Interior point algo-
rithms, such as the ACCPM (Analytic Center Cutting Plane Method). AC-
CPM vas implemented On two-stage stochastic programming [3]. In Chapter
3 we broaden the idea and propose a new decomposition method to solve the
three-stage SP initially and extend it to multi-stage SP finally.
In Chapters 4 and 5 We address two production planning problems —
namely, stochastic capacity planning, and stochastic in-house production
and outsourcing planning. We propose variants of the state-of-art two-level
decomposition algorithm to solve these problems. Chapter 6 presents the
conclusion of this dissertation and describes sonie unsolved problems.
Basic Definitions and Modeling Uncertainty 3
1.2 Probability Spaces and Distributions
In real-world applications several parameters of a problem can be uncertain
and hence appear as a random occurrence. Future sales of a particular brand
depend on market. conditions. The replenishing and reordering point of an
inventory depends on the price changes in the market. The political and eco-
nomical situation in Asia, may affect the price of some options in the North
American market. Due to the recent strategy of globalization adopted by
many corporations, nowadays modeling uncertainty is an indispensable part
of decision mak-ing on the operational management level. Outsourcing raw
materials, labor and finished goods from so-called low cost countries to the
developed part of the world introduces mai' parameters of uncertainty to the
decision-making process. In contrast, stochastic programming provides es-
sential modeling and solution techniques to reach the optimal solution under
uncertain conditions.
Since stochastic progranuning deals with random data, we need to in-
trochice sonne notation. Letter is used to denote a random variable, w
represents a possible realization of out of the set of ah l possible outconnes
Q. The outcomes may be combined into a subset of Q called random events.
A is a collection of random events (set of subsets of Q). To each event A e A is associated a probability P(A); the triplet (Q, A, P) is called a probability
space.
A random variable cari be discrete or continuous. It is a dis crete van dom
variable if it eau assume onlya finite or countably infinite number of val-
ues. On the other land, it is a continuons vandom variable if it cari assume
any values in an interval of real numbers. A list of possible values of a dis-
Basic Defin,itions and Modeling Uncertainty 4
erete random variable, together with their assoeiated probabilities give us
a discrete probability distribution. The probability of a continuous random
variable is flot defined at specific values. Instead, it is defined over an inter-
val of values and is represented by the area !limier a eurve. For a particular
random variable e, the cumulative distribution is defined as
Fe(x) = x),
or more precisely
Fe (x) = P < .
Continuous random variables cari be deseribed through their density function
f() iffhere
P(a < b) = f (6,- )ck. (1.1)
The expeetation of a random variable is computed as
E[e] = > f () for discrete random variable ; kER
or equivalently
E [ ] f+oc œ dk-'() for continuons random variable.
The expectation of er is ealled the rth moment of e and denoted by =
Basic Dejïnitions and Modeling Uncertainty
1.2.1 Decision Variables and Stochastic Parameters
Stochastic programming deals with mathematical programming models where
some parameters are "incertain. For stochastic programming, some variables
are to be set by a decision-maker (these are called the decision variables),
while soine model parameters are the random variables. There are a variety
of situations one might consider in tins context. One differentiation is based
on when the deeision maker must make decisions relative to the time when
the random variables are realized. There are several possibilities winch are
described as follows:
No Recourse Model: The decision maker must choose values for the
deeision variables before any of the random variables are realized.
Single Stage Recourse: Some of the decision variables must be set
before the random variables are realized, while others may wait until they
are realized.
Multistage Recourse: Decisions and random variables are determined
in stages. The first set of decision variables must be fixed before ail realiza-
tions. Then the first set of random variables is realized and the second set of
decision variables must be fixed. The process continues through a series of
decisions and realizations. Typically, the stages represent intervals of time.
Wait and See: The decision maker makes no decisions until ail random
variables are realized. In the wait and see approach, the optimal decisions cari
be made with no uncertainty using deterministic optimization algorithms.
Among the above possibilities, the no recourse case is the most difficult
because all the decisions must be made before the model is completely known.
whereas decision making under the wait and sec paradigm is tire easiest.
Basic Definitions and Modeling Uncertainty 6
1.3 Modeling of a Stochastic Program
This section discusses the Stochastic Linear Program (SLP) and its Doter-
ministic Equivalent Program (DEP). DEP is a large scale linear programming
problem with ail scenarios and constraints, winch is used in many real-life
applications.
In general, stochastic linear programming consists of
min z = ex
s.t. As = b, (1.2)
Tk x > hk , k = 1, . K,
> O.
where the deterministic constraints are represented by As = b (a; is a decision
variable) and the stochastic constraints are formed by
Tics > hk , k = 1, K.
The real value of (Tk , hk), k = 1,. K is not known, but the probability dis-
tribution of (T, h) is assume(' to be defined. Hence, the existing uncertainty
is expressed by probability distributions. i.e.,
P { (T, = (Tk. hk)} = pk, k= 1, . . K.
In most cases a stochastic programming model decides on s here-and-now
without knowing the value of (T, h) but knowing its probability distribution.
Basic Definitions and Modeling Uncertainty 7
The stochastic constraints, Ti > h (we drop index k for simplicity), have
to be replaced by sonie sort of deterrninistic equivalent constraints to be
able to solve the SP model. These stochastic constraints are interpreted
as goal construiras. There are a variety of approaches that one might take
for replacing goal constraints by their deterministic equivalent, among them
expected value, scenario tme, chance construiras and recourse problems are
explained next.
1.3.1 Expected Value Program
If T and h follow discrete distributions, TX > h can lie replaced by Ta; > h where
T = E pkTk and h =)khk
where T and h are expected values of T and h respectively. Replacing the
probabilistic ternis by their expected values 'flakes the model deterministic.
This approach models the no-recourse situation.
Expected value solution serves the solution to this problem. However,
the resulting solution may be 'instable in the sense that it might perform
poorly under realization of some random variables while performing well
under others. In this case, the optimality may be sacrificed to obtain a
robust solution, which is suboptimal in terms of expected value, but lias a
lower risk. This approach is simple and easy to compute.
Basic Definitions and Modeling Uncertainty 8
1.3.2 Scenario Tree Analysis
In many real-life applications, the distribution of a random variable is not
known, or it is too expensive to consider a discrete distribution with many
possible outcomes, or to handle a continuons distribution with numerical
integration. in such cases a question arises whether one cari select a small
munber of realizations called scenarios to represent the random events. in
strategic planning, scenarios obtained through expert judgments and some
of the realizations of random events are considered in detail; in other cases,
scenarios may be a quintile of a kno-wn distribution or historical data.
One way to deal with such difficulties concretely is the scenario tree anal-
ysis. In this approa.ch each scenario is given a probability. The scenario
informations and their probabilities are then organized in a tree structure.
Scenario trees provide fiexibility to choose scenarios which need to be con-
sidered and their relative importance. But it is impractical to consider too
many scenarios, and this limas the level of detail of the model. It is common
to use scenario trees in financial planning models. Sirice it is (-casier to predict
trends in the near future, the scenario tree in such models lias more branches
at each node of the earlier stages and fewer towards the end of the planning
period.
A simple solution method to scenario analysis woukl be solving the SLP
in (1.2) for each scenario separately. The proposed optimal solution to SLP
would then be the average or the sweighted average of solutions of ah l sce-
narios.
The scenario tree imposes limitations to modeling by not including ail
possible scenarios in the model. However, it provides a flexible approach to
Basic Definitions and Modehng Uncertainty 9
modeling problems with stochastic parameters, especially t'ose in winch the
random data forecast some events in the future (for example, the price of an
option in the market), in which the distribution of random parameters is flot
known in advance.
1.3.3 Chance Constrained Program
Chance construira programming is mathematical programming in which some
constraints are stocha.stic. In model (1.2) chance constraint replaces Tx > h
by
P{Tx > > a, (1.3)
where a E [0,1]. This can be interpreted, for example, as requiring the
power supply system to be able to cope with customer demand at least 9,5%
of the time. This formulation is mostly used when it is impossible or too
expensive to guarantee that the conditions eau be held at ail times and the
cost of failing those conditions is dillicult to quantify. In the power supply
example, equipments may fail but it is important to keep the failure at a
low level to avoid the high cost of power shut clown. A chance constrained
program is often converted to a deterministic equivalent to be solvable. The
model is taking care of risk, (i.e.. 1 — a is the maximum acceptable risk)
but, it may introduce non-convexity into the problem. If the probability
distribution of the random parameter satisfies some convexity requirements,
then the feasible region is convex and closed, and techniques of mathematical
programming cari be used to solve the problems. In chance constrained
programming [13], the objective is an expectational function (the E-model)
Basic Definitions and Modeling Uneertainty 10
or it cari be the variance of some result (the V-model), or the probability of
some occurrence (the P-mode!). Another type would be the objective that
is a: quintile of a distribution function [44]. For a summary of properties and
solution methods to chance constrained programs, sec for example [10].
1.3.4 Recourse Programs
Stochastic programming with recourse is a general purpose technique that
can deal with uncertainty in any of the model parameters. The simplest
model of recourse problems have two stages:
In the first, stage, the decision maker makes a decision. In the second
stage, the decision maker observes a realization of the stochastic elements of
the problem but is allowed to make further decisions to avoid the constraints
of the problem becoming infeasible.
This model introduces corrective (recourse) actions. It replaces
> h by TX+Wy = h,
where y is a decision vector on the second-stage problem. The choice of y is
dependant on the realization of (T, h) in the first-stage. The model associates
the cost q to be added to the objective function for acquiring information
Basic Definitions and Modeling Uncertainty 11
and taking a recourse action. Therefore the recourse model is
miii CT X qkPkYk
S. t. Ax = b.
'Tk x + .147yk = hk,
Yk > 0, x > O.
k = 1, . (1.4)
This model takes care of the risk effectively but the problem may get large
in ternis of the constraints and variables very easily.
As it is mentioned above, there is a distinction between the first stage and
the second stage. The first stage is represented by vector x and corresponding
first-stage vectors and matrix A .b, c of sizes in x n, ru x 1 and n x 1. In the
second stage, a number of random events w E may be realized. For a
given realization of w, the second-stage problem data q (w) ,T (w), and h (w)
become known where q (w) is k x 1, h (w) is I x 1, and T (w) is I x n, W is called
the recourse matrix of size I x k. In most cases we consider W to be fixed.
It allows us to define the feasibility region winch facilitates computations.
Each component of q(w),T(w) and h(w) is a possible random variable. Let
Tj (w) be ith row of T (w). Putting together the stochastic components of
the second-stage, one obtains the random vector
(w) = (qT (w ) 112- (W) 211 (W) ; • • • Trn2 (W))
Thus the cla,ssical two-stage stochastic programming with fixed recourse
Basie Definitions and Modeling Uneertainty 12
can be written as
min cTx + E [qT (w) y (w)]
s.t. Ax = h, (1.5)
T x 11Ty (w) -= h (w)
y (w) >0, x > 0.
Let
Q (x (w)) = min {qT (À)) y Wy = h (w) — T (w) x y > 01 (1.6)
be the second-stage value fUnction, and define the expected second-stage
value function as
(r) (1.7)
Then the deterministic equivalent program, (DEI') to (1.5) would be
min cTx + Q (x)
s.t. A.E = b, (1.8)
> 0.
On the other hand the recourse function to a stochastic programming prob-
lem with fixed recourse (W is deterministic) and fixed T, is represented by
(w)) = min {qT (w) y Wy = h (w) — x, y >
W (x) = (x ,(w))1, (1.9)
Basic Definitions and Modeliny Uncertainty 13
where the deterministic equivalent program (DEP) with the sanie assump-
tions would be defined as
min c7'3: + q (X)
s.t. Ax = h,
Tx —X = 0,
> 0.
(1.10)
1.4 Feasibility Sets
The recourse structure can be fixed or deterministic. For most problems we
try to restrict ourselves to the fixed recourse. Since studying the fea:sibility of
the second-stage solution depends on whether W is fixed, here, we consider
the general case, where W is a random matrix, by that the recourse function
would be
Q (w)) = min { (w) y I W (w) y = h (w) — T (w) ,x , y >
Q (x) = E [Q (x,(c..,;))1. (1.11)
Let us define the feasibility set for the deterministic constraints as K IL =
{x Ax = h, x > 0} and K2 = {x Q(X) < Do} be the second- stage feasibil-
ity set. We may then define DEP as
min cTx Q (x)
E K1 n K2. (1.12)
Basic Definitions and Modeling Uncertainty 14
Definition 1.1 C R.n is the support of 4", where it is the smallest close
subset in R," such that P E = 1.
The definition of K9 is flot very helpful since we often would like to know
in advance if the second stage solution is limite for a particular z. Therefore,
we define elementary feasibility sets as an alternative &finition
K2 () 7= {x I (2(3 . < +00 . (1.13)
The possibility inteipretation of second-stage feasibility sets also defines as
follows
{x V e Q. 3 y > 0, s.t. Wy = h — Tx} = neest 11.-2((„q
The decision vector x belongs to 4 if the a feasible second-stage solution y
can be taken for ail values of The following theorems clarify more details
about the feasibility and convexity of the feasibility set in the second-stage.
Vve may skip the proofs and refer the reader to [10] for more details.
Theorem 1.1: I. For eaeh the elementary feasibility set is a closed
conyez polyhedron, hence the set .1.q) is closed and convez. II. When Q is
finite, I then is also polyhednd and coin cides with K2.
Proposition 1.2: If lias finite second moments, then
P C2(x,) < oo) = 1 implies Q(z) < o
Basic Definitions and Modeling Uncertainty 15
Theorem 1.3: For a stochastic program with fixed recourse where has
finite second moments, sets K2 and Kr coincide.
Theorem 1.4: When W is fixed and has fuite second moments:
((2) K2 is closed and convex.
(b) If T is fixed, K2 is polyhedral.
(c) Let QI- be the s'apport of the distribution of T. If h() and T() are in-
dependent and Ç-1),7-, is polyhedral, then K2 iS .polyhedral.
Theorem 1.5: For a stochastic program with fixed recourse, Q (x,) is:
(a) a piecewise linear convex function in (h. T);
(b) a piecewise linear concave function in g;
(c) a piecewise lin car convex function in x for ail x in K = K1 n K2.
Definition 1.2: 4"has absolutely continuous distribution F() if there
exists a density function such that
Fe (l') = fe(t)dt.
Definition 1.3: A function g :R. ---> R. satisfies a Lipschitz condition
with d is a metric and a constant C if
d(g(ti ), g(t2 )) < C d(ti .t2 ).
A function that satisfies the Lipchitz condition for fuite C is called Lipchitz
continuons, which is a stronger condition than regular continuity.
Basic Definitions and Modeliny Uncertainty 16
Theorem 1.6: For a stochastic program 'with fixed recourse where lias
fuite second moments:
(a) Q(3.:) is a Lipschitzian convex lanction and is fuite on K2.
(b) When is finite, Q(x) is piecewise linear.
(c) If F(C) is an absolutely continuons distribution. Q(x) is differentiable on
K2.
From the above theorems, we learned that if e cornes from a fuite dis-
tribution, then 14 is polyhedral and (2(x) is piecewise linear and convex in
K. Or if e cornes from continuons distribution with fuite second moments
then (2(x) is differentiable and convex.
1.5 Properties of a Recourse Function
Recourse models are the most commonly used class of models in stochastic
programming. In many SP models, it is not possible to require that ail
constraints be satisfied for realization of ail random variables. Therefore, the
decision maker may either require the satisfaction of some constraints with
given probability, or the incorporation of a corrective action (recourse action)
into the model, when a constraint is violated. Recourse models incorporate
the corrective action to overcome the stochastic infeasibility of the decision
variables. However, in many recourse modeling situations we need to cho ose
°n'y a few scenarios to be built into the model. These choices are based on
the penalty cost winch one may be willing to pay for taking the recourse
action for a particular constraint.
The deterministic equivalent program (DEP) to the recourse model is
Basic Definitions and Modeling Uncertainty 17
represented by
min fc + EK2(x,41 = min {c + (2(x) Ax = xe 'A x=b xER4 1:Ç
where Q(x..(w)) in (1.6) is a. minimization problem involving, say In goal
constraints in the form of W y = h(w) — T(w)x. In cases where it is too
expensive or simply impossible to include all goal constraints in the model, we
have to be selective about the goal constraints winch are to be incorporated
in the objective function. To make this choice, the following recourse model
which gives a clear interpretation of goal constraints is often used
min {cTx + E[v(h(w) — T (44)1 zEnT4:Ax=1)
= n'Un {cTx + E[v(z)] I T(w)x + z = 14» , (1.14) xETtlft
where z is n vector of deviation froin the goals, with components z = h i —
ta;, and v(z) is a penalty cost lanction for the deviation of z. 1f the linear
programming duality holds, v(z) eau also lie interpreted as
v (z) = min {qty I Wy = = max{z t I W`t, < (1.15) yETZT4.' tETC
The recourse function y gives the minimum recourse cost to compensate for
deviation z in the goal constraints. In a simple application of the two-stage
recourse problem this minimum cost to take a corrective action helps to
decide on the combination of scenarios winch are to be built into the model.
Basic Definitions and Modeling Uncertainty 18
1.5.1 Special Recourse Structures
For sorne value of z the recourse penalty function v, given in (1.15), ma
get the value v(z) = +oc where the DEP is infeasible; or it is possible that
for some z, the value v(z) = —oo, then DEP is unbounded. In the former
situation there may exist deviations h(w) — T(w)x that cannot be corrected
in the second stage. In the latter case however, there might be a deviation
winch is extremely advantageous where the penalization for the deviation
is not possible. To avoid such situations the following designs of recourse
structure are made.
Definition 1.4: A stochastic program lias relatively complete reeourse
if Ki, C K2 i.e., every solution x that satisfies the first-period constraints,
Ax = b. has a feasible completion in the second stage. Equivalently it means
that v(h(w) — T(w)x) < oo for ail w e Q and X C C X I AX =- . Definition 1.5: A stochastic prognun lias complete recourse if there ex-
ists y > 0 such that Wy = z (z = h — Ta:) for ail z E R.m. This implies that
for ail x ,T (w), h(w), we have Q(x, w) < oo. Complete recourse is a property
of W. 1f the columns of W spans 7"?,' then Vz e R_ : Wy = z, and we
have complete recourse.
There is a special case where the penalty cost q is separable in torils of
surpluses and shortages. For example, this can be interpreted as a production
planning, where the demand is a non-deterministic parameter. Modeling the
problem as a two-stage stochastic programming with recourse ; in the first-
Basic Definitions and Modeling Uncertainty 19
stage, the mode decides on the quantity of the production. However, in the
second-stage the recourse action is taken, i.e., if there is a shortage of supply,
it is needed to satisfy the demand from other sources at the penalty cost of
whereas the surplus of supply eau be inventoried at the cost of a - .
Suppose that the penalty costs q+, a- , corresponding to second-stage
variables y+ and y- , as defined above are given, the general SP model can
be reformulated as
min cT r ± Emin {q+ (w) y+ + g- (w) y- (4]
s.t. = h. (1.16)
T (w)z + W (y+((.4 + ?[(w)) = h (4 ,
Y± (w) O. Y- (w) > O, z > O.
Definition 1.6: A stochastic program lias simple recourse if
1 0 • 0 -1 0 • 0 \
0 1 • • 0 0 -1 0
W = [I, -.I"] (1.17)
0 0 1 0 0 -1
and P > = 1. Here we construct W so that it spans
For a simple recourse problem with W = [I, - I], y and q are divided as
Basic Definitions and Modeling Uncertainty 20
(y+, y- ) , (g+ q- ), and the second-stage model changes to
min E [g+ (w) y+ (w) + q (w) y- (w)]
y+ — y- = (w) — (w) = 1, rn,
y>0y>O,e> O.
Theorem 1.7: Suppose the two-stage stochastic program is feasible and
has simple recourse and that (e, has finite second moments. Then (2(x) is
finite if and only if qi+ (.17 > 0 with probability one.
Above definitions and results can be used to obtain a specific optimality
condition for SP, sec for example, [22, 74. 75].
1.6 Subdifferential Characterizat ion
Marly solution methods to stochastic programming problems are dependant
on the description of subdifferentials of the model. In problems with com-
plete recourse ancl fuite support, Q(x) is convex and piecewise linear. These
properties help to build the subdifferential of Q. The characterization of this
subditferential is the topic of this section.
Definition 1.6 A vector g E is a subgradient of f : 1Zn at
E domf if for ail y E dom f, f (y) > f(x) + gT —
If f is convex and differentiable, then its gradient at z is a subgradient.
But the subgradient exists even if f is net differentiable at z. There can also
Basic Definitions and Modeling Uncertainty 21
be more than one subgradient of the function f at point x. In other words a
vector g is a subgra.dient of f at z if the affine function f (x) + (y — is
a global underestimator of f. A function f is called subdifferentiable at z if
there exist at lea.st one subgradient at z. The set of subgradients of f at point
is called the subdifferential of f at z and is denoted by f (i;). A function
f is called subdifferentiable if it is subdifferentiable for ail z E dom f
Subgradient of a differentiable function
1ff is differentiable and convex at z , then ôf(z) = {77 f (x)}, i.e., its gradient
is its only subgradient. Conversely, if f is convex and Of(x) = {g}, then f
is differentiable at z and g = V f (x).
The minimum of a nondifferentiable function
A point z* is a minimizer of a convex function f if and only if it is subdif-
ferentiable at z* and 0 E 0 f (x*), i.e., g = 0 is a subgraffient of f at z*. This
condition 0 e 0 f (x*) reduces to (x*) = 0 if f is differentiable at z.
1.6.1 The Subdifferential of a Stochastic Program
Stochastic programming is a class of nondifferentiable optimization problems.
In the case of complete recourse, (2(x) is a finite convex function. Moreover,
it is â piecewise linear function if the distribution of w is discrete with finite
support. To obtain the subgradient of (2(x) let us redefine it as Q (z) =
E[v (h(w) — T(w)x)]. Consider z to be muai to h(w) — T(w)x, then v(z) cari
be interpreted in terms of the extreme points of the dual feasible region. Let
Basic Definitions and Modeling Uncertainty 99
A = {Al; A2 A1 1 1} be a set of extreme points of ft e R.tm I tliTt <
Since cadi of those extreme points Ak. k = 1, . .14 is a potential optimal
solution to (1.15), v(z) cari be represented as
v(z) = inax {zt Ak} z E 7Z7n. k=1,...,Ini (1.18)
Therefore, v is a pointwise maximum of a finite set of linear functions of z.
Furthermore suppose Li hm a discrete distribution, say P{w = LiS} = ps ,
s =1, . , S. then the convex function
Q(x) psv(h.(ws) — sES
is piecewise linear in :c. In cadi piece one lias Q (x) = EsEs ps Ask (h(ws) — T(ws)x).
h follows that
OQ (x) = P.sAZ ( — T(ws )) (1.19) scs
Iii general, one can derive a formula for the subdifferential of Q at x by
using a. chain nue:
0Q(z) =E[ax — T(w)x)] = A*T(4], (1.20)
where À* is any optimal solution to max,,, E7z.{A(h(w) — T(w)x) AH/ < (IL
Subgradient inequalities and subdifferentials are used in many decompo-
sition techniques. These techniques are discussed in the following Chapters
2 and 3.
Chapter 2
Optimization Methods for
Stochastic Programming
2.1 Introduction
Marly stochastic prograniming problems have very large number of scenarios
and special structures which are tractable by decomposition. Decomposition
methods are ways of solving a sequence of optimization problems whose sce-
nario sets var y increasingly in size based on the information feedback given
by the algorithm. Hence, the decomposition algorithms recluce the size of
the problem to make it solvable. The classical decomposition methods for
mathematical programming trace back to Dantzig-Wolfe and Benders. These
decomposition principles are translated into L-shaped method in stochastic
linear programming [72]. L-shaped method lias been extended to the nested
decomposition method and developed as a computer code [29] for linear nml-
tistage recourse problems.
Optimization Methods for Stochastic Programming 94
The L-shaped method is viewed as a simplex-based &composition prin-
ciple in contrast to the method whieh is based on interior point algorithm
such as the Analytie Center Cutting Plane Method (ACCP11 ,1), sec [34, 33].
ACCPM lias been implemented as a &composition method for two-stage
stochastic programming [3]. The goal here is to apply tins method to multi-
stage stochastic programming as a nested decomposition algorithm.
2.2 Simplex-based Decomposition Algorithms
2.2.1 The Simplex Method
The simplex method considers the standard LP problem
min CTX
S.t. AX = h, x > 0,
where e E R.n , A E lZmx" and b G TC . It uses the filet that n — rn variables
must be zero at a vertex solution to partition the variables into basic, x > 0,
and non-basic variables x i = O. Similarly, A and e are partitioned as: A =
[BI AT], e = (eB, eN ). The basic variables and objective ftinction in ternis of
xN are expressed as
xB = NxN ), T CTX = CBXB CNXN = CT B —l b (CN — NT B -1 cB)xN
Optimization Methods for Stochastic Programming 25
If any component i of c/N = CN - NTB-l cB (the reduced cost vector) is
negative, then the objective function value cari be decreased by letting x ,
become positive. i.e., basic. The change in x 8 then would be B-1 (b- Nixivi )• Variable xN, is increased until one of the basic variables becomes zero and
non-basic. The main computational cost of the simplex rnethod is calculating
B-1 cB and B-1 Ni . This can be clone by using QR or LU factorizations of B
and updating factors at each iteration. The simplex method moves between
adjacent vertices untill no improvement can be made. If the simplex steps
are non-degenerate, then the algorithm terminates after a finite number of
iterations.
2.2.2 The Dantzig-Wolfe Decomposition Principle
The revolutionary paper by Dantzig-Wolfe in 1960 [19] introduced a new
venue in solving a large-scale linear programming problems. In practice the
Dantzig-Wolfe decomposition method is most effective when the coefficient
matrix of a linear program have a block angular structure. The independent
blocks of such matrices are linked by coupling or complicating constraints.
This can be interpreted, for example, as requiring the production of several
items in a single machine with limited capacity (capacitated lot-sizing prob-
lem [21]) and a finite horizon. The objective function of such a problem
minimizes the total cost, consisting of the set up cost, the variable produc-
tion cost, and the inventory holding cost. Moreover, the constraints repre-
sent the inventory balancing constraints in each period, the set up forcing
constraints, and the capacity constraints. In the absence of the capacity
constraints, this particular problem decomposes into an uncapacitated lot-
=1, Ai > 0, = 1, ... k, i= 1
Optimization .illethods for Stochastic Programming 96
sizing problem for each single item. The independent block of the coefficient
matrix emerges from the set up and inventory balancing constra.ints whereas
the capacity constraints are regarde(' as the coupling or complicating con-
straints. In Dantzig-Wolfe decomposition the capacity constraints are kept
in the master problem. The linear program is then solved for each item winch
is regarded as a subproblem. Therefore the general idea behind this decom-
position principle is to keep the linking structure in the master program level
and to address the subproblems independently in order to take advantage of
their -special structure.
hi the next paragraphs, we study the Dantzig-Wolfe reformulation along
with the column generation technique winch leads to the decomposition al-
uorithm.
Consider the linear program
min z = cTx
s.t. (2.1)
To simplify the discussion, assume the feasible set X = {xl D x > d, x >
is a bounded convex polyhedron. Therefore, any E X eau be represented
as a convex combination of the extreme points of this polyhedron [55], i.e.,
where xi , j = 1 k, are the extreme points of X. Note that the original
Optimization Methods for Stoehastic Programming 97
linear programming problem (2.1) eau be represented by d'ose elements of
X which satisfy the set of inequalities Ax > b and minimize z. Therefore by
substituting x = Àx into the constraint set Ax > b, problem (2.1) eau
be re-expressed in terms of À as
min z = c Aixi) À i=1
s.t. Aixi > b, i=1
E = LÀ >O. i=1
Defining Axi = pi CT x = A the original problem is reformulated as
min fi Ai
s.t. (2.2)
i=1
The previous program is called the master problem winch lias fewer con-
straints but a larger number of variables than the original problem. The
constraint = 1 forms a convexity row for the n'aster program.
When the simplex method is applied to this master problem instead of
considering ail the columns we use the column generation technique, gener-
ating colunms to enter the basis as needed. To observe how this is done,
let (7r, 7r0) be the set of dual multipliers associated with the constraints
Optimization Methods for Stochastic Programming 28
Eik_i pi > A i = 1 ,respectively in (2.2). In order to choose the vari-
able As to enter the basis, the simplex criterion asks us to find the minimum
negative reduced cost, i.e., — ps — } winch by the &finition of
p„ and f, muais to the mins {(cT — 7r 9 A)x8 — 701.
Sirice xi is an extreme point of X and the optimal solution of a linear
program whose feasible set is bounded takes place at the extreme points of the
bounded polyhedron, then finding the minimum reduced cost is equivalent
to solving the subproblem
min ((T — z A)3; — (2.3)
s.t. D x > (i.x > O.
Hence by solving (2.3) and obtaining the solution xs , one can find the
column ps = (Axs , 1) T to enter the basis of the master program. The cost
coefficient for this variable then would be = cTx s .
As mentioned above, tins method is most efficient when the coefficient
matrix of the original problem lias a block angular structure. Suppose that
the coefficient matrix and the right hand sicle of the constraint equations in
(2.1) are partitioned into
A2 • Ani . b=[b] .
and
D= D2
D771
b=
(11
(12
(Lm
Optimization Methods for Stochastic Programming 29
where D has a block angular structure. Then the subproblem (2.3) becomes
ni
min (CiT - 7F Ai):Ci - 7F0
i=1
s.t. Dx > d , i =- 1 in (2.4)
Since the objective function and constraints in (2.4) are separable in ternis of
1„ in, the resulting subproblem is split into ni independent smaller
problems:
min (ciT — 7F Ai):Ci - 7r0
S.t . Dx >
xi > O.
As a result of this disag regation the master program now contains ni
convexity rows. In this case there exist varions strategies as how to add
columns to the master problem. One simple possibility is to solve ail the
subproblems and to add a column to the n'aster problem for cadi subproblem
with negative reduced cost.
2.2.3 The Lagrangian Relaxation
As ‘ve have seen in the capacitated lot sizing problem in the last section,
many optimization problems can be modeled as an casier program in the
absence of its complicating constraints. One way to deal with this type of
constraints would be to replace them in the objective function with a penalty
Optimization Methods for Stochastic Progiumming 30
terni. Lagrangian relaxation incorporates the violation of the complicating
constraints in the objective function and penalizes it by the weight of its dual
multiplier [54]. The resulting lagrangian problem provides a lower bound on
the objective value (for a minimization problem). The lagra.ngian relaxation
is closely related to the Dantzig-Wolfe decomposition. It eau also be donc in
the dual for complicating variables which is the topic of the next section and
was first developed by Benders.
To formulate the lagrangian problem consider the following LP
min cTx
s. t. > b, (2.5)
Dx > d,
x > 0.
Suppose that the problem is complicated the constraints Ax > b. To
transform this problem to the relatively casier one, we define a set of multi-
pliers u > 0 and add the negative tenu, —u(Ax—b), to the objective function
min cTx — u(Ax — b) (2.6)
D x > d, X > 0, u > 0.
By choosing u as a non-negative variable it is clear that the optimal value
of the lagrangian problem is a lower bound on LP. However, at optimality
the optimal value of the lagrangian problem is equal to that of LP. This
happens due to the fact that at the pair of (e, u*), primal-chiai optimal
values, u* (Az* — 0. This is known as the complementary slackness
Optimization Methods for Stochastic Programming 31
conditions.
The lagrangian method iteratively solves a sequence of relaxed programs
as it appeared in (2.6). In each iteration, the vector of multipliers u is
computed, sec for example [27] for details on computing u. The most popular
method however to compute 'IL is based on subgradient methods [27], but it
can be shown that lagTa.ngian relaxation is in fact totally equivalent to the
Dantzig-Wolfe decomposition (again sec [54]).
2.2.4 The Benders' Decomposition Principle
The Benders' Decomposition principle is based on a variable partitioning pro-
cedure for large-scale mathernatical programming problems [4]. The method
partitions the variables by fixing the so-called complicating variables. This
reduces the given problem to a more tractable hence casier problem to solve.
The later problem is parameterized in ternis of the complicating variables
vector. The dual to this problem forms the subproblem. The subproblem
passes forward the current dual solution (for a fixed vector of complicating
variables) to a n'aster program in the form of a Benders' eut utilizing the
linear programming duality to reflue the value of the complicating variables
vector in cadi iteration.
To illustrate the Benders' decomposition method consider the following
linear programming problem:
min cT x+fT y
s.t. Ax + By > h, (2.7)
y E Y,x > O.
Optimization Methods for Stoehastic Programming 32
If y = y is fixed the resulting LP model to solve becomes
min (Ta. + fT
S.t. Ax > b — By,x > o.
Therefore, the original problem is equivalent to
min[f Ty + min{cT xlAx > b — BO]. yEY x>0
The dual to (2.8) is
max. ( 1) — BOL +
s.t. Anu, < c, iL > O.
Utilizing the linear programming duality, this program provides a lower
bound to the original problem, i.e., for cadi solution u i*, cTx > (b — R)ui* +
ry. Note that the dual feasibility is independent of the value g ; hence the
inequality crx > — By)n7 + fT y is valid for ail y. liherefore, each time a
new y is considered, the valid eut can be added to the master program where
alter I iterations the master problem becomes
min z
s.t. > (b — By)u: + fT y, i =
C
In each iteration i, the master program solves for y:, next the optimal dual
value Ît; is needed to be obtained from (2.9), assuming the dual is bounded.
(2.8)
(2.9)
Optimization Methods for Stochastic Programming 33
(In cases of unbounded dual, another category of Benders cuts can be added,
sec for example [55].) Finally the corresponding Benders' eut is added to the
n'aster program. This procedure stops when the distance between the upper
bound and the lower bound zi (i.e. jz — I) fails below a predetermined
threshold.
By solving the Benders' master program and the subproblem ; it is possi-
ble to generate a sequence of lower and upper bounds on the original problem.
Since the master problem is a relaxation of the original program (we assume
a minimization problem here), it generates a lower bound, whereas the sub-
problem discussed earlier provides the upper bound on the original progTam.
Benders' decomposition technique is translated to L-shaped method in linear
stochastic programming (SLP).
2.3 L-shaped methods
2.3.1 The Single-cut L-Shaped Method
The basic idea behind the L-shaped method is to approximate the recourse
function C2(x) in the objective of (1.8). The recourse function Q(x) is the
expected value function in the two-stage model. It involves a solution to ail
second stage recourse linear programs and its approximate value is repre-
sented by the single variable O. The optimal value of 0 is then determined
iteratively by using the LP duality to construct a piecewise linear approxima-
tion to Q(x). The Benders' cuts which are generated to obtain the optimal
value are called the feasibility and optimality cuts.
To make this approach possible, consider a two-stage SP model with finite
Optimization Methods for Stochastic Prognenming 34
support. The two-stage recourse formulation in (1.5)-(1.6) eau Le reformu-
lated as • min cTx + .T + PetT
Y2 + • • + Psq Ys
s. t. Ax =h,
+1-4/Yi
T2x + W y2 =h2, (2.10)
nx + Wy, =hs ,
X E X, Yi, • • Ys e
where y, = y (w,), is the recourse action to take if scenario w, °ceins. The
above matrix of the linear program bas a dual-block-angular structure which
bas led to the naine L-shaped method. The method is basically a Dantzig-
Wolfe decomposition [191 of the dual formulation or a Benders' &composition
[4] of the primai formulation.
Algorithm 2.1: The Single-cut L-Shaped Method
Step 0 Set r k, y =-- 0
Step 1 Set V = y + 1 and for (xi', solve the following linear program:
min eTx + 0
s.t Ax = h,
Do; > dt , = 1 r .
0 > + Etx, = 1, 2,
> 0, 0 E R,.
(2.11)
Optimization Methods for Stochastie Programming 35
Step 2 For s = 1 S, solve the second-stage feasibility problem:
min w = eTv + + .
s.t.
,v+, 1) > O.
Wy v+ — = lis
s
— Ts xv, (2.12)
If w > 0, let u be a dual multiplier and set r = r 1. Set Dr = o-TTs , dr = uT hs and return to step 1;
else w = 0 for ail s = 1, • • S, go to step 3.
Step 3 For s = 1. • • , S solve the second-stage problem:
Qs (xv) = miii
s.t Wy = h — Tsx L . (2.13)
y
Let the dual multiplier be
Set k = k + 1. Ek = — Es s T p Es p h,. s=1 s, - ek — s=1 s s If ely > ek + Eke then xv is the optimal solution, STOP;
else return to step 1.
Optimality Cuts: From sections (1.5) and (1.6) we know that for any
xv E 7Z11 the dual to the second-stage function is
ns (x") = max{AT (lis — Ts)v) I A W < A
Optimization Methods for Stochastic Programming 36
and the subgradient of Q3 (xv) is
V 5 = E 0Q, (e),
where is an optimal dual solution to the recourse problem. Since Q s is
convex, the subgradient inequality writes as
(23(x) (2(xv) 3 —
it is known by duality that
A .T9 - v ) = ( v
Combining the last two equations gives
(23( ) > h, — ) sTs Tsx.
Taking the expectation for ail realizations of random data gives
(23(x) ps AsT — s=1 s=1 s=1
where the first terni in the RHS represents e and the second term represents
Ex in Algorithm 2.1.
In other words. Q(x") uT (x — is a supporting hyperplane of Q at
:y'. This insight helps to build an increasingly better approximation of Q(x).
Note that the convex function Q(x) is equal to the pointwise maximum of ail
Opbmization ilfithods for Stochastic Programming 37
its finitely many supporting hyperplanes, and it is approximated by variable
O. which is bounded below by one of these supporting hyperplanes (optimal-
ity cuts). The idea here is to add the optimality cuts at each iteration to
construct a good description of Q at the optimality.
Feasibility Cuts: At a first stage solution xv, the feasibility set for the
second-stage problem is {y litry = h, — Tszy y > 0}. This shows that for
any feasible first stage solution x, there must be at least one y > 0 such
that Wy = h ., — Tsx. 1f this x is infeasible, the L-shaped method iteratively
generates separating hyperplanes, called feasibility cuts which only eut off
3: that leads to second-stage infeasibility. These hyperplanes are written in
terni of u (the dual multiplier of model (2.12)) and satisfy 0.T(/18 — Tsxv) > 0
and WTo. < O.
Sirice h, —Tsx" is assume(' to be infeasible, the optimal value to (2.12) is
strictly positive by duality, the optimal value of
max UT (h, — ne)
WT 0
s.t.
o- <
(2.14)
that is, o-T(//, —*ne), is strictly positive, which leads to inequality o-Ths >
°lyse. From the first constraint of (2.14), u satisfies 1,17Ta < 0, that is
aT W y < 0 for ail y > O. Therefore, u satisfies ail the requirements and cuts
off the x winch leads to second-stage infeasibility.
Optimization Methods for Stochastic Programming 38
2.3.2 The Multi-cut L-shaped Method
In the previous section, we generated a single eut for the SP model. How-
ever, the structure of stochastic program allows the introduction of several
cuts instead of a single one. Here, we explain the multi-cut version of the
L-shaped method taken from [9].
Algorithm 2.2: Multi-cut L-Shaped Method
Step 0 Set r ks , r = 0, s = 1 . S.
Step 1 Set y = + 1, and for (xi', • ,O's ), solve the linear program:
min e
s.t Ax = h
Dex > de , L = 1, . , 7.,
0 > ees + ees x, es =1, 2 ks . s 1
x > 0, 0 E
(2.15)
Step 2 For s = 1 S solve the second-stage feasibility problem:
min w = eT v + eT y- .
s.t. Wy + y+ — = h 8 —
y,v+, - > O.
(2.16)
If w > O. let a be a dual multipliers, set r = + 1, = o-TT ,
Optimization Methods for Stochastic Programming 39
dr = aThs , and return to stop 1;
else, w = 0 for ail s = 1, • • • , S, go to step 3.
Step 3 For s = 1, • • • S solve the second-stage problem:
(2,(e) = min
s.t.
(1, Y
Wy = h, —
Y > O.
(2.17)
Let the dual multiplier be A.
If 0: < NAT, (h, —ne) ; set k, = ks Eks = —PsXrsTs eks = P8A.Ths• If 0 > ps AT, (h, — ne) for ail s, xv is the optimal solution, STOP;
else return to step 1.
2.4 Sampled Based L-Shaped Methods
Stochastic Decomposition and importance Sampling generalize the L-Shaped
method for two-stage stochastic linear programming. In these methods, only
a subset of realizations are samples in each iteration to solve the subproblem.
This is useful for problems with a very large nmnber of scenarios or when the
random variables are distributed continuously. We give more details about
these methods in this section.
2.4.1 The Stochastic Decomposition Algorithm
In section (2.3) we introduced the L-shaped method as a stochastic decom-
position technique. The L-shaped method incorporates a given discrete dis-
Optimization Methods for Stochastic Programming 40
tribution or an approximate discrete distribution of a continuous one in the
model. Then it generates feasibility and optimality cuts at each iteration to
obtain the optimal solution. Stochastic Decomposition (SD) algorithm by
Higle and Sen [41] employs the same framework as the L-shaped method ex-
cept that it uses a sample froni the distribution at each iteration to generate
approximate optimality cuts at the current solution.
Iteration k of the L-shaped algorithm solves a ma.ster problem of the form
unin c x + 0
s.t Ax = b,
0 > es + Esx, s = k — 1,
> 0, 0 E
(2.18)
where 0 is an auxiliary variable which takes the place of the expected value
function Q (feasibility cuts can be added to the master problem if it is re-
(luired).
To illustrate the SD method, let (xk , 0k) be an optimal solution to (2.18).
To construct an optirnality cut at Xk generate e and compute the cone-
sponding hk = h(e) and Tk = T(), and solve the following second stage
dual problem:
max{A(hk — Tkxk ) < g} . A
If we assume that the recourse structure is complete, then the dual feasible
set V = {A E < is nonempty and bounded, hence the optimal
value alliais to Ak (hk — Tkxk ) where Ak is one of the vertices of V. Let
Optimization Methods for Stoehastic Programming 41
vk = u {,\u.} be the set of dual vertices at iteration k, then for the
current solution Xk and ail previously generated samples s = 1 k-1,
solve
max{A(hs — xk) c
to °Wahl an optimal solution À. Then the new optimality cut at Xk would
be
AZ(hs —1'4= ekk Ex. (2.19)
The computation of cuts in the SD method is fast but inexact. In addi-
tion, the cuts may not be valid since they rua y not be lower bounds for the
expected value function Q. To fix this let L be a lower bound on the dual
optimal value with probability 1, then the old cuts 0 > esk _ i + =
1, • , k —1 are translated to
1 s = 1, • •• ,k —1. k k-1
Therefore the SD algorithm explained above would be:
Algorithm 2.3 Stochastic Decomposition
Step 0 Let k =1, Xk = O. V = O.
Step 1 Draw a random sample (just a single draw) ,e, k and solve
Ak = max{Ak(h(e) — Tk (es- )xk )111/T <
Set V = V U
For s = 1, 2, • • • , k — 1, solve = maxAcvlAk(h() —
Optimization Methods for Stochastic Programming 49
Step 2 Create a eut as 0 > Esk=1 Ais,(t() —
Call the eut ek Ek x.
Step 3 Solve the Master Problem
(xk , = min cTx +0 s.t. 0 > ek + Ekx. Vk=1. 2.... K. xEx
Go to 1.
Since it is not possible to sample ail the realizations in sueh an algorithm,
the optimality cuts generated are only statistically valid. and convergence can
only be proven in a statistical sense [41].
2.4.2 The Importance Sampling
The Importance Sampling [17, 42] combines Monte Carlo sampling [68] with
an L-Shaped method. Importance sampling reduces the variance of the so-
lution estimate. In each iteration, a subset of realizations is used to generate
the feasibility or optimality cuts. Optimality cuts are only statistically
Probabilistic lower and upper bounds are eomputed and the algorithm ter-
minates when the bounds are statistically indistinguishable. Because ail the
scenarios are not tested, the second-stage feasibility is flot always guaran-
teed. The L-shaped method with importance sampling can be parallelized.
Computational experiments [42] show that importance sampling is capable
of solving problems with a large number of scenarios to high accuracy within
95% confidence interval of the solution.
Optimization Methods for Stochastic Programming 43
2.5 Multi-stage Stochastic Programming
Many real-world problems sua as transportation problems and financial
planing cï.in be fornmlated as multi-stage or two-stage stochastic linear pro-
grains with recourse. In these types of problems, a stage usually represents
a tiine period. At the beginning of cadi stage, we know deternnnistically
ail the data in the current stage, but ail the data in the future stages are
stochastic. At the beginning of the first stage, decisions must be made before
the realization of the random data in the future stages. Once the random
data in a stage become known, correction (recourse) actions are allowed to
compensate the decisions for this stage made earlier. The effectiveness of the
clecision-making is measured by the total cost consisting of the deterministic
cost in the first stage and the total expectecl cost in the future stages.
In the previous sections we were mostly concerned with two-stage SP.
Here, we would like to extend the idea to multi-stage stochastic linear pro-
gram with fixed recourse. The general formulation of such a problem is,
min z = c1 x 1 + E [min c2 (W)x2 (w 1 ) E ii [min cH (w)xli '(wH-1 )] ...]
s. t. -1/171 :y 1 =hl
Ti xl w2x2 = h2 (2)
(2.20)
TH-1 (w ) xH-1 (wH-1) ± wHxli (wH-1) =hH (wH-1)
x1 > 0 , xv (wv) > y = 2 H.
Optimization Methods for Stoehastic Programming 44
where e1 is a known vector in Rn', h1 is known vector in R.Ini and ev(w) T = (c1) (w), le(w) 774 1 ) v = 2„ H, is a Ni, random vector and
each 11711 is a known x nv matrix. The decision x is dependant on the
history up to time v which is indicated by wv. As an example, we demon-
strate how to formulate a simple financial planning problem by multi-stage
stochastic programming.
A Financial Planning Example: Suppose we wish to provide for a child's
college education Y y-cars from now. We currently have 5b of budget to invest
in an y of I = 2 investment plans. After Y years we could have a wealth that
we would like to have exce.ed a tuition goal of $G. If it exceeds then it would
be equivalent to having an income of qa/0 of surplus money, while not meeting
the goal would lead to borrowing for a cost of r% of amount that is short.
We also suppose that we eau change the investments every v years so we
have H = Y/v investment period.
Suppose that we have 3 stages and 2 types of investments
S = s2, 531 s1 = 1, 2; s2 = 1,2; s3 = 1,2.
The major uncertainty in this model is the return of each investment i within
the period t. We denote the random variables to be t. w) and the decision
variable to be z w). The scenarios tree then would be represented as
follows:
The decision variables at euh node of stages 1 to 3 are represented as
t= 1 decision (i, 1) ; j = 1,2
t = 2 decision (i, 2, s i ) = 1, 2; = 1, 2
Optimization Methods for Stochastic Programming 45
S3=2
Figure 2.1: Seenario Tree.
Optimization Methods for Stoehastic Programming 46
t = 3 decision 1, si, s2); j = 1.2,8 1 ,8 2 = 1,2.
Similarly we denote the returns as
t = 1 decision 1) ; j = 1.2
t = 2 decision 2, 81) ; = 1,2; si = 1.2
t = 3 decision 1, , s2) = 1,2, s i s2 = 1,2.
In general for H stages and N types of investment, the multi-stage stochas-
tic program consists of:
• The objective function for s i , sH is
max (s1, ((.1Y(si. — rw(s i . ... , sH)) SJj si
• The first-period constraint is the budget constraint. It indicates that
in the first period we can only invest as much as the initial wealth.
Id x(i,1)= b. iEN
• Constraints for periods t =- 1 H —1. for each s st_ i are
— 1, s, — 1,s 1 , 9e-2) = E :E(i, t — 1,81, ,St_). iEN
• Constraints for period H to meet the tuition goal of G' is
t-1, sH)x(1,t-1, s SH-i) — Y(Si, s 11)+71(s:h 8H) = C.
iEN
iEN
Optimization illethods for Stochastic Programming 47
2.5.1 The Nested Decomposition Method
The nested decomposition [76] extends the L-shaped method to solve multi-
stage linear prograumfing problems, where the uncertain parameters have
finite realization. Nested decomposition method solves the first stage and
treats the remaining stages as another multi-stage stochastic programming
and solve the problem recursively. At cadi iteration, the algorithm passes
forward the current solution to the later stages (forward passes), while infor-
mation passes backward in the form of the feasibility and optimality cuts to
ancestor problem (backward passes). There are different sequence procedures
that tell in winch order the problems corresponding to different nodes in the
scenario tree are solved. One of the most common sequencing procedure is
called Fast, Forward-Fast Backward. It vas initially proposed in [76] for de-
terministic problems and implemented for SP in [29]. This method suggests
that the algorithm should proceed in a direction as far along as possible until
an infeasible subproblem is fourni (when going forward) or when there are
no more cuts to pass back (when going backward). The rate of information
exchange is minimized in this method.
As with the L-shaped method, the optimality cuts need to be developed
for nested decomposition schema. Assuming the problem lias (relatively)
complete recourse the optimality eut, 07 1 > et!-1 — can be con- - -3 3 '
Optimization Methods for Stochastie Programming 48
structed with the component e and E as follows
( I)v-i)[Al lek ± kEDi(i) 3
Sk
i= 1
.ic'„v
" kkt
rk
i= 1
Ev-1 = 3 ( v-1)
Pvk \vrrv-1
kED,(j) Pj (2.21)
where Dv(j) denotes the stage y deseendent's of a seenario j at stage /.) — 1;
p, and u are the dual multipliers corresponding to the fixed constraints,
the feasibility cuits ; and the optimality euts in the multi-stage stoehastic
programming formulation. We assume that ail of the above variables have
finite upper bounds.
The nested decomposition method ean be implemented via parallel coin-
tutting. Parallel programming allows problems to be solved more quickly or
with more details. By parallelization, sets of scenarios are assigned to proces-
sors to ereate feasibility or optimality cuts simultaneously. Ruszczynski [65]
suggested a different way to parallelize the nested &composition algorithm:
all subproblems are solved at the same time to generate new cuts and current
solutions. This new information is then passed to a special buffer so that the
task can be perform asynehronously. This method significantly reduees the
idle time of the processors waiting for new information to corne in. Sec also
[7] for other parallelization sehema.
Chapter 3
A Decomposition Based on an
Interior Point Algorithm
3.1 Introduction
In the last chapter we went through simplex-based &composition algorithms
which are used in SP problems. However, the use of an interior point method
is often a. better choice in solving large-scale linear programming problems.
The linear SP problems are very large and normally have special structures
which are exploitable by interior point decomposition methods. This chapter
foeuses on an elegant interior point based &composition method, namely, the
Analytic Center Cutting Plane Method (ACCPM) and its implementation
on two-stage and multi-stage stochastie programs.
The ACCPM uses an interior point algorithm combined with classical
cutting plane methods (or column generation). The &composition and par-
titioning principles have been built in ACCPM takes advantage of the special
A Decomposition Based on an Interior Point Algorithm 50
structure of stochastic programming problems. The significant contributions
to this method are by Coffin and Vial [34, 33].
3.2 The Analytic Center Cutting Plane Method
The main contribution of the ACCPM is attributed to the minimization of
nondifferentiable convex functions approximated by the intersection of sup-
porting hyperplanes to its epigraph. The algorithm \\Thiel' is implemented in
ACCPM integrates two powerful techniques, namely, cutting planes and in-
terior point. Ï.1.1gorithm. The general mathematical format of an optimization
problem in which the ACCPM is applied to, is represented by
minff(x) = coT + fi (x) x E X = xo n, Xe}, (3.1)
where sets X, c R. L = ....r and functions --> 7, j = 1
are convex. Moreover we assume that X0 is a bounded polyhedron defined
by lineïn• inequalities Ax < b. Due to the convexity of the problem, the
epigraph of both f (z), i = 1, , / and X, , e = 0, ,r eau be approximated
by the intersection of their supporting hyperplanes. This gives piecewise
linear approximations to a feasible region as well as an objective function.
In each step, the ACCPM tries to refine these approximations. To achieve
this, in ea.ch query point (or test point), say ACCPM calls an oracle to
obtain the objective function value f(:), and subdifferential f Based
on the information which is provided by an oracle, ACCPM generates cuts
or improves the approximation to the objective function (from now on, we
A Decomposition Based on an Interior Point Algorithm 51
assume that X = X0 nre=i Xt, is a bounded polyhedron).
3.2.1 The Oracle
As mentioned, the ACCPM uses a cutting plane technique to localize the
polyhedron within the current relaxed feasible region in which the minimizer
x* of (3.1) exists. Such polyhedrons are called localization„sets. Localization
set are defined by an outer approximation of the epigraph of the current LP
relaxation and bounded above by the best objective function value observed
so far. Computing the analytic conter of the localization set. at each iteration
provides the query point for the next iteration. However, the query point
might not be strictly feasible for the next iteration due to the introduction
of a cutting plane. In this case we have to find a new query point by adding
a feasibility eut. The information that is stored in the oracle at each query
point, in addition to the fact that oui problem is convex, helps us to construct
the current localization set as well as generating the feasibility cuts.
Given a query point î. the oracle first tests whether G X. If J; E X, then
the oracle computes fi (5;) and a subgradient E f GO with the property
(x) fi () + -yi (z — Va; E X. j = 1 (3.2)
The epigraph of f is defined by the intersection of the above inequalities.
Each of the inequalities provides an optimality eut. When the query point is
feasible, the objective function value at would be an upper bound on the
optimal value, i.e.,
fi (j;) i=
A Decomposition Based on an Interior Point Algorithm 59
On the other hand, if the query point is not feasible, x X then the oracle
generates a hyperplane (^,(0 , -y) E 7? x Riz that separates § from X i.e.,
- 4 + 7o <O Vs E X. (3.3)
This Iwperplane represents a feasibility eut.
To have a better understanding of how the fea.sibility and the optimality
cuts improve the description of the feasible set and the objective fimetion let
us divide the query points into feasible and infeasible ones. Suppose that the
oracle lias been called K times and the following two sequences are made,
where K=IK U JR -,
/1( = { k Xk is infeasible} , (3.4)
{ k I Xk is feasible} . (3.5)
The optimality cuts at sk , k E 1K define a piecewise linear approximation
to the convex function f whereas the set of feasibility cuts at , k E /K
define an outer approximation of the feasible set X. These new definitions
of the objective function and of the feasible set provide a way to define a
relaxation to the master problem (3.1). We shah l call it the relaxed inaster
problem.
3.2.2 The Localization Set
Suppose that JK and define the .upper bound 6,„ = mink f (51 k E JR}
The following polyhedral approximation which is called a localization set at
A Decomposition Based on an Interior Point Algorithra 53
the iterate k is denoted by £K:
Vk E JK , i=1 ,
0 > - Xk ) 74e , (3.6)
éK > + • • + Z1,
The first inequality is a set of optimalitv constraints. The second set gives
the fea.sibility cuts. The third constraint makes z = z 1 + • • + zj fall bellow
the upper bound K• The last set represents the fixed constraints, winch
gives a boundary to the feasible region. These sets of constraints provide the
best outer approximation of the optimal set in (3.1).
Lower bound: A lower bound on the objective function cari be obtained
by using the dual of (3.6). Let us associate the dual variables aik > 0 to the
set of optimality cuts, ftk > 0 to the set of feasibility cuts, and 6 > 0 to the
fixed constraints in (3.6). The variable vectors ak. •k, 6 are dual feasible if
they satisfy the following system of equations:
E aik-Yi + kEJk
+ ATS =
E aik kEJ,„
= 1, i=1 J (3.7)
Multiplying each of the optimality, feasibility, and fixed constraints in (3.6)
1)3,7 aik , [I, 6, respectively, and simplifying the equation by the result in (3.7)
A Decomposition Based on an Interior Point Algoritirrn, 54
provides
_ i=i i=l kEJK
ik (fi (xk) _ „yika;k) ilk ey:)c ,ykxk) (3.8) ke it-
The right-hand side of (3.8) is a lower bound on the objective function (this
lower bound will automatically fans bellow OK in each iteration).
The initial convex feasible dornain shrinks repeatedly by the removal of
regions which caillot contain the optimmn. The ACCPM selects the query
point by fincling the analytic center of the localization set. There are also
other cutting plane methods such as Levin's center of gravity and the max-
imum inseribed ellipsoid, which maintain the volume reduction property as
well as the centering. The idea of cutting plane method originated with Kelly
[43].
3.2.3 The Analytic Center
The ACCPM finds an miter approximation of the polytope of the feasible
region at each iteration, the analytical center of this polytope (localization
set) is chosen as a query point. Cuts are generated from tins analytic center
of the set of localization and are added to the &finition of the polytope.
To compute the analytic center from now on, ive use the compact def-
inition of the localization set. Here the index K is dropped for simplicity,
therefore the localization set ,C is defined as
ty E TC1 I ATy > 01,
A Decomposition Based on an Interior Point Algorithm 55
where A E TL' and c Given a point in L, define a metric d by:
1. d(y,£) = 0 if y is on the boundary of G,
2. d(y, G) > 0 if y is in intG,
3. 1f < c and G = fy e Rni I c' Any > 01, then d(y, G') < d(y,
This metric is dependent on the analytic representation of L. one choice
of this function would be
(1 (y , £ 71
aT : y)•y E G,
where a . is the j'il column of A. The logarithm of d(y,£) is called the dual
potential function:
(I) D (y, G) = log d(y. G) = E log (c:i — (!Ty)
(3.9) j=1
where si = ci — ary. The pair of interior points y' and sa = c — ATya in G
that maximize the potential function is called the anal tic center of G, i.e.,
op(ya. ) = max log d(y. G), yEC
or simply.
max{cbD (s)ls = c — ATy > 0}, (3.10)
where ØD(S) = E3n. log si . Note that (ya. sa) is a unique maximizer, since
the potential function is strictly concave.
A Decomposition Based on an Interior Point Algorithin 56
Let xa = (Sa) -l e, the analytic center ( lia , 3) together with xa satisfy the
following optimality conditions:
X8 e ,
Ax = 0,
AT y + s (3.11)
where X = diag(x). Therefore, an analytic center is the point that maximizes
the product of the distances to ail faces of the polytope defined by L. An
approximate analytic center relaxes the non-linear constraint Xs = e in the
first order optimality condition (3.11), i.e.,
X s
ATy s s > 0, (3.12)
where any pair of (x, s) defines an 17-analytic center.
Example: Consider the system G = fx I Ax = 0, eTx = n,x> which is
called Karmarkar's canonieal set. If x = e is in G, then e is the analytic center
of G. Tt would also be the intersection of the simplex {x I eT x = n, x >
and the hyperplane fx, I Ax = 01.
The analytic center can be alternatively computed as the optimal solution
of
max-Pp(x) j Ax = 01, (3.13)
A Deeornposition Based on an Interior Point Algorithin 57
where Op(x) = --erx Ein=i log xi . We ¬e the OD(S) in (3.9) is a dual
potential function whereas (;) p (X) is a primai potential function. The primai-
dual potential is then defined as
OPD = OP(-1.) OMS).
3.2.4 The ACCPM and Interior Point Algorithms
The ACCPM uses an interior point algorithm to compute the analytic center
at cadi iteration. The Primai Algorithrn, Dual A lyorithm, and Primal-Dual
Algorithrn are considered the most efficient interior point algorithms in prac-
tice. We avoid the detailed discussion about these algorithms and refer the
reader to the existing literature; for example see [77, 78].
A primai algorithm needs only a. primai initial point. This algorithm up-
dates the solution so that it converges to the analytic center, dual solutions
are by-products of the procedure. A primai algorithm is very efficient, spe-
cially when additional cuts are added to the definition of the localization set
and the ACCPM needs recentering.
A dual algorithm needs a dual-feasible initial point. The algorithm then
updates the dual solutions and the primai solutions corne as a by-product
of the procedure. Each solution is generated by solving a system of linear
Expiations. Hence, a sequence of simpler problems solves the original linear
program to optimality.
The primal-dual algorithm is very popular in practice. The algorithm
treats both primat and dual variables equally, and updates the primai and
the dual solutions in each iteration.
A Decomposition Based on an Interior Point Algorithin 58
3.2.5 Multiple Cuts
The efficiency of the ACCPM improves if we are able to (i) add stronger
cuts in each iteration ; (ii) add multiple cuts simultaneously. The addition of
multiple cuts means allowing the oracle to return a set of cuts for eaeh query
point instead of just one. The ouality of the added cuts is measured by the
portion of the localization set which it eliminates. Suppose ; in iteration k.
Bk,yk - rk,
is placed. If -y > 0, the eut is deep; if -y < 0 the cut, is shallow; finally if -y ---- 0
the eut passes through yk and it is called a central eut. Deep cuts are the
strongest cuts among them.
Goffin and Vial [361 investigated the multiple eut generation in analytic
center cutting plane scheme. They proposed a method for finding a new
analytie center after adding multiple central cuts that required a solution to
a nonlinear program.
3.2.6 Adding and Dropping Cuts
Feasibility and optimality cuts in algorithms like L-Shaped method (section
2.2) eliminate either a portion of the feasible region or a contour of the
objective function in which the optimal solution does not happen to be. In
the ACCPM, cuts function similarly except that each time a eut is added ;
the coordinates of the analytic center are changed and hence a recentering is
required.
On the other hand ; the ACCPM includes a mechanism to prune the
A Deeomposition Based 071 an Interior Point Algorithm 59
redundant cuts where the problem is getting very large as a result of the
addition of single or multiple cuts in each iteration.
In this section we expiait' how to restore the anal tic center after adding
or dropping a cut.
Adding Cuts: Unlike many decomposition methods, in the ACCPM [33]
the relaxed master problems are not solved to optimality. Instead it seeks
the analytic center of the localization sets as a starting point for the next
iteration. This analytic center then, will be fed to the oracle to generate new
(nits. Generating cuts at each iteration of ACCPM means adding colunms
to the coefficient matrix of the relaxed master probleni. This changes the
definition of the localization set, hence the coordinates of the analytic cen-
ter. Therefore, designing an efficient way to find the new coordinates of the
analytic center alter adding cuts significantly improves the method. This
requires obtaining a new search direction after adding cuts.
To compute the search direction, let us partition the variables into two
sets. The first set of variables, winch corresponds to the old problem is de-
noted by z. s, whereas the variables corresponding to the new problem (which
differs from the old problem by a set of new cuts or columns) are denoted by ,new ,new . .
The first order optimality conditions to compute the analytic center as
A Decomposition Based on an Interior Point Algorithnt 60
we have seen in (3.12) is
Xs
As = 0,
AT y + s = e. (3.14)
Assume that a new cut BTy < r is added to the above problem. This causes
the old analytic center (y, s) not to be feasible to the cuiront localization set.
In fact adding a eut elhninates a portion of the dual space, which causes the
infeasibility of the current analytic center. This corresponds to introducing
a new variable in the primai space. It is often not practical to restore the
dual feasibility and maintain the centrality condition satisfied with the last
barrie'. parameter. However, the primai feasibility is always casier to restore.
The idea here is to restore the primai feasibility by adopting a ne -w search
direction Ar" and step size /3 that ensures s+,6z.Vx1ew > 0 gets to the interior
of the primai feasible region.
The analytic center after adding the cut BTy < to (3.14), is computed
from the following system of expiations:
ATy + s = e, s > 0,
BTy + = r, > 0,
As + Ba
( IaX s) (3.15)
where a and a are new primai and dual slack variables respectively, r =
A Deeomposition Basal on an Interior Point Algorithm 61
diag(a). Therefore, the coordinates of the new analytic center in the primai
space, say satisfies 1new = + Ax + Ba = O. x > O. c > O. We
adopt the direction Axne" winch is proposed in [56].
Arr e_ix2 AT (Ae_1x2 AT ) -1 Be,
=
where e = XS- I is a primai-dual scaling matrix. Hence,
AAr _Ae-1x2AT pe-1x2AT) -1 Be
AT' ± Ba =0, Ax = O.
Vector x is a strictly positive solution to the old problem, the maximum value
of /3 > 0 such that x"ew= x + /31rr > 0 is given by the ratio test
xi = min Anew
x3 Arxir < 0, .
Choosing the step size fl e (0,13max ) directs xnew to strictly primai feasible
interior, where the search for a new analytie center begins by using the primai
potential reduction interior point algorithm [78].
Coffin and Vial [36] propose a rigorous way to recover the primai fea-
sibility and the recentering alter adding a eut. The method is particularly
interesting when it brings the newly added variable to the primai space, say
into feasibility. It proves that a is a maximizer of a quadratic function
with self-concordant barrier[24], F(a). Hence, a feasible a cari be obtained
in finite Newton steps.
A Decomposition Based on an Interior Point Algorithin 62
Dropping Cuts: The number of constrains in the relaxed master problem
increases as the cuts are added. This number increases by one (for a single
eut), or more (for multiple cuts) at each iteration. This makes the coin-
putation of the query point at each iteration a tedious ta.sk . The ACCPM
includes a mechanism for dropping a set of redundant cuts as the algorithm
progresses. Dropping the redondant constraints does not change the defini-
tion of £K.
The simplest way to check whether a linear inequality aT <b is redun-
dïmt is to solve the linear program
MaX
s.t • 1) <- j =1 rn, .1 —
Then <b is redondant if the optimal value is Inger than or equal to
b. Checking the redundancy of each constraint by solving a linear program
is very costly and normally a,voided. Another method defines an ellipsoid, F,
which is inscribed in the current localization set. If the maximum value of
o,Tx over F is smaller than or equal to b , the constraint ara; <b is redondant.
ln other approaches, constraints are dropped even if they are not known to
be redondant. Heuristics are used to rank the constraints in relevance, the
least relevant ones being dropped first.
In some cases, the ACCPM uses the inscribed ellipsoid method [3, 36] to
detect the redondant constraints. After detecting and pruning the redondant
constraints, the ACCPM lias to compute the next analytic center and recover
the feasibility. Here we show how to recover the dual feasibility after dropping
a number of constraints. Suppose that the first order optimality conditions
A Decomposition Based on an Interior Point Algorithin 63
to find the current analytic center (1:, are given as
c, > 0.
± d = r, > 0,
AT + B 0 ; x > 0, à > 0,
= rd = p.. (3.16)
Alter dropping the cut Bny < r ; we need to compute the new analytic center
that satisfies
Xs = 7/:
ATy + s = c ; > 0,
Ax = 0, x > 0. (3.17)
Since erj .) are dual feasible (dropping cuts (1oes not change the definition
of the localization set), we use them as the initial point to the dual Newton
method to search for a new analytic center. Let the new iterate be x =
y =V + dy, and s = .5-;" + ds , thon (3.16) reduces to
Adx = —Bd.
Sdx + Xds =
ATdy ds = 0,
A Decomposition Based on an Interior Point Algorithra 64
where dx.ds is ignored. The above system has a solution
dy
ds = —ATdy,
da; Xds+
where V = AXÉ-1 AT. To compute the step size in each of the directions
(1s, dx, let us denote
fi = + X -1 S-1 1),
f2 =
Therefore dx = Xfi and ds = Sf2 . Taking the step size of là along dx
and Ilft2ii along dy and ds, where t E (0, 1) gives us the new iterate:
t , t X 1; + ,
l a x = X ( C ±
Il fi II fi) Il i 1 l s , ± t. d s = S(e +
11:ti.2 11 fi) I I J2 11
t t
ij + dy = 9 ATds II f2 II II fi II
winch satisfies ATy + s = e, x > 0, s > 0, hence y is dual feasible. Primai
feasibility is not achieved since Ax 0.
A Decomposition Based on an Interior Point Algorithm 65
3.3 The ACCPM for Stochastic Programming
with Recourse
There are different formulations and corresponding solution methods for
stochastic progra.mming. One formulation is based on tue deterministic
equivalent. Deterministic equivalent is a large-scale linear program winch
can be solved b simplex or interior point methods. A deterministic equiv-
aient problem includes ail constraints and scenarios, hence it becomes very
large, specially in the case of multi-stage stochastic progriumning.
On the other hand, stochastic programming with recourse lias I: special
structure that can be exploited by decomposition. The primal of a recourse
problem lias a dual block angular structure, where its dual has a primai block
angular structure. These structural advantages makes the dual problem to
be handled by the Dantzig-Wolfe decomposition method. Equivalently, the
primai can be tackled by Benders decomposition or Kelly's cutting plane
method. Other algoritlmis such as the L-shaped method are designed to
exploit this special structure.
The ACCPM combines the interior point algorithm with a cutting plane
method, providing a fast algorithm for solving two-stage stochastic program-
ming with recourse [3]. The decomposition and partitioning principles have
been built in the ACCPM and it takes advantage of the special structure. In
ACCPM, new cutting planes are generated from the analytic center of the
set of localization as compared to Kelly's cutting plane method, where cuts
are generated from the optimum of a linear programming relaxation. The
analytic center contains the information from the past iterations and it is
A Decomposition Based on an Interior Point Algoritirm 66
a guide to the convergence of the method to optimality. In the next sec-
tion we investigate the use of the ACCPM for solving two-stage stochastic
programming with recourse.
3.3.1 Stochastic Programming with Recourse
The general formulation for two-stage stochastic programming with recourse
is
min z = co Xo Q (X0, ) d:1:0eeE (3.18)
SA. A0:1:0 > bo ,
where :ro is a first stage decision and is a random variable with finite sup-
port E. Furthermore, we assume a feasible recourse, i.e., for some x 0 , with
Aoxo feE ., Q (4, 4) chro is finite.
Moreover, the function Q : 7Z" x E R, is defined by
Q (x) min {J(03; I BWx > 140 — A()3;0 1 . (3.19)
Since E is a finite problem, (3.18) can be written hased on its deterministic
equivalent, i.e.,
min z = cPco +
s.t. il0X0 >
i=1 (3.20)
A Decomposition Based on an Interior Point Algorithm 67
with
L (x0 ) = min { eiTx Bi x > h — Aixo } j = 1 1 (3.21)
where x is a second-stage variable.
The relaxed master program after k iteration, as it is described in section
3.1, is defined as
min z = c0 X0 E zi i=1
s.t. A0 X0 > bo , zi > (x0 _ xok) k E jKi
-yoki (-yik ) T (x0 — xt;) E i 1.
The dual formulation to (3.22) is
(3.22)
max bonu0 + (fik _ ( ,yik )Tx0k ) ,ik E
i=1
k■T k k ' ) X 0 !Li
kE Jk
_
i=1 kcJi, kElk
k k "Yi = co, (3.23)
E = 1, 1 = 1 /, kEe
uo , A125, > 0, i e
where A li' are dual variables associated with the optimality cuits, and 145 are
dual variables associated with the feasibility cuts. The query point x10̀ is a
A Decomposition Based on an Interior Point Algorithin 68
solution to the relaxed linear program(3.22), at winch the oracles are called.
The answers of the oracles are then acide(' to (3.22) to get a better approxi-
mation.
is the lower bound to (3.22) which is given by any feasible solution to
(3.23)...f1( is the best feasible objective to (3.19) winch is so far known, i.e.,
1_ E J iy_.k 1 , i Aoxo > bo . 10; o I 7' 1' = min c0 3, 0 k=1,...,K
The optimum of (3.19)„ occurs at 71( < 7* <
3.3.2 The Potential Function and the Analytic Center
For the sake of simplicity, we drop the iteration index k and rewrite (3.22)
as the following linear problem. Further, we assume that the problem lias a
bounded feasible region:
min z = cTx
s.t. As — s
s > O.
(3.24)
where s is a primai shck vector anci TT — (x', if). The dual to
(3.24) is:
ax bTy
s.t. A .,_•T y = c,
y > O.
(3.25)
i=1
A Decomposition Based on an Interior Point Algorithro, 69
The set of localization is
£2 = I > cTx, > .
The potential function is defined by
.1)(x, = log ( c'Tx) log si . (3.26) j=1
The analytic center of the set of localization C is the unique solution to
vx0 = O. i.e.,
— cTx + ATS-1 6 = O.
Therefore we eau define the dual variable y as
y = — cTx)Se > O.
The analytic center eau equivalently be computed from the system of equa-
tions obtained from the first order optimality conditions
AX -s = b, AT y =
S y = (2 — er e, (3.27)
where S = diag(s). The above conditions express the primai feasibility, dual
feasibility, and the centering condition. To get the fuite convergence to the
A Decomposition Based on an Interior Point Algorithm 70
analytic center, we need to relax the centering condition to
— (2— (.T.r) e M
where 0 < < 1. The ACCPM algorithm is then described as follows:
Algorithm 3.1: The ACCPM Algorithm
initialization
A feasible point ro and its corresponding upper and lower bounds f, z,
The initial set of localization Gi•
While z — 2> c
Compute the n approximate analytic center of L. say r, and its cor-
responding dual solution y.
Update the lower bound z by using z = max{17y,
Call the oracle at point r and update the upper bound 2 if there is
no feasibility eut added.
Update the localization set G 2 .
End-while
3.3.3 ACCPM on Two-Stage Stochastic Programming
The implementation of ACCPM on two-stage stochastic programming [3]
uses a variant of Karmarkar's [20] primal potential function on the dual of
A Decomposition Based on an Interior Point Algorithm 71
(3.25). It computes the dual analytic centers by a primai projective algo-
rithm. The algorithm obtains an efficient search direction to recover feasi-
bility and find the new analytic center after adding new cuts. ft also has
the ability to remove the redundant information 1)y dropping cuts. The al-
gorithm identifies the redundant cuts by the procedure proposed on [351. ft
then efficiently restores the feasibility and find the new analytic center for
the current localization set.
The algorithm lias a polynomial thne complexity and it is faster (in the
number of iterations) than the class of decomposition methods, namely, Ben-
ders' decomposition and Dantzig-Wolfe decomposition, winch are used in the
stochastic programming context.
3.4 Two-level ACCPM for Three-Stage Stochas-
tic Programming
3.4.1 Column Generation on the Dual
R,ecall the multi-stage stochastic programming formulation in (2.20). To
formalize the two-level ACCPM, consider the following three-stage SP:
T .(1) (1) (1) (2) (2) (2) min cx 2—k=1 ck Pk Yk Lik=lek Pk Yk
s.t. Ar =
+ 1171 y(1) =
112Y(1) + 14/2 Y (2) = 112, > on
A Decomposition Based on an Interior Point Alyorithm 72
where the stage 0 is deterministic, the number of scenarios in the first-stage
ism, and the number of seenarios in the second-stage is n = rn2 .X s the
variable vector associated with stage 0, whereas y (1) , y (2) are variable vectors
associated with the first-stage and the second-stage, respectively. Denote the (2) (2)1 th en q( 1 ) by [c (11) . , c( ,, ) lig)] and (2) by [c(12) P.12) • Pri tnen
the three-stage stochastic program cari be reformulated as
min
s. t.
(3.28)
(3.29)
(3.30)
(3.31)
(3.32)
The dual to the above program is
max br A + hir + h.T p, (3.33)
s.t. AT À + TrF < c, (3.34)
+ "'Lu < q, (3.35)
14'11/ < (12, (3.36)
where À is the dual variable associated with (3.29), Ir is the dual variable
associated with (3.30), and p, is the dual variable associated with equality
(3.31).
The stochastic linear recourse program in (3.28)-(3.32) is a determinis-
tic equivalent program and its constraint matrix lias a dual block-angular
A Decomposition Based on an Interior Point Algorithm 73
structure, moreover its dual constraints matrix in (3.34)-(3.36) has a primai
block-angular structure. Therefore, the stochastic linear program in (3.28)-
(3.32) is amendable to an approach by Benders' decomposition whereas the
dual program in (3.33)-(3.36) can be tackled by a Dantzig-Wolfe reformula-
tion and column generation technique.
Since the structure of the dual program is more supportive of decom-
posing the SP along the scenarios, we start the decomposition using the
dual. Hence the neural decomposition choke to tackle the problem would
be Dantzig-Wolfe. The combination of Dantzig-Wolfe reformulation, AC-
CPM and colunm generation would solve the three-stage problem in two
levels. In the first level we decompose the problern into a master program
(sec section(2.2.2) for more details) which is solved by ACCPM (section 3.2)
and subproblems. Each subproblem thon again decomposes into a master
program and subproblems. Figure 3.1 illustrates the two-level &composition
along the scenarios for a three-stage SP. Figure 3.1.a pictures the first-level
partitioning whereas the Figure 3.1.b shows how the subproblem in the sec-
ond stage is decomposed into a master program and subproblems.
To observe how the decomposition is donc, in the dual prograni, let the
first constraint, AT < c, to be kept in the n'aster (as the complicating
constraint). The master program is obtain by a Dantzig-Wolfe reformulation
on the objective and the complicating constraints in each stage (sec section
a) b)
Master Problem
Subprobleml Subproblet
I Suhp °bleui
Subproblerni Subproblenl Subprobleml Subproblemi Subprohleml
A Decomposition Based on an interior Point Algorithia 74
Figure 3.1: Two-level decomposition along the scenarios for a three-stage SP.
2.2.2 for more details). Then the first-level subproblem appears as
min — Ii5JT ir + h p.
s.t. 117T + T jia <
11/11-1 < (12 (3. 37)
where :7; is the primai value from the stage 0 to be carried over to subproblem
from solving the master program in the first level. This subproblem can be
decomposed into independent subproblems for cadi first-stage scenario. To
achieve this, let the dual variable in the first-stage TC , to be decomposed
to m distinct vectors for cadi sce,nario; i.e., 7r (1) = [741) [7r 2) if (2)1
k2 1 , 7r (771) = [Tr Irn) 7r i(rm ) ] . Now corresponding to cadi vector
variable 7r (i) ; i = 1, , m, we make a dual vector in the second-stage, i.e., fi ()) bil (11) t12)] ft(2) {142) , , p4n2)] fi (m ) [itr) J. Each
(k1 ) 7, 7 ( 2 )
vector = 1 m , is made by concatenating ail the m vectors in
the second level which is associated to one parent node in the first, level.
A Decomposition Based on an Interior Point Algorithm 75
Therefore, by concatenation the pi , j = 1, . , m2 , vectors in the second level
couvert to ft(i) i = 1, , m, vectors each associated to a parent node in the
first level. As a result the subproblem in (3.37) becomes
min >I Rhi — +
s. t. 1V7T(i) + T ft(i) «1 (1i ) , = 1, m
tiirift ( z ) < (h(22) Tri. (3.38)
Since the objective function and the constraints in (3.38) are separable in
terms of the variables the resulting subproblem reduces to Tfl independent
subproblems:
min (h 1 —171 ,Ï;)T'F (i) +
s.t. wb(i)-+Trift(i)<e),
1171iii (i) < (gi) . (3.39)
Such disaggregation adds m convexity rows to its master program. In
the next level, each of the m subproblems are decomposable in tenus of
the second-stage dual variable p. This tinte, in each subproblem we keep
the constraint + < e) , in the n'aster program level as a
complicating constraint. Each master progra:m in the second levet contants in
convexity rows as a result of the disaggregation in the second level. To have
a better understanding of how the second-stage subproblems are created,
we look at a specific subproblem in the first-stage, the ith subproblem for
example as it appeared in (3.39). For titis problem, let u to be the dual
A Decomposition Based on an Interior Point Algorithin 76
multiplier associated with constraints 1,1Dr(i) + Trie <q then the ith
subproblem eau be reformulated as
min (h2 — T2a)Tp,U )
s.t. Wrip,U) < (.0 j = 1 ?TL
Similarly, the above program decomposes into m independent subproblems
min (h2 —
s.t. 111.;P(3)
for j = 1 m. The above decomposition is repeated for each of the rn
subproblems in the first-stage, as a result there are m 2 subproblems in the
second-stage.
The n'aster programs in the first, and the second stages are solved by the
ACCPM, where it passes the query points (sec section 3.2), as central dual
prices to subproblems. In return, the subproblems in each level generate
columns to be added to the master program.
3.4.2 Benders' decomposition of the Primal
The Benders' decomposition principle is based on a variable partitioning
procedure in a large-scale primai problem, sec [4]. The method partitions
the variables by fixing the so-called complicating variables. This reduces the
A Decomposition Based on an Interior Point Algorithm 77
given problem to a more tractable, and hence casier, problem to solve. The
latter problem is parameterized in ternis of the vector of the complicating
variables. The dual to this problem forms the subproblem. In each iteration,
the subproblem passes forward the current dual solution (for a fixed vector
of complicating variables) to a master program in the form of a Benders eut
that utilizes the linear programming duality to reflue the value of the vector
of complicating variables.
In the particular case of MSP problems, the paditioning would be in
terrils of the variables in cadi stage. Shnilar to what we implemented on
the dual problem when we used the column generation technique, we the
decompose three-stage stochastic problem:
max cT X
S.t . A x Tl x
B
X. , y2 > 0.
+q2y2
= b
+ 1171 Y1
+Tel +1472 ://2 = h2,
(3.40)
in two levels by using Benders' decomposition. This can be donc by first con-
sidering the vector of the first-stage variables — namely yi = '/A }
as the complicating or fixed variables. This forms the following subproblem,
Tjx
B
X. 712 > O.
(3.41 )
+(hTh -1- (12y2
+14/2 Y2
= min ( T J . - .9PB1ffl i )
S.t. A x
A Decomposition Based on an 1nterior Point Algorithm 78
given that Yi is fixed at
Sol-ving (3.41) provides au upper bound on the original problem (asssuming
that this subproblem is bounded). Therefore, the solution to the original
problem consists in solving the above subproblem for the Lest y]. value, and
it is given by the solution of the following (first) master problem:
[111PB] : min yl>0 :' Z PB 1 (Y1)'
Note that the dual feasibility of the subproblem is independent of the value
Let À. 71", and it be the dual vectors associated with the primai constraints
of stages zero, one, and two, respectively. `nen, the dual constraints of (3.41)
are:
AT +T"7 +BT,u. <C.
1-47Tp, <
The dual solution of (3.41) is given by an extreme point (4, -Fp , p E P,
for which the objective function takes value
bAp — W1 Yibtp (h2 TWitp qiTh.
A Decomposition Baser] on an Interior Point Algorithm 79
Therefore, the optimal solution to the subproblem cari be interpreted in ternis
of the best extreme point in P, that is:
'SPB1(Yi) = maxpE p
= min
s.t.
bAp + (h1 — 14711.ii)irp + (h2 — Tei)Pp+ giTh
el
ei > + (h1 - Wl yi)irp + (h2 - T2Y1)Pp + (eh,
p E P.
The first master problem, which provides a solution to the original problem,
(;an flow be written as follows in terms of the extreme points in P and the
vector of complicating variables 71 1 :
2;;L' 1 PB
s. t .
min el
el> + (hl - np + (h2 - T2yi)pp + q1y1, p E P
iii > O.
To complete the two-level Benders' decomposition, note that problem
(3.41) cari also be solved by Benders' decomposition b -y taking the vector of
the second-stage variables 712 = fy?, y y n” as the complicating variables.
Therefore, given that 712 is fixed at F2 , the second subproblem appears as:
-* -SPB11(i 172 ) = min +gill i +q2Y2
s.t.
= b
= h 1 - 14/1
= h2 — Teh — 1472F2,
(3.42)
A Decomposition Based on an Interior Point Algorithm 80
Solving (3.42) provides an upper bound on problem in (3.41) (assuming that
the second subproblem is bounded). Therefore, the solution to the first
subproblem given 7j consists in solving the above second subproblem for the
besty2 value, and it is given by the solution of the following (second) master
pro bleui:
plIP/31(y1 )] : min Y2 >0
-* 'SPB11(-e 1 ,y2)•
The set of dual constraints for (3.42) appears as:
AT À +/T7r +Brp, <G.
The dual solution of (3.42) is given by an extreme point (4,7,7 ,1.1,J, q e Q,
for which the objective function takes value
bAq + — ) 1ra + (h2 — Tel — 1V2 Y2)tiq + (1.1 Ùi + Q2 F2.
Therefore, the optimal solution to the second subproblem cari be interpreted
in ternis of the best extreme point in Q. that is:
,* ‘'SPB11(-gi ,1/2)
maxgeQ bAq (h1 — W1Tir)irq (h2 712y 1 - w2 ://2)flq + (11 Th + Th
min (1) 2
e2? b + (h1 — 1)7r + (h2 — T2F 1 — 1472 Y2)I-tq +(11j 1 + (1.2 Th:
E Q.
The second master problem, winch provides a solution to problem (3.41),
A Decomposition Based on an Interior Point Algorithm, 81
can now be written as follows in ternis of the extreme points in Q and the
vector of complicating variables y2 :
,* -AIPB1(-y1) = min e2
s.t. 02> bAq + (hi — Ti1) 7rq + (h2 - - W2 y2)ilq
+ql ± q2 Y21 E Q
Y2 > O.
The two-level Benders' decomposition consists of two master problems
and a subproblem, where the master problems are solved by the ACCRM
and the subproblem is solved by the simplex method.
3.5 Conclusion
Marly real-word problems require that decisions be made periodically over a
finite time-horizon in the face of uncertainty. These types of problems are
°lien formulated as multi-stage stochastic programs (MSP). The determinis-
tic equivalent problems for 1MSP are very large, and direct solution methods
sud i as the simplex or interior point algorithms are very costly to be applied
to the problem. Given the size of the problem instances involved, decompo-
sition and partitioning methods are commonly used in MSP to reduce the
size of the problems a.t the time. The decomposition methods replace the
direct solutions of a large-scale problem with a: sequence of solution of smaller
problems.
In Chapter 2 and Chapter 3, we considered two forms of decomposition
A Decomposition Based on an Interior Point Algorithm 82
approaches: the Benders and the Dantzig-Wolfe algorithms, winch are viewed
as simplex-based decomposition algorithms; and the ACCPM. In the remain-
ing Chapters we present the variant of the two-level &composition algorithm
to solve problems. This algorithm is implemented by using ACCPM,
Dantzig-Wolfe and Benders methods. Finally, the numerical results of these
decomposition techniques are compared.
Chapter 4
A Two-Level Interior-Point
Decomposition Algorithm for
Multi-Stage Stochastic
Capacity Planning and
Techn.ology Acquisition
4.1 Introduction
Manufacturing flexibility is recognized as one of the key strategies to address
uncertain future product demands. Flexibility is regarded as an adaptive
response to an 'incertain environment hi which it enhances the competitive
advantage of a manufacturer or a service provider. Therefore, there is a
growing need to investigate the strategic aspects of flexibility.
A Two-Level Interior-Point Decomposition Algorithrn for Multi-Stage Stochastic Capacity Planning and Technology Acquisition 84
A body of scholarly research lias developed over the years, sec for example
[14, 51, 66], to explore the many lacets of manufacturing flexibility. Among
them, [66] provides a taxonomy for the classification of different kinds of
fiexibilities throughout the entire manufacturing chain, from the acquisition
of the raw material to the delivery to the final customers. This chapter em-
phasizes the expansion, product, volume, and market flexibilities according
to their classification.
Expansion flexibility allows many companies to adopt a consistent growth
strategy. Such organizations retain flexibilitv by investing slowly and keeping
their options open for graduai growth. The aforementioned trend is seen
particularly in the electric utility industry where the graduai investrnent on
the capacity balances out the advantage of having a costly large plant. By
having a large plant instead of two or three smaller ones, companies reduce
the unit costs, and therefore, the coherent decision would be to gradually
invest in the new capacity and add a large plant cadi time.
Alternatively, product and volume flexibilities help compa nies to be re-
sponsive to market changes by quickly introducing a new product design or
adjusting the production volume upward or downward within a wide vari-
able range. Since the future design and volume of a product is unknown,
an acquisition of flexible machinery - or in other words, flexible technology
is essential to being more adaptive to change. The car industry is a good
example of a flexible technology acquisition that supports the assembly of
diffèrent models on the saine production hue.
Finally, market flexibility is regarded as overall manufacturer adaptabil-
ity to the environment that fluctuates constantly. The market fluctuates
A Two-Level Interior-Point Decomposition Algorithm for M'alti-Stage Stochas tic Capacity Planning and Technology Acquisition 85
because of the fast pace of technological innovations, shorter product life
cycles, and changes in consumers' behavior. As the market changes in a dy-
namic fashion, companies ma y neeci to adopt a new product design, acquire
a new technology, or cope with volume fluctuation.
To capture the different aspects of market tlexibility in tire face of this
dynamic demand, this chapter focuses on tire role of product, volume, and
expansion flexibility in the context of the multi-stage stochastic program. By
using tins multi-stage program, one cari capture the evolution of the demand
across the planning horizon. 'Technology ïtcquisition or facility expansion can
be obtained once in cadi period before the realization of the random demand,
tire recourse action would be to either have an inventory for excess production
or to be penalized for flot meeting the demand of certain customers.
Given a finite number of stages, scenarios and their probabilities, the
capacity expansion and technology acquisition problem is a large-scale opti-
mization mode', which is tractable by decomposition. Decomposition meth-
ods are ways of solving a sequence of optimization problems virose size vary
increasingly based on tire information feedback given by the method used.
Hence, the decomposition algorithms reduce the size of the problem to make
it solvable. The classical decomposition tools for linear programming prob-
lems trace back to Dantzig and Wolfe [54] and Benders [4]. These &compo-
sition principles are translated into the L-shaped method [72] in stochastic
linear programming. The L-shaped method iras been extended to the nested
&composition method and developed as a computer code for linear multi-
stage rec ourse problems [6, 29].
The L-shaped method is viewed as a simplex-based decomposition princi-
A Two-Level Interior-Point Decornposition Algorithm for Alulti-Stage Stochastic Capacity Planning and Technology Acquisition 86
pie in contrast to a method such as the analytic center cutting plane method
(ACCPM) [33], which is based on an interior point algorithm. The ACCPM
combines the interior point algoritlmi with the cutting plane technique to
provide a powerful tool for solving large-scale optimization problems. The
ACCPM was first implemented for solving two-stage stochastic programming
with recourse in [3]. This specialized implementation is successfully used for
practical applications suc:h as the telecommunication network problem in [1].
A two-level decomposition algorithm via ACCPM is also addressed in [?]
to solve production-distribution problem. In the prescrit chapter, we impie-
ment a new two-level decomposition method based on the ACCPM to solve
large-scale capacity planning and technology acquisition problems imder un-
certainty. The resulting algorithm is then compared against the two-level
shnplex-based classical colunin generation technique.
The decomposition methods that are used for solving stochastic programs
eau be classified into three categories: i) direct methods that solve the de-
terministic equivalent to a stochastic program by using a simplex or interior
point algorithm [8, 5, 15], ii) cutting plane algorithms that are mostly based
on the Benders' decomposition of the primai formulation [3, 72, 79], a.nd iii)
Lagrangian based, dual decomposition methods [63, 64]. Cutting plane al-
gorithms such as the L-shaped method are initially designed and utilized for
solving the two-stage stochastic program and cari extend to multi-stage as
the nested decomposition method [6, 29]. The nested decomposition method
is a simplex-based decomposition algorithm.
hi the present chapter, we consider a two-level interior-point-based de-
composition algorithm that consists of two master programs and a subprob-
A Two-Level friterior-Point Decornposition Algorithrn for Multi-Stage Stochastie Capacity Planning and Technology Acquisition 87
lem. Both master programs have Dantzig-Wolfe reformulations that are
solved by the ACCPM to provide central dual puces. Note that the combina-
don of the second n'aster program and its subproblem acts as a subproblem
for the first master. Alternatively, oui' problem is also decomposed by using
the classical Dantzig-Wolfe reformulation and column generation technique
where both master prograrns are solved by the simplex algorithm for the
sake of comparison. In both decomposition approaches, the subproblem in
the second le.vel is always solved by the simplex method. In the classical
Dantzig-Wolfe decomposition approach, colunms are generated by using the
dual extreme points of the restricted master program, whereas the two-level
ACCPM takes advantage of using the central dual prices, i.e., the dual cen-
tral points that lie in the relative interior of the dual feasible region. This
strategy introduces a different venue in approaching the column generation
scheme. Note that the recent developments in classical Dantzig-Wolfe de-
composition and column generation focus on regularization and stabilization
techniques [61] to provide better dual prices.
Furthermore, our implementation is a serious step towards the efficient use
of the interior point method within the framework of the multi-stage stochas-
tic program. Previous efforts [5, 151 are limited to utilizing the interior-point
method in solving the deterministic equivalent to a mufti-stage stochastic
program. However, our algorithm uses an interior-point method in a cut-
ting plane (column generation) context radier than a direct solution to the
deterministic equivalent problem.
The present chapter is organized as follows. In Section 2, we provide the
formulation of a multi-stage technology acquisition and capacity expansion
A Two-Level Interior-Point Decornposition Algorithin for Multi-Stage Stochastic Capacity Planning and Technology Acquisition 88
model. In Section 3, we address a two-level decomposition approach that
leads to the column generation technique. ln particular, we introduce the
notions of the restricted n'aster problem and the subproblem in cadi level. In
Section 4, we deal with solving the restricted master programs by using the
ACCPM, and present â pseudo-code for the two-level ACCPM. In Section
5, we discuss the computational experiments; and finally, in Section 6, we
present our concluding remarks.
4.2 The Technology Acquisition and Capac-
ity Expansion Mode!
The technology acquisition and capacity expansion model cari be viewed as
a single plant model with multiple production capacity and a capability for
inventory holding. The goal here is to maximize profits in mufti periods of
uncertain product demand. The multi-stage stochastic program consists of
T planning periods (stages) with stochastic demand and a stage zero that is
characterized by a deterministic demand. The indices, variables, and input
parameters used in the model are denoted as follows:
Indices: N: number of products. T: number of stages (periods) in the planning horizon. Kt: number of scenarios in planning stage t, t = 1, • ,T T. cif(kt ): inventory index for scenario kt.
A Two-Level Interior-Point Decomposition Algorithm for Multi-Stage Stochastic Capacity Planning and Technology Acquisition 89
Variables: Xi0:
X iikt:
Iitkt :
mnount of product j manufactured in stage 0, i = 1, • • • N. amount of product i manufactured in stage t under scenario kt , i = 1 ; • • • N; t =1 ; • • • ,T, kt = 1 ; • • - Kt . inventory of product i at the end of stage 0, j = 1, • - . N. inventory of product i at the end of stage t Linder scenario k t ,
= 1, • • • , N. t = 1, • • • , T, kt = 1, • • • , Kt . unsatisfied demand for product j in stage 0, i = 1, - • • , N.. unsatisfied demand for product j in stage t under scenario
= 1, • • • , N, t = 1,• • • ,T, 1,- • • Kt . additional capacity at the beginning of stage t for product
Yilkt :
Uu : t = 1. • • • , T.
Parameters: Uo:
Pukt:
hitkt
cdukt :
Ptkt:
initial capacity at stage O. marginal profit or price for product i at stage 0, i = 1, • • • , N. marginal profit or price for product i at stage t under scenario kt; j = 1, • • • , N. t = 1 ; • • • ,T, k t = 1, • • • Kt . unit holding cost for product i at stage 0, i = 1, • • • N.. unit holding cost for product i at stage t under scenario =1 ; • • • ; N. t =1,• • • ,Tkt =1.• • • ; Kt .
unit penalty cost for flot meeting the demand for product i at stage t under scenario k 1 . i =1 ; • • • ; N, t =1,• • • ; T. kt = 1 ; • • • , probability of scenario kt in stage t ; t = 1 ; • • ,T, kt 1 ; • • • ,K.
Given the above definitions, a general T-stage model eau be formulated
A Two-Level Interior-Point Decornposition Algorithm for Multi-Stage Stochas tic Capacity Planning and Technology Acquisition 90
as follows:
[TS111] : max E(pio X
h,i0 — Cdio Yio) —
N T
cit i=1 i=1 t=1
N T Kt N T Kt
Ptk .Pitk X ik — Ptk hitk I itk i=1 t=1 k=1 N T Kt
— E E E Ptk CditkYitk i=1 t=1 k=1
i=1 t=1 k=1
s. t. (4.1) i=1
Xj0 < = (4.2)
E Xtk < uo + N T
t = • ,T, kt = 1, i=1 t=1
+ Xi1k1 lilki <
iita(kt ) XitkÉ 1 itkt < ditkt ,
liTkT =
X.Y,I,U > 0,
(4.3)
i = 1 • • Y, k1 = 1. • •
(4.4)
i = 1,••• IV, t = 1, • • T.
a(kt) = 1, • • • Kt _ i , (4.5)
kT = 1, ••• KT , (4.6)
where = EiN__ 1 (dio — 40 ) and 1/itk, — Xitkt )• In = EiN-1 ET -1 Eikc-ti(ditkt
[TMS], constraint (4.1) refers to the capacity constraints in stage 0, while
constraints (4.2) specify that the demand must be satisfied as much as possi-
A Two-Level Interior-Point Decornposition Algorithm for Multi-Stage Stochastic Capacity Planning and Technology Acquisition 91
hie in stage O. In stages t = 1, T, constraints (4.3) represent the capacity
restriction. Constraints (4.4) force the demand to be satisfied in stage 1, and
constraint (4.5) forces the demand to be satisfiecl in the rest of the stages.
Constraints (4.6) insure the inventory to be zero at the end of the planning
horizon.
4.3 A Two-level Decomposition Algorithm
The solving of multi-stage stochastic programming problems requires efficient
algorithms, since the resulting MSP problems are extremely large scale pro-
grains. Software packages built on simplex or interior point algorithms eau
solve instances of stochastic programming problems efficiently if their special
structure is fully exploited. However, the exploitation of this structure is not
always evident in stochastic programming. The recourse structure in [TMS]
formulation enables programmers to take advantage of the dual block-angular
structure of the multi-stage stodiastic program to decompose it into two lev-
els. To demonstrate how the &composition is performed, without loss of
generality, we consider the following three-stage formulation of [TMS]. This
formulation consists of a deterininistic stage (stage 0) and two other stages
with stochastic variables, where the first stage has ni scenarios while the
second stage lias n = in 2 scenarios. In the three-stage inodel ; x is the vec-
tor of variables associated with stage 0, whereas e • • , yn1 ,1 and
Y2 = A, • • • , yri2 1 are the vectors of variables associated with the first,
stage and the second stage, respectively. The cost structure consists of the
direct cost e of stage 0, and the expected costs of the stochastic variables in
A Two-Level Interior-Point Decornposition Algorithrn for Mati-Stage Stochastic Capacity Planning and Technology Acquisition 92
the first stage and second stage denoted by (h and q9, respectively. Then,
the generic primai three-stage stochastic program can be formulate.d as:
[Primai] : max cT +(hYl. 1- (12
s.t . A a; = b
Tl x + W1 Y1 =h
nyi +14'2Y2 = h2
x, Y2 > O.
To case the notation, we renaine the right-hand sicle of the above model
as: b = = c„ and h2 = cp . Then, the generic dual problern is the
following:
[Dual] : min cT `-' ,
itt ,
7r "
si AT +TTR.
1,11Tir + Tr
where À. -A- and il are the dual vectors associated with stage zero, stage one,
and stage two, respectively.
The primal stochastic linear recourse program is a deterministic equiva-
lent program, and its constraint matrix lias a dual block-angular structure.
Furthermore, its dual constraints matrix lias a primai block-angular struc-
ture. Therefore, the primai program is amendable to an approk -ich by Ben-
ders' decomposition, whereas the dual program can be challenged by the
Dantzig-Wolfe reformulation and colunm generation techniques. These two
approaches are equivalent by duality.
A Two-Level Interior-Point Decomposition Algorithrn for Multi-Stage Stochas tic Capacity Planning and Technology Acquisition 93
In the present chapter, we look at the stochastic dual program and de-
compose it along the scenarios. Hence, by utilizing the Dantzig-Wolfe re-
formulation, the ACCPM and column generation can solve the three-stage
problem in two levels by way of the following procedure. In the first level, the
&composition is composed of a master program that is solved by the AC-
CPM, and subproblems. Each subproblem &composes into a second-level
master program that is also solved by the ACCPM, and subproblems that,
are solved by the simplex method. Figure 4.1 illustrates the two-level decom-
position along the scenarios for a three-stage stochastic program. The loft
side of Figure 4.1 illustrates the first-level partitioning, whereas the right sicle
shows how a subproblem in the second stage is decomposed into a n'aster
program and its subproblems.
Figure 4.1: Two-level &composition along the scenarios.
To observe how the two-level &composition is donc on the dual program,
keep the first, constraint, AT A + /Tri- < c, in the master (as the complicating
constraint). Thon, assmning that the subproblem [SPI.] is bounded, the
(A, n, p.) cari be expressed as a convex combination of the extreme points
A Two-Level Interior-Point Decomposition Algorithm for Multi-Stage Stochastic Capacity Planning and Technology Acquisition 94
( )1/” 7rp: Pp); P e P:
( A l 7r 711) = 1 7TPJLP) 6P;
t9„ = 1.
Op > 0, p E
Therefore, the master problem at the first level becomes:
[Il Pl : min 4/P = (E, Ap op) + c(Ep 7rp Op) + (Ep tep Op)
s. t. AT („ AP eP) + (Ep 9P) < c, Ep Op -= 1,
Op > 0, 1) e P.
Now in [MM, let a l be the vector of the dual variables on the first set of
constraints and ao be the dual variable associated with the convexity row.
Thus. the subproblem at the first level can be written as:
[SPI] : miii z pi = [cr — A cl i ]À +[c7; — T1 ctilz
s.t. WT r
This subproblem is a linear program that also can be decomposed into a
master program and its subproblems. Hence, the natural linking constraints
would be the first set in [SPI.] that links variables Tc and p. This linking struc-
ture remains in the second master program. Assmning that the subproblem
A Two-Level Interior-Point Decornposition Algorithra for Multi-Stage Stochastic Capacity Planning and Technology Acquisition 95
[S1)11] is bounded, the (A, 7, ft) cari be expressed as a convex combination of
the extreme points (A„,7F q , itg ), q e Q:
(A, Ti;. ( [tg ) 0,1 ,
> 0q = 1,
0,1 > 0, e Q.
Hence, the second master program appears as:
: = min [cr — Aai ] (Eq Ag o-q ) [eTr — T1 a1 ] (E 7i Gr )
-I-C (Eq itg 0-q )
S.t . i4/1. (Eq q 0-q ) (Eq fig o-q ) qi ,
= 1,
0 q >O , q E Q.
Similar to what vas clone above, let a 2 be the vector of dual variables on
the first set of constraints in [MP1] and a, be the dual variable associated
with the convexity row. Thus, the second subproblem appears as:
[SP11] : min = — Aad À + {e,T — T1 ai — Wiaz] ir
+ [et,. — T2 az] [L — aa
s. t. Wr IL < q2.
Note that the Dantzig-Wolfe reformulation is donc in the first and sec-
ond master programs. The second master program is a subproblem for the
A Two-Level Interior-Point Decomposition Algorithm for Multi-Stage Stochastic Capacity Planning and Technology Acquisition 96
first master, providing columns to be dynamically added to the first n'aster
teogram. Moreover, the second subproblem provides columns to be added
to the second master.
4.4 ACCPM
The analytic center cutting plane method (ACCPM) [33] combinas the inte-
rior point algorithm with cutting plane techniques to provide a powerful tool
for solving the large-scale optimization problems in general, and stochastic
programs in particular. The ACCPM bas a superior global convergence prop-
erty and is amendable to decomposition methods. In solving [TSM], we take
advantage of the fact that the convergence rate of the ACCPM is insensi-
tive to the problem scale[38]. This unique property allows us to solve luger
problems while using almost the same number of iterations as the smaller
ones.
As explained in the previous section, we design a two-level decomposition
algorithm for solving a class of multi-stage stochastic programs. Using this
algorithm, WC solve the master problems, namely [MP] and [MP1] with the
ACCPM. The following section describes how the ACCPM is implemented
in the master programs.
4.4.1 Solving the Master Problems
The decomposition principle along with the dynamic colunm generation tech-
nique imposed on the dual of a stochastic program works with a small subset
of columns of the full master programs in [MP] or [MP1]. This subset of
A Two-Level Intcrior-Point Decornposition Algorithin for Multi-Stage Stochas tic Capacity Planning and Technology Acquisition 97
columns forms the restricted master program that in general format eau be
represented as:
ZR* Afp = min{c,Ty • ATy < _
where Ar and cr represent the subset of columns of the full n'aster and their
associated cost vector, respectively. Index r is a particular iteration at the
master program level.
At a given iteration. ACCPM solves a restricted master problem -namely
[Mei associated with [MM, and [RMP11 associated with [MP1]--- to send
the query points (central dual prices) to the oracle (subproblem). hi addi-
tion, the solution of the restricted n'aster program provides a feasible primal
solution to the full master, winch updates the upper bound,, in each level,
for example,271 = mint.7; 1 , zl for the first level or 7-,2 = mire2 , Z *Rdwpd
for the second level program. In return, the oracles [SPI. and [SP 11] generate
columns to be &Med to their corresponding n'aster program. Tire stopping
criteria imposed on the problem measures the solutions of the restricted mas-
ter program against that of the full n'aster at optimality. This means that the
algorithm stops iterating when the restricted n'aster program demonstrates a
solution that is nearly optimal to the full n'aster program. Moreover, the AC-
CPM uses a column generation technique to localize tire polyhedron within
the current relaxed feasible region related to [MM or [MP1], in which their
optimal solutions exist. Such polyhedrons are called localization sets. Based
on that we deline the localization set as:
y > z,A,T. y <
A Two-Level Interior-Point Decomposition Algorithrn for Multi-Stage Stochastic Capacity Planning and Technology Acquisition 98
where z is a lower bound provided by the oracle to the master program, and
is denoted by
z i = max{ z i , +
for the first master program [MM and
2:2 = 11{ 2 , z2 + 4P11}
for the second master program [MP1].
Optiinizing a log-barrier function over the localization set provides an
analytic center and the corresponding dual value. This dual value is sent
to the oracle to generate column Br and its associated cost value dr , winch
in turn is added to the n'aster problem. This action updates the constraint
matrix and the cost vector in the restricted n'aster problem as follows:
4r+1 = [Ar Br]; cr+1 = [cr- dr].
The algorithm terminates when the difference between the lower and upper
bound falls below a predetermined threshold. The basic steps of a generic
ACCPM algorithm eau be summarized as follows:
Algorithm 4.1: ACCPM
Initialization
A feasible point y° and its corresponding lower and upper bounds z, >.=5.
The initial set of localization
Main iterations While lz — >
A Two-Level 1liter-1:or-Point Decomposition Algorithrn for Ilitlti-Stage Stochastic Capacity Planning and Technology Acquisition 99
1. Compute the analytic center of L. say y, and its corresponding dual solution x.
2. Cali the oracle at point x and update the lower and lippe' bounds z,
3. Update the localization set
End—while
Unlike many decomposition methods, the restricted master program in
ACCPM is not solved to optimality. Instead, it seeks the anal tic center
of the localization set as a starting point for the next iteration. Then, tins
analytic center passes forward to the oracle to generate new columns. You eau
calculate an analytical center by the primai Newton method. dual Newton
method, or primai-dual Newton method. In our implementation, we use the
primal Newton method, since no guarantee exists that the dual feasibility cari
be recovered after cutting a portion of the dual space. Note that ACCPM
needs a feasible point to start with at each iteration, and only a primai
feasible point can Le recovered perfectly after adding columns [36, 56].
4.4.2 The Two-level ACCPM
The two-level decomposition method is implemented to solve the multi-stage
stochastic program in the following mariner: ACCPM solves the first stage
and passes forward the current central solution to the later stages, while infor-
mation passes backward in the form of colunms to be added to the ancestral
problems associated with the descendent stages. This can be summarized by
the following pseudo-code.
A Two-Level Interior-Point Decomposition Algorithm for Multi-Stage Stochas tic Capacity Planning and Technology Acquisition 100
Algorithm 4.2: Two-level Column Generation by ACCPM
mit ialization
Starting primal feasible points are y? and A for the first and second n'aster programs. Their corresponding lower and upper bounds are fi and z2 . z2 .
The initial sets of localization for the first and second masters are characterized by = {y]. : b1 Vi > < £2,= {y2 : b2 P2 > Z2 , A2 y2 < c2} where A = A2 = [I, — Vectors are
= (1 and c2 = (12 , u2 )T , where 1 1 , u 1 . 12 , u2 provide bounds on variables yi and y2 for the first and the second masters.
Main iterations While — > e
1.1 Compute the analytic center of say pal , and its corresponding dual solution ra i .
While 1f2 — z2I > f 9.1 Compute the analytic center of .C 22 , say ya2 , and its corresponding
dual solution xa2 ; update the lower bound
9.2 Call the subproblem [SP1 1] at point (ra i , xa2 ). It returns column Bq and its associated cost vector (1q. Update the lower bound Z 2 .
2.3 Update the localization set £ 22 by updating the coefficient matrix = .897 and el dqr•
2.4 Update matrix Dq+ 1 = [Dg Bq], which consists of ail extreme points of subproblem [SP1 1] generated so far.
2.5 -Update the upper bound by using i = min{21 , c2 ya2}.
End—while
1.2 Compute the convex combination of colmnns gathered in D by using the analytic center ya,2 , and generate the column BP and its associated cost rP.
A Two-Level hiterior-Point Decomposition Algorithin for Multi-Stage Stochastic Capacity Planning and Technology Acquisition 101
1.3 Update L by updating the coefficient matrix Ari = BPI' and (,p-Fr ,491T . -1 '
1.4 Update the upper bound Z2 by using Z2 = Min{ yai }.
End-while
Note that in the above algorithm ; we need the convex combination of
ail columns that are generated in the second loop to be passed forward as a
new column to the first loop (the first master problem). Therefore, in 2.4 we
design matrix D to gather ail the columns that are generated so far in the
second loop. In the implementation of Dantzig-Wolfe, each thne we enter the
second loop, we keep the previously generated columns, whereas in ACCPM
matrix D is reset to the empty matrix. In the above pseudo-code. the convex
combination of the colunms that is gathered in D is denoted by B.
4.5 Computation.al Experiments
For the computational experiments, WC implemented the two-level decompo-
sition algorithm on the capacity expansion and technology acquisition model
[TSM]. The code is written in MATLAB version 7.1 for both the two-level
ACCPM and the two-level Dantzig-Wolfe. In the Dantzig-Wolfe decompo-
sition approach, the two linear relaxation of the restricted n'aster problems
and the subproblem are solved using LINPROG, the build-in library within
MATLAB 7.1. Moreover ; in the two-level ACCPM ; the two linear relaxation
of the restricted master problems are solved with an in-house version of AC-
CPM (with a duality gai:) set at 10 -g ), and the subproblem is solved 1 -.)y using
LINPROG. [TSM] proposes a multi-stage stochastic program that models
A Two-Level Interior-Point Decomposition Algorithm for Multi-Stage Stochastic Capacity Planning and Technology Acquisition 102
demand through a SCellari0 tree and can accommodate any discrete demand
distribution with any number of products. We explored this possibility to
create different joint probability distributions of two products demand, so
to be able to compare the two-level ACCPM against the two-level Dantzig-
Wolfe in different dimensions. For example, in the first row of the Table 1,
2-Demand means that we have 2 types of forecasted demand D8), with
probability Vs. s = 1,2. Silice we consider two products, it generates 4 possi-
ble scenarios in the first stage and 16 scenarios in the second stage. Similarly,
the scenarios can be constructed through 3-Demand, 4-Demand, 5-Demand,
and 8-Demand, processes that generate 81, 256, 625, and 4096 scenarios
respectively.
The decomposition and partitioning procedure is (loue on the dual of
[TSM] to take advantage of decomposing the problem along the stages. In
ternis of the number of variables in the dual problem, we solved a range
of problems from 63 variables to 12 483 variables, whereas the number of
constraints ranged from 54 to 8 454. The MATLAB limitations did not allow
us to make the constraint matrix beyond 12 483 x 8 454, which corresponds
to the 8-Demand model. The features of the test problems are presented in
the Table 1:
An inner iteration of both ACCPM and Dantzig-Wolfe (DW) corresponds
to an inner loop of the program in which the second n'aster problem and the
subproblem interact. Alternatively, an outer iteration corresponds to an
outer loop where the first master progTam interacts with the information
received from the inner loop. As is presented in Table 1, the nurnber of outer
iterations in bath ACCPM and DW are comparable for the aforementioned
A Two-Level Interior-Point Decomposition Alqorithm for MUlti-Stage Stochastic Capacity Planning and Technology Acquisition 103
Table 4.1: Computational R.esults
Number of Scenarios
Primai var.
Dual var.
ACCPM DW Inner Outer muer Outer
2-Demand 24 = 16 :54 63 30 5 95 5 3-Demand 34 = 81 204 273 98 4 348 6 4-Demancl 44 = 256 582 819 15 3 709 5 5-Demand 54 = 625 1356 1953 15 3 1939 4 8-Demand 84 = 4096 8454 12483 26 3 7262 3
problem sizes. However ; the real difference is in ternis of the nurnber of inner
iterations, where ACCPM performed much better, especially for the luger
problems. For the 2-Demand problem, the number of outer iterations is the
saine, but less muer iterations occur for the Dantzig-Wolfe decomposition
method. For the last four problems ; the nimber of inner iterations is much
larger for the Dantzig-Wolfe method as compared to ACCPM ; that is, from
10 to almost 300 times.
The first four models in this table were solved on a DELL workstation,
whereas the last model vas submitted to Sun Grid due to the large dimen-
sionality of the problem. Although we did the computing experiments on two
different machines ; in both cases ; ACCPM performed inuch faster in ternis
of the CPU thne. ACCPM solved ail problems in a few minutes. Except
for the first problem, winch the Dantzig-Wolfe approach a,lso solved in a few
minutes ; it took this method hours to solve the rest of the problems.
A Two-Level Interior-Point Decornposition Algorithin for M'alti-Stage Stochas tic Capacity Planning and Technology Acquisition 104
4.6 Conclusion
The improvement of manufacturing flexibility is becoming an increasingly
important method to achieve a competitive advantage in the market place.
Virais are seeking new ways to integrate flexibility in their manufacturing
process to flot only survive in an uncertain environment but to create core
competencies that are absent in their competitors strategies.
Researchers have explored many aspects of flexibilities; however, the
prescrit chapter focuses on capacity, production, and volume flexibility in an
environment that is characterized by variable product demand and multiple
production. We use the multi-stage stochastic program to capture the evolu-
tion of product demand over Urne. Since the multi-stage stochastie program
is a large-scale optimization problem, and in many cases not tractable by
standard optimization packages, we developed a two-level decomposition al-
gorithm combined with a dynamic colurrm generation approach to reduce the
dimensiomdity of the problem at cadi level. The resulting algorithm is irn-
j)lenientecl on the standard Dantzig-Wolfe decomposition method and tuât of
ACCPM. Computational experhnents show that, the two-level interior-point-
based method ACCPM solves the problems in a small number of iterations
as compared to that of the simplex-based Dantzig-Wolfe method.
Chapter 5
Solving Multi-Stage Stochastic,
In-house Production and
Outsourcing Planning by
Two-level Decomposition
5.1 Introduction
In the past few decades, manufacturers have paid increasing attention to out-
sourcing products and processes. Traditionally, outsourcing by organisations
vas clone to reduce the marginal production cost and to exploite the exter-
nal supplier's capabilities. However, in many cases a number of important
factors such as product quality, lead tulle rehability. technical capability ; and
the financial sta,bility of the supplier were flot taken into accolait. Many com-
parlies decided to outsource rather than make other potential decisions based
Solving Multi-Stage Stocho„9tic, In-house Production and Outsourcing Planning by Two-level Decornposition 106
on a short terni analysis of cost reduction and capacity [12]. This approach
may have resulted in high contingency costs for corporations. For example, a
recent massive toy recall cost Mattel nearly 30 million U.S. dollars [39]. The
recall was made alter Chinese-manufactured toys were found to have paint
with excessive amounts of lead or small parts that could be swallowed by
children.
In 1988, after comprehensive research vas carried out by McKinsey and
Company [49], Mattel started a Vendor Operation Asia (VO.A). The research
suggests that Matte' manufactures the core products like Barbie and Hot
Wheels in-house and outsources ail non-core products. By using this strat-
egy, Mattel eau produce a large number of toys with short life-cycles without
any intensive investment. Over recent years, Mattel has introduced over 300
new, non-core toys each year. The vendors are working with Mattel un-
der an agreement called Original Equipment Manufacturing (OEM) [40] by
which l\lattel transfers the design and knowledge to vendors in China, and
sells the final products under its own brand naine. The recall of millions of
toys by such a giant corporation, as well as recalls on other unsafe products-
including tainted pet food, seafood for human consumption, and toothpaste-
indicate that quality failure is a large risk assoeiated with outsourcing to
overseas suppliers, especially when the vendors have no stake in the brand
naine. Moreover, product safety standards can be improved as the existing
infrastructure improves in a country, since what is considered to be unsafe by
Western standards may be perceived as sale in developing countries. There-
fore, Western importers should benchmark a safety standard and monitor
the quality during the manufacturing process. By doing Ulis, a corporation
Solving Multi-Stage Stochustic, In-hou,se Production and Outsourcing Planning by Two-level Decornposition 107
like Mattel could have stopped or improved the manufacturing process before
20 million unsafe toys were introduced into the market. However, the extra
cost associated with quality assurance may prevent many corporation from
participating in a consistent quality monitoring.
Considering these challenging issues, in recent years, most manufacturers
have addressed the question of how extensively they should outsource, and
at what point should they stop? How cari manufacturers assess the risks of
making or buying to best leverage their resources for increased profitability?
What is the optimal combination of make and bu?
To help to answer these questions, we developed a multi-stage stochastic
programming model which takes into account the uncertainty involved in
the production of the quality of outsourced products in the face of stochastic
demand. The goal is to find the optimal way to choose between in-house
capa.city expansion and buying from the local suppliers with assured quality
versus buying from overseas suppliers.
Mufti-stage stochastic programs (MSPs) are large-scale optimization prob-
lems. The size of MSPs constraint matrix increases exponentially with the
number of stages and the 'nimber of scenarios. Hence, soine MSPs are in-
tractable by direct optimization solvers, and decomposition methods become
good alternatives for solving them. However, in classical decomposition ap-
proaches, the number of iterations eau become unmanageable, especially as
the dimension of the n'aster problem increases. To resolve this issue, we pro-
posed in [62] a two-level interior point decomposition method for solving
class of multi-stage stochastic programs with recourse. This two-level decom-
position method uses dynamic column generation and consists of the solution
Solving Multi-Stage Stochastic, In-house Production and Outsourcing Planning by Two-level Decomposition 108
of the two master programs and a subproblem. Both master programs have
Dantzig-Wolfe reformulations that are solved by an interior point-based de-
composition algorithm, such as the Analytic Center Cutting Plane Method
(ACCPM) (sec [33]) to provide central dual prices. The coinbination of the
second n'aster program and its subproblem acts as a subproblem for the first
n'aster. The ACCPM has a superior global convergence property, and its
convergence rate is insensitive to the problem scale. These unique properties
allows us to solve large-scale MSPs in a manageable number of iterations.
In this chapter, we take adwmtage of the recourse structure of the in-house
production and outsourcing planning mode! to &compose and solve it with a
state-of-the-art two-level decomposition algorithm. We extend the two-level
decomposition via the ACCPM to Benders' decomposition for the first time
so to be able to compare the computational results with the two-level column
generation that uses the ACCPM. Moreover, to achieve a higher accuracy,
we implemented a two-level decomposition algorithin where the first level is
solved via the classical Dantzig-Wolfe &composition and the second level is
solved via the ACCPM.
The present chapter is organized as follows. In Section 2, we formulate
the in-house production and outsoursing planning problem. In Section 3, we
explain the approaches to solving the problem, that is, a two-level column
generation and a two-level Benders' decomposition, and an interior point-
based cutting plane algorithm — the ACCPM. In Section 5, we discuss the
computational experiments. Finally, in Section 6, we present oui concluding
remarks.
Solving Multi-Stage Stochastic, In-house Production and Outsourcing Planning by Two-level Decomposition 109
5.2 In-House Production/Outsourcing Plan-
ning Mode!
As product demand increases, corporations face decisions about where and
how to expand their production capacities. Many business organizations
consider outsourcing their products to the emerging, low-cost overseas mar-
ket. Outsourcing allows corporations to meet customer demand at lower
marginal costs, while also maintaining flexibility of °pendions in changing
environments. However, the real evaluation of the cost-savings associated
with outsourcing requires more than a mere consideration of the production
cost differences. Many manufacturers face a 'nimber of costly, outsourcing
challenges including lead times, quality, and communication. To capture the
real cost associated with outsourcing, we use the scenario tree approach to
model the evolution of the uncertain demand and risk associated with the
quality of outsourced products in multiple periods. Elements of the model
are illustrated in Figure 1.
This figure shows that in each stage the decision ma,kers face two alterna-
tive paradigms. The first, paradigm, which is illustrated in the upper part of
the decision tree, is concerned -with buying products of uncertain quality from
overseas suppliers. This eau be explained if, for example, we assume three
different categories associated with quality — namely, E (Excellent quality),
A (Acceptable quality), and B (Bad quality). Shnilarly, we assume three
other categories for demand — namely, H (High), M (Medium), and L (Low).
Therefore, the first scenario EH in the upper part of the tree shows the pos-
sibility of having excellent quality and high demand. On the other hand, in
Solving Multi-Stage Stochastic, In-house Production and Outsourcing Planning by Two-level Decomposition 110
Figure 5.1: In-house production and outsourcing planning, decision tree
the lower part of the tree, for manufacturers facing the decision of whether to
expand in-house capacity or to buy from local suppliers with assured quality
and higher cost, the only stochastic parameter is the demand. The goal of
the proposed model is to find the optimal way to choose between in-house
production, buying from local suppliers with higher cost, or buying from low-
cost overseas suppliers while taking into account the risk associated with the
quality level. In this chapter, we refer to the upper scenarios as the first set
of scenarios and to those in the lower part as the second set of scenarios.
The first set lias two parts starting from stage 2 as illustrated in Figure
5.1; the number of scenarios in the upper part is denoted by Kt and the
number in the lower part it is denoted by J. Similarly, the 'nimber of
scenarios in the lower part is denoted across the planning horizon t.
Solving Multi-Stage Stochastic, In-hanse Production and Outsourcing Plan,ning by Two-level Decomposition 111
The indices, variables, and input parameters used in the model are as
follows:
Indices: T: number of stages (periods) in the planning horizon Kt : number of first set scenarios (concerned with outsource plan-
ning) in stage t. t =1, • • • ,T Jî: number of first set scenarios (concerned with buying from
local suppliers) in stage t, t = 2. • • • . T Jt : number of second set of scenarios in stage t, t = 1. • • • ,T oi(kt ): capacity index for scenario
capacity index for scenario j t clUn: capa.city index for scenario
X0:
tke:
Yo: Yti?:
1/ki t :
Ztkt:
Wti?:
Vti?:
Vi • •
Utkt :
U .2. t../t •
Utit :
Solving Multi-Stage Stochastic, In-house Production and Outsourcing Planning by Tivo-level Decomposition 112
Variables: amount of product winch is manufactured in stage 0, amount of product which is manufactured in stage t,under scenario k1 . t =1. • • ,T, 1c1 = 1, • • amount of product bought from local suppliers in stage 0, amount of product bought from local suppliers in stage t under scenario jt2 , t = 1, • • • T, jt2 = 2, • • • Jt2 amount of procluct bought from local suppliers in stage t under scenario j t , t = 1, • • T, j = 1, • , jt amount of product which is outsom•ced at stage t under scenario kt , t =1,• • ,T, kt = 1, • • , Kt the amount of product which is manufactured in stage t, under scenario kt , t = T. kt = 1, • • ; Kt the amount of product which is manufactured in stage t, under scenario j. t = 1, • • • ,T, jï =1,• • • amollie of product bought from local suppliers at stage t and for scenario jî, t = 1, • • T, jt2 = • • ft2 amount of product bought from local suppliers at stage t ami for scenario jt , t = 1, • • • T, t= 1. • • • Jt additional capacity at the begirming of stage 0 additional capacity at the beginning of stage t, 'Eider scenario
t =1, • • ,T, kt = 1,- • • , additional capacity at the beginning of stage t, under scenario
t = 1 , • • • , T, = 1, • - • , additional capacity at the beginning of stage t, under scenario it2 , t = 1 , • • • T, it = 1 , • • • , Jt.
Solving Multi-Stage Stochastic, In-house Production and Outsourcing Planning by Two-level Decomposition
Parameters:
Uo: initial capacity at stage 0
Co: marginal cost of production at stage 0 etkt: marginal cost of production at stage t -ander scenario k1 , t =
1,• • • ,T, kt =1,- , Kt Cdo: unit buying cost from local suppliers at stage 0 Stkt : unit cost for outsourcing at stage t, under scenario kt
ePtkt : unit cost of in-house capacity expansion at stage t, under
scenario k1 , t = 1, • • • ; 71 , = 1, • • Kt CPti?: unit cost of in-bouse capacity expansion at stage t, under
scenario j?, t 1, • • , T, j? = 1. • • • cpti,: unit cost of in-bouse capaeity expansion at stage t, under
scenario Lt, t = 1, • - • T. Lit =1,- • unit production cost at stage t, ululer scenario j?, t = 1, • • • 7", = 1, • • •
atit : unit production cost at stage t, under scenario jt , t =1,- •• ,T, :ft = 1,- • •
b. 2 •
unit buying cost from local suppliers at stage t, under scenario t
1, • • • ,T, ft2 = 1, • ft2
btit: unit. buying cost from local suppliers at stage t,
under scenario Lit, t = 1,• • ,T, Lit =-1,• • „It dtkt : unit demand at stage t, under scenario kt , t = 1, • • • T, kt =
1, • • • , Kt dti?: unit dernand at stage t, under scenario t t =1,- • • ,T,
1, • , dtit : -unit demand at stage t, under scenario Lit, t = 1, • • • T, jt =
1, • , ,It Ptkt: probability of scenario kt in stage t, t = 1, • ••,T, kt =
1, • • , Kt Ptj;2 probability of scenario .9 in stage t, t = 1, • • , T, =
1, • • • P. probability of scenario Lit in stage t, t = 1, • • • T, Lit =
1, ••• , Jt
Solving Multi-Stage Stochastic, In-house Production and Outsouming Planning by Two-level Decomposition 114
Given the above definitions, the general T-stage model can be written as
follows:
[TA1B] : max (c0 X0 + cd 0 Y0) + T Kt
E c.Ptk Utk t=1 k=1
ePti rjtj
T
T Kt
cPti 2 Utj 2 +
Ptkt ( Cikt tkt Stkt Ztkt) t=1 j2 =1
T
E E Pti? t=1 .•2_1 - T Jt
t=1 k=1
(ati? Vti? + bti? Vii?)
Pti, (atit + hti, Vtj t )
Solving Multi-Stag e Stochastic, Iii-hanse Production and Outsourcing Planning by Two-level &composition 115
s. t. X0 < rio + U0, (5.1)
.X0+ Yo > do , (5.2) T cx(Kt )
Xtkt 5_ U 0 + Uo + E ut„,k, ) , ck(À:t) = 1. • • • , Kt, t=1 k=1
T ct(it)
1 4, <Po + + tet(jt), Cit ) = 1, • ,
t=1 k=1
i'Vt3? 5_ 1 o E t=2
X tk, Ztk t > dtkt ,
Wij? Vti? ?_ dt
Wt.it >
X, Z, U, W, > O.
T
(5.4)
Ute,u2), a(e) =- 1, • •
k=1
(5.5)
= 1, A 1 , t = 1, T
(5.6)
jt = 1„ • • . t = 2, ..,T 2
(5.7)
• Jt . t = 1, ..,T
(5.8)
(5.9)
In [TM13], constraint (5.1) refers to the capacity constraint in stage 0,
while constraint (5.2) specifies that the dernand must be satisfied in stage O.
Solving Multi-Stage Stochastic, In-ho use Production and Outsourcing Planning by Two-level Decornposition 116
In stages t = 1, ... T. constraint (5.3) represents the capa.city restriction on
production when a. manufacturer considers buying from overseas suppliers
at a lower price and risky quality. Constraint (5.4) represents the capacity
restriction on production when manufacturers also have the choice to buy
from local suppliers at a higher price ami assured quality. ln stages fi = 2, ...
T, constraints (5.5) refers to the capacity constraint when buying from local
suppliers (in the upper part of Figure 1). Constraints (5.6). (5.7) and (5.8)
force the demand to be satistled in both situations —namely, to outsource to
overseas suppliers or to buy from local suppliers.
5.3 Solution Approaches
Decomposition approaches become methods of choice for solving large-scale
MSP problems. They can reduce the dimensionality of the large-scale prob-
lems, since the y are amendable to a parallel computer environment. The
well-known nested decomposition method [6] for solving MSPs cari be impie-
mented via parallel computing, sec [65]. By parallelization, sets of scenarios
are assigned to processors to create cuts (columns) simultaneously, and thus,
problems are solved more quickly.
The simplex-based decomposition approaches are prevalently utilized for
solving MSP problems. However ; the use of an interior-point method is often
better choke for solving large-scale optimization problems. In [62]. we SUC-
cessfully developed a two-level interior-point-based &composition algorithm
for a class of MSP problems. In the present chapter, we use variants of the
two-level approach to solve the [TMB]. In addition, we implement a new
Solving Multi-Stage Stochastic, In-house Production and Outsourcing Planning by Two-level Decomposition 117
two-level decomposition based on the Benders decomposition of the [TMB].
This strategy allows US to propose the best approach for solving the in-hand
problem.
5.3.1 A Two-level Column Generation Approach
To best explain the Dantzig-Wolfe decomposition [54] and the column geler-
ation approach that is implemented on [TM13] in two levels , we formulate a
generic form of [TMB] in three stages. This formulation consists of a deter-
ministic stage (stage 0) and two other stages with stochastic variables, where
the first stage lias ru scenarios and the second stage lias Tt --= et' scenarios.
In this three-stage model, w is the vector of variables associated with stage
0 ; whereas yi = {iA ; W, • ; } and 1/2 ={y?, y.. ; y} are the vectors of
variables associated with the first, stage and the second stage, respectively.
The cost structure consists of the direct cost e of stage 0 ; and the expected
costs of the stochastic variables in the first and second stages, denoted by
and q2 , respectively. Then, the generic primai three-stage stochastic program
can be formulated as follows:
[Primat] : max eTx +qiyi +(h y2
=b
+ 13/4y1 = //1
+ nyi +1472 y2 = /12
Solving Adulti-Stage Stochastic, In-house Production and Outsourcing Planning by Two-level Decompositio 72, 118
To case the notation. we rename the right-hand side of the above model as:
b = cx, li = c.„ and h9 = c. Then, the associated generic dual problem is
formulated as the following:
[Dual] : max c7; À +C,TTC +CiDt
s.t. AT +TiTz +BT/t <
1/1"T7r +TT/c. < qi
< q2
where À. r arid /2 are the dual vectors associated with the primai constraints
of stages zero, one, and two, respectively.
The decomposition along the scenarios is donc on the dual program by
using the column generation technique and is explained in this section. More-
over, the decomposition along the stages in the scenario free, donc on the
primai program by utilizing Bender's cuits, is the topic for the next section.
Although these &compositions are equivalent by duality, they generate dif-
ferent computational results in terms of the number of iterations and the
precision of the algorithms.
To best exploit the structure of the dual program let us keep the first
dual constraint, A T À + TiTz + BT /.2 < c, in the first master as the compli-
cating constraint. Then, assuming that the subproblem [SPI.] is bounded,
the (À, ir . )1.0 cari be expressed as a convex combination of the extreme points
Solving 114ulti-Stage Stochastic, In-house Production and Outsourcing Planning by Two-level Decomposition 119
(4, 7rp, fip), P e P:
( À ' =
E op Op > O. p E P.
Therefore, the master problem at the first level becomes:
[111P] : min z*mp = (Ep Ap Op) ± CI; (II, 7Fp Op) + c(Ep PpOp )
s. t. AT (Ep Ap Op) ± (Ep '7p Op) ± BT (Ep P p Op) < C,
Ep Op = 1.
Op > O. P E P.
Now in [MP], let a i be the vector of the dual variables on the first set of
constraints and (19 be the dual variable associated with the convexity row.
'nus, the subproblem at the first level can be written as:
[SP1] : max z, 1 = — A a i jA + [cT — ai fr + [ct7; — B ai ] p — ao
S .t 1/17T TC
+r21 1-1 qi ,
< (12.
Assuming that the subproblem [S13 11} defined below is bounded, the vec-
tor (A,7r,p) can also be expressed as a convex combination of the extreme
Solving Multi-Stage Stochastic. In-house Production and Outsourcing Planning by Two-level Decomposition 120
points (Aq , Trq , [tg ) ; q G (2:
=
jj 0„ -= 1.
Oq > 0. (J E Q.
Hence, the solution of [S1)1] eau be replaced by an iterative process by using
a second n'aster program [MP1] and its column generator [SPI 1]:
[M Pl] : 'A1P1 MaX [cl Ct (E q q) [CT ad (Eq rcq o-q )
+ (ct,T — B ct ij (Eq aq )
s. t. (Eq zq o-q ) + T27 (Eq p q
Eq a,=1, a >0 g e Q.
—
<
Similar to what vas clone above, let a 2 be the vector of dual variables on the
first set of constraints in [MPI.] and a, be the dual variable associated with
the convexity row. Thus, the subproblem is given by:
[SP11] : max = — -4c/1iA + [c„T — T1 c — I'V1ct217
+ - T2 a2 — B a2 jp. —
s.t.
At any given iteration, bounds on the optimal value of the master pro-
Solving Multi-Stage Stocha,stic, In-house Production and Outsourcing Planning by Two-level Decomposition 121
gTams are updated. The lower bound on the first master program [IVIP] is
computed as:
Zmp = IrlaX{,?_;mp, -Z-mp Zs* pi} = MaX1,1„mp, -27mp ZMp l
while, for the second master program [MP1], it is given by
zmpi = max{_&mpi, 72 4- z*.spn
The master programs are initially defined on a small subset of columns of
the full n'aster programs, and the former are called restricted n'aster pro-
grams. We denote the restricted n'asters by [RMP] and [IIMP1] for the first
ami second master programs, respectively. The upper bounds are updated
by the current optimal solution to the restricted master programs and given
by 1vIP = Z*Riti p and 7111 Pi = 4111P1 . In each love!. the iterations termi-
nate when the difference between the upper and lower bounds falls below a
predetermined threshold. A pseudo-code for the proposed two-level column
generation approach is presented in Algorithm 5.1.
Algorithm 5.1: Two-level ColuTrif i Generation Algorithin
Initialization Feasible or artificial columns for [R1\413]; lower and upper bounds (z mp ,
Outer iterations While Ifmp — zmp i > Emp
1.1 Solve the restricted n'aster problem [RMP]; compute primal variables Op; p e P, and dual variables (ai, ae).
Solving Multi-Stage Stochastic, In-house Production and Outsourcing Planning by Two-level Decoraposition 122
Inner iterations Feasible or artificial columns for [RMP1]; lowers and upper bounds (z mpl , .z7mpi)•
While > emPi
2.1 Solve the restricted master problem [RMP1] at point (ai, a0); compute primai variables 61q . q E Q. and dual variables (a2, au).
2.2 Cal! subproblem [SPB11] at point (a l , a0.a2 , au ); generate extreme point (A g , 7rq , mq ), q E Q. and its associated cost
dq .
2.3 Update [RMP1] using extreme point (Ag; 7r q : itg ) of cost dq .
2.4 Update lower and lippe' . bounds (zmpi ,
End—while
1.2 Generate extreme point (4, ir , pp) = 7,, mg ) 61q , for p e P. the convex combination of columns in [R.MP1], and its cost rp .
1.3 Update [R,MI1 using extreme point (4. 7Tp, tip) of cost rp .
1.4 Update lower and upper bounds (2, mp , fiwp).
End—while
In the inner iterations, given the extreme point (A g . 71- flq ) , for q G Q,
the constraint matrix of R.MP1 is updated by adding a new column, 14/TIr q +
TI ft which is independent of Ag . The cost (.4 of this new column is computed
IS
de = [c7; Aa l'Ag + [C,T — T1 1 clip— ( f + T B
Similarly, in the outer iterations. given 7rp , pp ) = Eq (Aq , tiq ) 0q , for
p E P, the constraint matrix of R,MP is updated by aclding the generated
Solving Multi-Stage Stochastic, In-house Production and Outsourcing Planning by Two-level Decomposition 123
column AT Ap Inrp BT itp The cost rp of this colmnn is given by
T T P + Cr fi P -I- P . P
5.3.2 A Two-level Benders' Decomposition
The Benders' decomposition principle is based on a variable partitioning
procedure in a large-scale primai problem, sec [4]. The method partitions
the variables by fixing the so-called complicating variables. This reduces the
given problem to a more tractable, a:nd hence casier, problem to solve. The
latter problem is parameterized in ternis of the vector of the complicating
variables. The dual to this problem forms the subproblem. In each iteration,
the subproblem passes forward the current dual solution (for a fixed vector
of complicating variables) to a n'aster program in the form of a Benders' cut
that utilizes the linear programming duality to refine the value of the vector
of complicating variables.
In the particular case of MSP problems, the partitioning would be in terms
of the variables in cadi stage. Similar to what we implemented on the dual
problem when we used the column generation technique, we decompose the
generic primai three-stage stochastic problem in two levels by using Benders'
decomposition. This can be donc by first considering the vector of the first-
stage variables - namely yi = { • • , yn1 ,} as the complicating or fixed
Solving Multi-Stage Stocha„stic, In-house Production and Outsourcing Planning by Two-level Decomposition 124
variables. This forms the following subproblem, given that y i is fixed at
[SPB1(y1 )]: zs* pm(yi) = min cTx +q2y2
s.t. A x = cA
Tlx = c„. — 1471
B x ±W2 il2 = (111 T2Th.
Y2> O.
Solving [SPB1(11)] provides an upper bound on the original problem (ass-
suming that this subproblem is bounded). Therefore, the solution to the
original problem consists in solving the above subproblem for the best
value, and it is given by the solution of the following (outer) master prob-
lem:
PlIPB1 : min yi>0
,* --SPB1(Y1)'
Note that the dual feasibility of the subproblem is independent of the value
Th. Let A, 7r, and p. be the dual vectors associated with the primai constraints
of stages zero, one, and two, respeetively. Then, the dual constraints of
[SPB1(ïj 1 )] are:
AT 1-BT1t < c ;
14772 < q2 •
The dual solution of [SPB1(7j1 )1 is given by an extreme point (4, Irp, itp).
Solving Multi-Stage Stochastic, In-housc Production and Outsourcing Planning by Two-level Decomposition 125
p E P. for which the objective function takes value
Ap + (c7, — WlYiPrp + (cf, —112Y0iip + (1.171.
Therefore, the optimal solution to the subproblem eau be interpreted in ternis
of the best extreme point in P. that is:
* '''SPB1 (y-) = rn aXp e P Àp 11/1 )7rp + (c )fip + 1
min el s.t. el> c),Ap + (c7, — 1411 yi)7rp + (cp. — + giF1, P G P.
The outer master problem, which provides a solution to the original problem,
can now be written as follows in ternis of the extreme points in P and the
vector of complicating variables
P 131 : 211 p B = inji e
s. t. el> c), + — 14/1 )7rp + T27Milp P E P
> O.
To complete the two-level Benders' decomposition, note that problem
[SPB1(y1 )] eau ais° be solved by Benders' decomposition by taking the vec-
tor of the second-stage variables y 2 = { y?, A, • • • as the complicating
variables. Therefore, given that y 2 is fixed at. Tj2 , the second subproblem
Solving Mulfi-Stage Stochastic, In-house Production and Outsourcing Planning by Two-level Decomposition 126
appears as:
[SPB11(p1 , F2 )1 min T ---'s* PBmp,,p2 ) = (.T2.(.T2. + (12T/2
s. t. A x = na; =
— P1
B X = — T27/1 — 14/2
X >0.
Solving [SPB11(1h, y2 )] provides an upper bound on problem [SPB1(7/1 )]
(assuming that the second subproblem is bounded). Therefore, the solution
to the first subproblem given Th consists in solving the above second sub-
problem for the best y2 value, and it is given by the solution of the following
inner master problem:
[MPB1(Th )] : min Y2 >O 2"S* ,y2)"
The set of dual constraints for [SPB11(Th, Th) appears as:
AT +T;rn +/3T < c.
The dual solution of [SPB11(Th,F2 ) is given by an extreme point (Aq , 7rq , tg ),
E Q, for which the objective function takes value
c), Ag + (c7, — Wl Th ) 7fq + (cm — Teh — W2 Y2)/1q + qi Vi + q2
Therefore, the optimal solution to the second subproblem eau be interpreted
Solving Multi-Stage Stochastic, In-house Production and Outsourcing Planning by Two-level Decomposition 127
in terms of the best extreme point in Q, that is:
P B 11 (Y1,2)
= 111aXgEQ C),Aq (C7 — T471 1) 7r — T2731 — W2 Y2)11q (11 7:1 + g2 F2
= min e 2 s.t. e2> c,,Aq + (cff — yl)7rq + (c, — T2/ — W2 Y2) ibq (11 Ti 1 + Y2 Y2 q E Q.
The inner n'aster problem, which provides a solution to problem [SPB1(7a,
cari now be written as follows in ternis of the extreme points in Q and the
vector of complicating variables 712 :
[iVIPB 1 (Ti1)] • -mpEn(ei ) = min 92
S.t. e2> C),Aq (c, — W1 Y1) 11-9 4- ( cg — 212Y1 — 1'V2 Y2)/tq
±Y1 ± Y2 Y2 • E
Y2 > O.
In Benders' decomposition, the n'aster programs are initially defined on
small subset of cuts, and they are called relaxed n'aster programs. .We
denote the relaxed n'asters by [rMPB] and [rMPB1(Tj1 )] for the outer and
inner master programs, respectively. Moreover, in the two-level Benders' de-
composition, an miter iteration refers to solving [rMPB] where the algorithm
refines the value of the complicating vector y i of the first stage. Alternatively,
amimuer iteration refers to solving [rMPB1(F 1 )] and, given a fixecl value
it refines the value of the complicating vector y2 of the second stage.
Solving Multi-Stage Stochastic, In-house Production and Outsourcing Planning by Two-level Decomposition 128
To case the notation, in the outer iterations, we &Ilote the solution to
[rIVIPB1(F 1 )] as ;92 and the optimal solution to [MPB1(77 1 )] as y, at the end of
the iterative process. A eut whieh is added to [riVIPBieV] can be represented
as e2 > Bq y2 + dg , where Bq is the constraint rnatrix associated with the
vector of variables y 2 , and dg is the remaining constant term. To generate
this eut, we use the dual extreme point that is eomputed in [SPB11(fh,9-2 )].
Alternatively, in the outer iterations, we denote the solution to [rMPBI as9 1
and the optimal solution to [MPB] as y at the end of the iterative process.
A eut which is added to [rMPB] can be represented as e 1 > Bp +4. where
Bp is the eonstraint matrix associated with the vector of variables y i and dp
is the constant term. This eut is generated by using the dual extreme point
computed in solving [SPB11(Tji ,y)], where y is the optimal solution of the
inner loop for a given value of rh. (Note that this dual extreme point cari
be computed at the last iteration of the muer loop beforc the algorithm gets
into the outer loop.)
In both irmer and outer iteration loops, bounds on the optimal value of
the master programs are updated. The lower bounds are updated by the
current optimal solution of the relaxed master programs and are given by
MPB and ,.z_mpsicgo zi*.mpsnpi ) = erMPB . Moreover, the lipper bound on
the outer master program [MPB] is computed as:
m p g = Mill fi mpg Z - IL* 1PB1(pi)} = ZS* PB11( 1 ,y; )1
while, for the inner master program [MPB1(M, it is given by:
= min{-7.A/Pm(rii) , Z.S* PB11(ui,-2)}.
Solving Multi-Stage Stocho„stic, In-house Production and Outsourcing Planning by Two-level Decomposition 129
The iterations terminate when the differenee between the upper and lower
bounds falls bellow a predetermined threshold. A pseudo-code for the pro-
posed two-level Benders &composition is presented in Algorithm 5.2. For
performance comparisons, we have also implemented a single love! Benders'
&composition where the complicating variables are both Yi and y2 • The
subproblem turns out to be SPB11(91 , 92) while the master problem simul-
taneously optimizes the y-variables of both stages.
Algorithm 5.2: Two-level Benders' Decomposition Alyorithrn
Initialization initial value 71 ;initial lower and upper bounds (zmpB , f'mpB ).
Outer iterations While IzmpB - zmpB i > Emps
Inner iterations Initial value 72 ; lowe.r and upper bounds (zmpin(y,), ZmpRiw,))•
While If-A/p./31( 1 ) _2•Afp./31( 1 )1 > EMPB1
2.1 Cadi subproblem [S P B11(Th ,T j2 )]; eompute the dual extreme point (Ag, 7rq, [tg ), q e Q.
2.2 Update [7-MPB1(7 1 )] with eut "02 > Bq y2 + dg .
2.3 Solve [ritIPB1(71 )]; update the value of 92 .
9 .4 'Update the lower and upper bounds fmpin om).
End-while
1.1 Set y2* := v2 • 1.2 Call subproblem [SPB11(71 , y;)];
compute the dual extreme point (A p , np ,i.tp ),p E P.
1.3 Update [r1\413B] with eut 9 1 > Bp dp.
Solving Multi-Stage Stochastic, In-house Production and Outsourcing Planning by Two-level Decomposition 130
1.4 Solve [r1\413B]; update the value of
1.5 Update the lower and lippe'. bounds (zmpB ,
End-while
Set y1 Solve [SPB11(e y)] to complete the optimal prima:1 solution in terms of x*.
5.3.3 The Analytic Center Cutting Plane Method
The analytic center cutting plane method can be viewed as an interior-point
decomposition algorithm; see [33]. This method provides a fast algorithm for
solving large-scale optimization problems by mixing an interior-point algo-
rithm and cutting planes. To solve a multi-stage stochastic program with the
ACCPM, one needs to define an oracle and a query point generator. The spe-
cial recourse structure in [TMB] is best exploited by a decomposition method
such as the .ACCPM. This special structure is an aid in designing the two-
level ACCPM by having the query point generators in two levels, namely -
the first master progra,m UMP] for colunm generation and the outer mas-
ter program [MPBI for Benders) and the second master program UMP11 for
column generation and the muer master program [MPB1(ij 1 )] for Benders).
The oracles are also defined as subproblems [SP11] for column generation and
[SPB11(f 1 , 92 )} for Benders. Moreover, this recourse structure forms a dual
block angular structure in [TM13], and a prima! block angular structure in
its dual. These structural advantages allocate the soultion of the dual prob-
lem to the Dantzig-Wolfe reformulation and coluinn generation technique.
Equivalently, the primal cari be tackled by Benders' decomposition. In the
prescrit chapter, we implemented both decomposition methods on [TM13] and
Solving Multi-Stage Stocho„stic, In-house Production and Outsourcing Planning by Two-level Decomposition 131
its dual The two-level column generation via ACCPM, denoted by C:AC-AC,
is implemented in such a procedure where the ACCPM solves both master
programs and forwards the central dual prices to the subproblem. In return,
the subproblem generates colunms to be added to the second level master
program. The pseudo-code for this algorithm appeared in [62]. This decom-
position by colurnn generation on the dual program can alternatively be clone
as C:DW-AC where the first level master program is solved by the classical
Dantzig-Wolfe decomposition, and the second level master program is solved
by the ACCPM. Equivalently, the two-level Benders' decomposition via AC-
CPM, denoted by B:AC-AC, is implemented in the following mariner: the
ACCPM solves both master programs and reflues the value of the vector of
the complicating variables in cadi level by the information feedback given by
the subproblem.
5.4 Comput ational Results
For the computational experiments, we implemented the two-level decom-
position algorithms on the in-house production and outsourcing planning
[TSM]. The code is written in MATLAB version 7.1 for C:AC-AC and C:DW-
AC by using two-level column generation. In the C:DW-AC approach, the
first linear relaxation of the restricted ma.ster problems is solved using LIN-
PROG, the built-in library within MATLAB 7.1, and the second linear re-
laxation of the restricted master problems is solved by the in-house version of
ACCPM. Moreover, in both the two-level column generation approaches, the
subproblem is solved by using LINPROG. Similarly, the two-level Benders'
Solving Multi-Stage Stochastic, In-house Production and Outsourcing Planning by Two-level Decomposition 132
decomposition B:AC-AC is donc by solving both n'aster progrmns by using
the ACCPM and the subproblem by using LINPROG.
To compare the success of the aforementioned approaches in solving
[TMB], we generate problems based on a different discrete probability distri-
bution of demand and quality. For example, in Problem 1, we have 3 types
of forecasted demand and 3 types of forecasted quality, making 9 scenarios in
the first set of decisions, and 3 scenarios in the second set of decisions in stage
1. In stage 2, this cornes to a total of 117 scenarios for first and second sets
of decisions. Similarly, we have 4 types of demand and 3 types of quality in
Problem 2, 8 types of deinand and 3 types of quality in Problem 3, 10 types
of demand and 5 types of quality in Problem 4, and 13 types of demand and
5 types of quality in Problem 5. The features of the test problems appear in
Table 1.
Table 5.1: Problem Sizes in the Column Generation Approach
Number of Scenarios
Primai var.
Dual var.
Constraints in MP
Constraints in MP1
Constraints in SP11
1 117 274 260 3 36 235 2 208 468 450 3 48 417 3 832 1764 1730 3 96 1665 4 3100 6384 6322 3 130 6251 5 5239 10716 10636 3 234 10479
Note that in the colunm generation approach, an additional constraint is
added to the -first and second master programs as a convexity row. In [62], we
showed that the two-level ACCPM (C:AC-AC) outperformed the two-level
DW (C:DW-DW), where in larger size problems the number of iterations of
Solving Multi-Stage Stochastic, In-house Production and Outsourcing Planning by Two-level Decornposition 133
the C:DW-DW approach was 300 times greater as compared to C:AC-AC.
This motivated us to use C:AC-AC in solving the [TME3]; the computational
results are given in Table 2.
Table 5.2: Computational Results for C:AC-AC
Outer-Gap Inner-Gap Outer-iter Inner-iter 1 le-5 le- 5 4 17 9 le-5 le-4 4 16 3 le-5 le-5 4 22 4 le-3 le-5 2 17 5 le-4 le-5 3 18
This table presents the number of outer and muer iterations required to
reach the optimal solution, along with its relative precision for each problem
size. Note that an inner iteration of C:AC-AC corresponds to an inner loop
of the program in winch the second master program and the subproblem
interact. Alternatively, an outer iteration corresponds to an outer loop where
the first master program interacts with the information received from the
biner loop. The relative precision for the outer and inner iterations in C:AC-
AC barely reaches to 10-5 ; beyond that, it takes the ACCPM hundreds or
even thousands of iterations to reach the optimal solution with 10 -6 relative
precision. Although ACCPM has an extremely good convergence rate, its
relative precision is not always satisfactory, as mentioned in [2]. The recent
attempts try to improve the algorithm by using the column (cut) elimination
[2]. In [32], it is indicated that there is a "trade-off between required relative
precision of optimum and time needed to reach it." In solving [TMI3], we
explore this trade-off so to reach its optimal solution with higher precision.
Solving Multi-Stage Stochastic, In-house Production and Outsourcing Planning by Two-level Decomposition 134
Since [TMB] lias a sinall first master problem, we solved the first master
program by classical Dantzig-Wolfe decomposition, using LINPR.OG with its
relative accuracy of 10 -8 ; whereas, we used ACCPM to solve the larger-sized
second n'aster program. The results of this experiment are shown in Table 3.
Table 5.3: Computational Results for C:D'W-AC
Outer-Gap Inner-Gap Outer-iter Inner-iter 1 le-8 le-6 4 21 9 le-8 le-6 5 27 3 le-8 le-5 5 34 4 le-8 le-5 5 29 5 le-8 le-5 6 27
A comparison of the results in Table 5.2 and Table 5.3 indicates that the
relative precision for both inner and outer gaps are improved in C:DW-AC
as compared to C:AC-AC. The number of outer iterations stayed relatively
the same, but the number of inner iterations are larget. for C:DW-AC. The
C:DW-AC reaches the optimal solution with an extremely good precision
and a small number of iterations.
Finally, we implemented the two-level ACCPM via Benders' cuts on
[M131 The computational results for this third decomposition method B:AC-
AC are shown in Table 5.4.
In Table 5.4, outer iteration refers to solving the first master program
by ACCPM where the algorithm refines the value of the complicating vector
of the first stage. Alternatively, mimer iteration refers to solving the sec-
ond master by ACCPM and refining the value of the complicating vector
in the second stage. The aforemention computational results show that the
Solving Multi-Stage Stochastic, In-house Production and Outsourcing Planning by Two-level Decomposition 135
Table 5.4: Computational R,esults for B:AC-AC
Outer-Gap muer-Cap Outer-iter Inner-iter 1 le-2 le- 3 6 5 2 le-1 le-2 7 7 3 le-1 le-2 5 4 4 - - - - 5 - - -
two-level Benders' decomposition failed for the larger size problems; for the
medium size problems, the level of accuracy only reached 10 -1 for outer it-
erations and 10-2 for inner iteration, winch is not satisfactory. In Problem
1, winch is considered a small sized problem, the two-level Benders performs
better. As vas mentioned previously, we also implemented a single levet Ben-
ders' decomposition where the complicating variables are both Yi and ry2 • Iii
that case ; a single master problem needs to be solved. For Problems 1 to 3,
we reached a preeision of 10 -3 in 17 iterations, which is better than what we
obtained for the two-level version. Our results are consistent with the com-
putational results presented in [50] for the decomposition of the two-stage
stochastic program via Benders' cuts and ACCPM.
5.5 Conclusion
In the present chapter, we formulated the problem of in-house production
and outsourcing planning via the seenario tree for a multi-stage stochastie
program. To address one of the recent challenges facing immufacturers in
ousourcing their products to overseas countries, we considered the stochas-
Solving Multi-Stage Stochastic, In-house Production and Outsourcing Planning by Two-level Decomposition 136
tic quality and demand in our model formulation. Moreover, we proposed
two types of &compositions namely - the two-level Dantzig-Wolfe reformula-
tion (continu generation) of the dual formulation and the two-level Benders'
&composition of the primai forinulation. The two-level decomposition in
both approaches consists of two master programs and a single subproblem,
where the master programs are solved by ACCPM, and the subproblem is
solved by the simplex method. The two-level column generation via ACCPM
(C:AC-AC) provides a more precise optimal solution compared to that of the
two-level Benders decomposition via ACCPM (B:AC-AC). To improve the
relative precision in solving [TMB], wc changed the C:.AC-AC to C:DW-AC,
where the first level program is solved by the classical Dantzig-Wolfe decom-
position, and the second level is solved by die ACCPM. Among the three
decomposition approaches. the C:DW-AC provides the best level of accuracy
within a small nurnber of iterations.
Chapter
Conclusions
Stochastic programming is mathematical programming obtained by intro-
ducing uncertainty into the problem data and decision process. Such models
are necessary when the effectiveness of current decisions is dependent on fu-
ture events. Fields in which stochastic programming models arise include
economics, financial planning [16], vehicle routing problems [48, 30, 31, 59],
ami telecommunications; many examples can lie fourni in [22, 26] .
Given an appropriate probability space (Q, À. P), this leads to a general
formulation for stochastic programming. Four rnain approaches to stochastic
programming have been developed: expected value solution, the scenario
analysis, the chance constrained pneu:num:lig, and the MC:MUSC formulation.
The recourse formulation or two-stage stochastic programming problem lias
a second stage that is &pendent on the decision vector from the first stage.
This model cari be extended to the multi-stage case when a set of decisions
is to be made over tirne up to horizon T.
The deterministic equivalent to the multi-stage recourse problem can be
Conclusions 138
created by replacing the expectations by probability weighted summations
over the finite number of data paths and writing deterministic constraints for
each path. The size of the associated constraint matrix increases exponen-
tially with the nuinber of time stages and the number of outcomes for cadi
random vector. Each of the data paths is called a scenario.
Multi-stage stochastic programming constitutes a very large-scale opti-
mization problem. Sampling methods are required to reduce the large di-
mensionality of realistic deterministic equivalent problems. In addition, par-
allel programming and decomposition principles are designed for multi-stage
programs to solve the problems more quickly or with more details.
Paraffelization can be donc within the framework of different solution
methods of stochastic programming. For example, parallelization of the in-
terior point method means Cholesky factorization of the matrix, which is
used in calculating the iterative step. Parallelization of the simplex method
on the other hand is extremely difficult. Dantzig et al. [18] have a code for
parallel, sample-based L-Shaped me.thod. A parallel nested &composition
code has been developed by Birge et al. [7].
The large-scale deterministic equivalent to multi-stage recourse problems
cari be solved by the simplex or interior point methods. Alternatively, decom-
position methods such as Benders' &composition or Dantzig-Wolfe decom-
position can be employed. These decomposition principles are translated into
the L-shaped method in stochastic linear programming [72]. The L-shaped
method lias been extended to the nested decomposition method and devel-
oped as a computer code for linear multi-stage recourse problems [29]. The
L-shaped method is viewed as a simplex-based decomposition principle in
Conclusions 139
contrast to the methods which are based on interior point algorithms such
as the Analytic Conter Cutting Plane Method (ACCPM) [33].
The ACCPM combines the interior point algorithm with a cutting plane
method providing a. powerful tool for solving large-scale optimization prob-
lems. The ACCPM was first implemented for solving two-stage stochastic
programming with recourse [3]• This specialized implementation is success-
fully used for practical applications such as telecommunication network prob-
lem [1, 53]. In this thesis we implement a two-level decomposition method via
the ACCPM using column generation technique or Benders' cuts for multi-
stage stochastic programming in the following n'aimer: ACCPM solves the
t'ulster programs and passes forward the current central solution to the sub-
problems, while information passes backward in the form of columns or cuts
to be added to the master programs. In a two-level ACCPM, the analytic
center of the localization provides a central dual price in cadi n'aster program
level to be carried to the subproblems. This provides a better alternative to
simplex dual prices which are used in the standard Dantzig-Wolfe decompo-
sition, and helps the method converge in less iterations.
Our computational results show that the two-level ACCPM using the
column generation technique works much botter than the two-level standard
Dantzig-Wolfe decomposition. In some instances of a multi-stage capacity
planning and technology acquisition problem (sec Chapetr 4), the number of
iterations in the standard Dantzig-Wolfe decomposition is about 300 times
more than the two-level ACCPM. To be able to compare the results of the
two-level ACCPM via Benders' cut against the two-level ACCPM via colurnn
generation, we implemented both approaches 011 a multi-stage stochastic in-
Conclusions 140
bouse production and outsourcing planning problem in Chapter 5. The com-
putational results show that the two-level ACCPM via Bender's cut failed
for the larger size problems whereas for the medium size problems, the re-
suit vas not veiy satisfactœy. To improve the relative precision in solving
the saine problem in Chapter 5, we changed the two-level ACCPM in the
following manner: the first level master program is solved by the classical
Dantzig-Wolfe decomposition while the master program at the second level
is solved by the ACCPM. Since the dimension of the first master program in
multi-stage stochastic in-house production and outsourcing planning prob-
lem is small, this approach provides the best level of accuracy within a simili
number of iterations.
In the future, we would like to efficiently solve multi-stage stochastic mix-
integer programs. To do so, we eau combine our two-level column generation
technique and the branch-and-price method. Branch-and-price is a gener-
alization of LP based branch-and-bound to handle integer progranis that
contain a huge number of variables. In tins general framework, each node of
the search tree is solved via column generation and branching occurs when
no columns price out to enter the basis and the LP solution does not satisfies
the integrality constraints. Note that this approach eau also be combined
with cutting plane algorithms. In this case, it simply suffices to apply the
decomposition procedure on the augmented problem. in [25], it is shown
how to adapt the 1)n:inch-and-price method when ACCPM is used as the LP
solver. Specialized branching and cutting strategies will have to be developed
to solve multi-stage stochastic mix-integer programs in a reasonable ïunount
of computer time.
Bibliography
[1] Andrade 11, Lisser A. Maculan N. and Plateau G. Telecommunication net-
work capacity design for uncertain demand. Computational Optimization
and Applications 2004; 29(2): 127-146.
[2] Babonneau IF, du Merle 0, and \fiai J-P. Solving large-scale linear mul-
ticommodity flow problems with an active set strategy and proximal-
ACCPM. Operations Research 2006; 54(1): 184 -197
[3] Bahn 0, du Merle 0, Coffin J-L, and Vial J-P. A cutting plane method
from anal tic centers for stochastic prograrnming. In: Nondifferentiable
and Large Scale Optimization. Mathematical Programming 1995; B 69:
45-73.
[4] Benders JF. Partitioning procedure for solving mixed-variables program-
ming problems. Numerische Mathematik 1962; 4: 238-252.
[5] Berkelaar A, Gromicho JAS, Kouwenberg R, and Zhang S. A primal-dual
decomposition algorithm for multi-stage stochastic convex programming.
Mathematical Programming 2005; A 104: 153-177.
BIBLIOGRAPHY 142
[6] Birge JR. Decomposition and partitioning methods for multi-stage
stochastic linear programs. J. Operations Research 1985; 33: 989-1007.
[7] Birge JR, Donohue CJ, Hohnes DF, and Svintsiski OG. A parallel impie-
mentation of the nested decomposition algorithm for multi-stage stochastic
linear programs. Tech Report 94-1, Department of Industrial and Opera
tions Engineering, University of Michigan 1994; 327-352.
[8] Birge JR and Holmes DF. Efficient solution of two-stage stochastic linear
programs using interior point methods. Computational Optimization and
Applications 1992; 1: 245-276.
[9] Birge JR and Louveaux FV. A mufti-eut algorithm for two-stage stodras-
tic linear programs. European J. of Operations Research 1988; 384-392.
[10] Birge JR and Louveaux FV. Introduction to stochastic progrannning.
Springer Series in OR 1997; New York.
[11] Birge JR and Wets J-B. Designing approximation scheme for stochastic
optimization problems, in particular for stochastic programs with recourse.
Mathematical Programming study 1986; 27, 54-102.
[12] Canez L, Platts K, and Probert D. Developing a framework for make-or-
buy decisions. International Journal of Operations and Production Man-
agement, 20, 11, 1313 - 30.
[13] Charries A and Cooper W. Response to decision problems under risk and
chance constrained programming dilemmas in the transition. Management
Science 1983; 29, 750-753.
BIBLIOGRAPHY 143
[14] Chen Z, Li S. and Tirupati D. A scenario-based stochastic programming
approach for technology and capacity planning. Computers and Operations
Research 2002; 7: 781-806.
[15] Colommbo M, Gondzio j, and Grothyey A. A warm start for large-
seale stochastic linear programs. Technical Report MS-06-004, School of
Mathematics. University of Edinburgh, 2006.
[16] Consigli G and Dempster MA. Dynamic stochastic programming for
asset liability management. Aimais of Operations Research, 1996.
[17] Dantzig GB and Glynn PW. Parallel processors for planning under un-
certainty. Aimais of Operations Research 1990; 22, 1-21.
[18] Dantzig GB, Ho JK, and Infanger G. Solving stochastic linear programs
on a hypercube multi-computer. Technical Report SOL 91-10. Department
of Operations Research, Stanford University 1991.
[19] Dantzig GB and Wolfe P. The decomposition principle for linear pro-
grains. Operations Research 1960; 8, 101-111.
[20] De Ghellinck G and \fiai J-P. A polynomial Newton method for linear
programming. Algorithmica 1986; 1, 425-453.
[21] Degraeve Z and Jans R. A new Danzig-Wolfe reformulation and brandi-
and-priee algorithin for the capacitated lot sizing problem with set
up times. Technical Report, 2003; Erasmus University Rotterdam, the
N eat her land .
BIBLIOGRAPHY 144
[22] Dempster MA. Introduction to stochastic programming. In: M. A.
H. Dempster (Ed.), Stochastic Programming, Academie Press,1980; New
York.
[23] Dempster MA. On stochastic programming II: Dynamic problems under
risk. Stochastics 1988; 25, 15-42.
[24] Den Hertog D. Interior point approach to linear quadratic and convex
programming. Kluwer Academie Piiblishers 1994; London.
[25] Elhedhli S and Goffin J-L. Efficient production-distribution system de-
sign. Management Science, 2005, 51:7, 1151-1164.
[26] Ermoliev Y and Wets RJ. Numerical techniques for stochastic optimiza-
tion. Springer-Verlag 1988; Berlin.
[27] Fisher ML. The Lagrangian relaxation method for solving integer pro-
gramming problems. Management Science 1981; 27:1, 1-18.
[28] Fragnière E, Gondzio J, and Via.! .1-P. Building a,nd solving large-scale
stochastic programs on an affordable distributed computing system. An-
nais of ()pendions Research 2000; 99,167-187.
[29] Gassman HI. MSLiP: a computer code for the multi-stage stochastic
linear programming problem. Mathematical Programming 1990; 47: 407-
423.
[30] Gendreau M, Laporte G, and Seguin R. An exact algorithm for the
vehicle routing problem with stochastic demand and customers. Trans-
portation Science 1995; 29, 143-155.
BIBLIOGRAPHY 145
[31] Gendreau M, Laporte G, and Seguin R.. Invited review: Stochastic ve-
hicle routing. Europeïm J. Operations Research 1996; 88, 3-12.
[32] Coffin J-L, Gondzio J, Sarkissian R, and Via]. J-P. Solving nonlinear
multicommodity flow problems by analytie center cutting plane method.
Mathematical Programming 1996; 76, 131-154.
[33] Goffin J-L, Haurie A. and Via]. J-P. Decomposition and nondifferentiable
optimization with the projective algorithm. Management Science 1992; 38:
284-302.
[34] Coffin J-L and Vial J-P. Cutting planes and column generation tech-
niques with the projective algorithm. Journal of Optimization Theory and
Applications 1990; 65(3), 409-429.
[3,5] Goffin J-L and \trial J-P. On the computation of weighted analytic centers
and dual ellipsoids with the projective algorithm. Mathematical Program-
ming 1993; 60, 81-92.
[36] Goffin J-L and Via! J-P. Multiple cuts in analytic center cutting plane
method. SIAM Journal of Optimization 1997; 11(1): 266-288.
[37] Gondzio J and Sarkissian R. Colunm generation -with a primai-dual
method. Logilab Technical Report 1997; 96.6.
[38] Gondzio Sarkissian R, and Vial J-P. Using an interior point method
for the master problem in a decomposition approach. European Journal of
Operations Research 1997; 101: 577-587.
BIBLIOGRAPHY 146
[39] Herald Tribune-Business. Mattel recalls millions more Chinese-made
toys. http://www.iht.com/articles/2007/08/14/business/toy.php
[40] Herbig PA and O'Hara BS. The future of original equipment manu-
facturers: a 'natter of partnerships. Journal of Business and industrial
Marketing, 1994; 9, 38 - 43.
[41] Higle JL and Sen S. Stochastic decomposition: An algorithm for two-
stage linear program with recourse. Mathematics of Operations R.esearch
1991; 16, 650-669.
[42] Infanger G. Monte Carlo (importance) sampling within the benders de-
composition algorithm for stocha:stic linear programs. Annal of Operations
Research 1992; 39 69-95.
[43] Kelley JE. The cutting plane method for solving convex programs. Jour-
nal of SIAM 1960; 8, 703-712.
[44] Kibzune AI and Kan YS. Stochastic programming problems with prob-
ability and quintile functions. John Wiley Inc 1996; U.K.
[45] King A and Rackafellar RI. Asymptotic theory for solutions in general-
ized M-estimation and stochastic programing. Mathematics of Operations
Research 1993; 18 148-162.
[46] Lageweg BJ. Lenstra JK, Rinnooy Kan AH, and Stougie L. Stochastic
integer programming. Statistica Neerlandica 1985; 39, 97-113.
BIBLIOGRAPHY 147
[47] Laporte G ami Louveaux FV. The integer L-shaped method for stochas-
tic integer programs with complete recourse. Operations Research Letters
1993; 13, 133-142.
[48] Laporte G, Louveaux FV, and Mercure H. An exact solution for a priori
optimization of the probabilistic traveling salesman problem. Operations
Research 1994; 42, 543-9.
[49] Lee HL and Lee CY. Building supply chain excellence in emerging
economies. Springer US. 2007, V. 98.
[50] Lisser A. Sarkissian R, and Via! J-P. Optimal joint synthesis of base ami
reserve telecommunication networks. Technical Report, Logilab, Depart-
ment of Management Studies, University of Geneva, Switzerland, 1995.
[51] Li S, Loulou R, and Rahman A. Technological progress and technology
acquisition: strategic decision imder uncertainty. Production and Opera-
tions Management 2003; 12(1): 102-119.
[52] Linderoth jT ami Wright SJ. Implementing a &composition algorithm
for stochastic programming on a computational grid. Computational Opti-
mization and Applications, special issue on Stochastic Programming 2003;
24, 207-250.
[53] Lisser A, Ouorou A, Vial J-P, and Gondizo J. Capacity planning under
uncertain demand in teleconnnunication networks. France Telecom Tech-
nical Report, FTRD, France, 1999.
[54] Liibbecke ME and Desrosiers J. Selected TOpieS iii Colunin Generation.
Operations Research 2005; 53(6): 1007-1023.
BIBLIOGRAPHY 148
[55] Martin RK. Large scale linear and integer optimization: a unifie(' ap-
proach. Kluwer Academie Publishers,1999.
[56] Mitchell JE and Todd M. Solving combinatorial opthnization proble.ms
using Karmarkar's algorithm. Mathematical Programming 1992; 56: 245-
284.
[57] Mulvey JM and Ruszczynski A. A new scenario &composition method
for large scale stochastic optimization. Operations R.esearch 1995; 43, 477-
490.
[58] Noël M-C, and Smeers Y. Neste(' decomposition of multi-stage Non-
linear programs with recourse. Mathematical Programming 1987; 37, 131-
152.
[59] Norkin V. Ermoliev Y. and Ruszczynski A. On optimal allocation of
indivisibles under uncertainty. J. Operations Research 1998; 46, 381-395.
[60] Norkin V, Pflug G, and Ruszczynski A. A brandi and bound method
for stochastic global optimization. Mathematical Programming 1998; 83,
425-450.
[61] Oukil A. Ben Amor H. Desrosiers J. and El Gueddari H. Stabilized
column generation for highly degenerate multiple-depot vehicle scheduling
problems. Computers and Operations Research 2005; 34(3): 817-834.
[62] Rasekh L and Desrosiers J. A two-level interior-point &composition
algorithm for inulti-stage stochastic capacity planning and technology ac-
quisition. HEC Montréal, Les Cahiers du GERAD G-2008-09, 18p.
BIBLIOGRAPHY 149
[63] Rockafellar RT and Wets RJ-B. A Lagrangian fuite generation tech-
nique for solving linear-quadratic problems in stochastic programming.
Mathematical Programming 1986; 28: 63-93.
[64] Rockdellar R.T and -Wets R.J-B. Scenario and policy aggregation in op-
timization under uncertainty. Mathematics of Operations Research 1991;
16(1): 119-147.
[65] Ruszczynski A. parallel decomposition of multi-stage stochastic pro-
gramming problems. Mathematical Programming 1993; •8A. 201-228.
[66] Sethi AK and Sethi SP. Flexibilty in manufacturing: a survey. Interna-
fichai Flexible Manufa:cturing Systems 1990; 2: 289-328.
[67] Shapiro A. Asymptotic analysis of stochastic progrïuns. Aimais of Op-
erations Research 1991; 30, 169-186.
[68] Shapiro A. Stochastic programming by monte carlo simulation methods,
published electronically. In: Stochastic Programming E-Print Series, 2000.
[69] Stougie L. Design and analysis of algorithms for stochastic integer pro-
grumning. CWI Tract, centrum voor Wiskunde en Informatica Amester-
dam 1987; 37.
[70] Sun j and Liu X. Scenario formulation of stochastic programs and the
homogeneous self-dual interior point method. To appear in INFORMS
Journal on Computing.
[71] .Vanderbei Rj. Linear programming: foundations and extensions,
Kluwer Academie Publishers, Boston, MA, 1996.
BIBLIOGRAPHY 150
[72] Van Slyke R and .Wets Rj-B. L-shaped linear programs with application
to optimal control and stochastic programming. SIAM J. Appl. Math.
1969; 17: 638-663.
[73] Wallace SW and Yan TC. Bounding multi-stage stochastic programming
from above. Mathematical Programming 1993; 61, 111-129.
[74] Wets R.J. Stochastic programming with fixed recourse: the equivalent
deterministic problem. SIAM Review 1974; 16, 309-339.
[75] Nets Rj. Stochastic programming. In: G.L. Nemhauser, A.H.G. Rin-
nooy Kan, and M.J. Todd (Eds), Handbooks of Operations Research ami
Management Science 1990; 1.
[76] Wittrock RJ. Advances in nested decomposition algorithm for solving
staircase linear programs. Technical Report SOL 83-2, System Optimiza-
tion Laboratory, Stanford University, 1983.
[77] Wright SJ. Primal-dual interior point methods. SIAM, Philadelphia,
1997.
[78] Ye Y. Interior point algorithms, theory and analysis. John Wiley Inc,
1997, U.K.
[79] Zhao G. A log-barrier method with Benders decomposition for solving
two-stage stochastic linear programs. Mathematical Programming 2001;
90: 507-536.