Submodular Optimization and Approximation Algorithm -

Submodular Optimization and

Approximation Algorithm

Satoru Iwata

岩田覚

RIMS, Kyoto University

京都大学数理解析研究所

Outline

• Submodular Functions

Examples

Discrete Convexity

• Submodular Function Minimization

• Approximation Algorithms

• Submodular Function Maximization

• Approximating Submodular Functions

Submodular Functions

• Cut Capacity Functions

• Matroid Rank Functions

• Entropy Functions

)()()()( YXfYXfYfXf

VYX ,

VX Y

R2: Vf

:V Finite Set

Cut Capacity Function

Cut Capacity

} leaving :|)({)( XaacX

X

s t

Max Flow Value＝Min Cut Capacity

0)( ac

Matroid Rank Functions

:

)()(

||)(,

YXYX

XXVX

],[rank)( XUAX

Matrix Rank Function

Submodular

A

X

V

U

Whitney (1935)

Entropy Functions

:)(Xh

0)( h

X

Information Sources

0

)()()()( YXhYXhYhXh

Entropy of the Joint Distribution

Conditional Mutual Information

Positive Definite Symmetric Matrices

0)( f

A ][XA][detlog)( XAXf

X

)()()()( YXfYXfYfXf

Ky Fan’s Inequality

Extension of the Hadamard Inequality

Vi

iiAAdet

Discrete Concavity

)(}){()(}){( TfuTfSfuSf

TS

V

T

Diminishing Returns

S u

Discrete Convexity

)()()()( YXfYXfYfXf

V

X Y

Convex Function

x

y

Discrete Convexity

},{ ba

},,{ cbaV

}{b

}{a

}{c

},{ cb

Lovász (1983)

f̂

f

f̂

： Linear Interpolation

： Convex

： Submodular

:]1,0[ Chosen Uniformly at Random

},)(|{: vxvX )](E[)(ˆ Xfxf

Submodular Function Minimization

X

)(Xf

?}|)(min{ VYYf

Minimization Algorithm

Evaluation Oracle

Minimizer

0)( fAssumption:


)( 87 nnO )log( 5 MnO

)log( 7 nnO

Grötschel, Lovász, Schrijver (1981, 1988)

Iwata, Fleischer, Fujishige (2000) Schrijver (2000)

Ellipsoid Method

Cunningham (1985)

|)(|max XfMVX

: Time for Function Evaluation

Base Polyhedra

Submodular Polyhedron

)}()(,,|{)( YfYxVYxxfP V R

Yv

vxYx )()(

}|{ RR VxV

)}()(),(|{)( VfVxfPxxfB

Base Polyhedron

Greedy Algorithm

v

))((

))((

))((

)(

)(

)(

111

0

11

001

2

1

2

1

nn vLf

vLf

vLf

vy

vy

vy

}){)(())(()( vvLfvLfvy

Edmonds (1970) Shapley (1971)

Extreme Base

)(vL

)( Vv

:y

2v1v nv

Theorem

Min-Max Theorem

)}(|)(max{ )(min fBxVxYfVY

)}(,0min{:)( vxvx

)()()( YfYxVx

Edmonds (1970)

Combinatorial Approach

Extreme Base

Convex Combination

Cunningham (1985)

L

L

L yx

)( fByL

x

|)(|max XfMVX

)log( 6 nMMnO

)()()( TfTxVx

Combinatorial Approach

Tvvx

Tvvx

,0)(

,0)(

:Ly

L

. ),()( LTfTyL

L

L

L yx

).()( TfTx

T

Extreme Base

:T Minimizer


)( 87 nnO )log( 5 MnO

)log( 7 nnO

Grötschel, Lovász, Schrijver (1981, 1988)

Iwata, Fleischer, Fujishige (2000) Schrijver (2000)

Iwata (2003)

Fleischer, Iwata (2000)

)log)(( 54 MnnO

Orlin (2007)

)( 65 nnO

Iwata (2002)

Fully Combinatorial

Ellipsoid Method

Cunningham (1985)

Iwata, Orlin (2009)

)( subject to

Minimize2

fBx

x

The Fujishige-Wolfe Algorithm Fujishige (1984)

sol. opt. :x

}0)(|{: vxvS

)(min)()( XfTfSfVX

}0)(|{: vxvT

Minimal Minimizer

Maximal Minimizer

x

Evacuation Problem （Dynamic Flow）

:)(Xo

:)(a

Hoppe, Tardos (2000)

:)(vb

:)(ac

SX XT \

TSXXoXb ),()(

TS

Capacity

Transit Time

Supply/Demand

Maximum Amount of Flow from to .

Feasible

Multiterminal Source Coding

X XR

Y

Encoder

Encoder YRDecoder

)(YH

)(XH

)|( XYH

)|( YXH

),( YXH

),( YXHXR

YR

Slepian, Wolf (1973)

Multiclass Queueing Systems

ii

Server

Control Policy

Arrival Rate Service Rate

Multiclass M/M/1

Preemptive

Arrival Interval Service Time

Performance Region

VXS

Xi

i

Xi

ii

Xi

ii

,1

:s

,: iii

YRjs

is

:js Expected Staying Time of a Job in j

Achievable

Coffman, Mitrani (1980)

:s1

Vi

i

A Class of Submodular Functions Vzyx R,,

))(()()()( XxhXyXzXf

:h

VXS

Xi

i

Xi

ii

Xi

ii

,1

Nonnegative, Nondecreasing, Convex

)( VX

i

iiy

:

xxh

1

1:)(

iii Sz :

iix :

Submodular

Itoko & Iwata (2007)

Zonotope in 3D

))(),(),(()( XzXyXxXw

}|)({conv VXXwZ

)(),,(~

xyhzzyxf

Zonotope

} ofPoint ExtremeLower :),,(|),,(~

min{

}|)(min{

Zzyxzyxf

VXXf

Remark: ),,(~

zyxf is NOT concave!

z

Line Arrangement

iii zyx

Topological Sweeping Method

Edelsbrunner, Guibas (1989) )( 2nO

Enumerating All the Cells

Symmetric Submodular Functions

. ),\()( VXXVfXf

)( 3nO

RVf 2:

Symmetric

Symmetric Submodular Function Minimization

Minimum Cut Algorithm by MA-ordering

Nagamochi & Ibaraki (1992)

Minimum Degree Ordering Nagamochi (2007)

?}|)(min{ VXXf

Queyranne (1998)

Submodular Partition

)( jiVV ji

k

i

iVf1

)(Minimize

subject to kVVV 1

Greedy Split

)11(2 k -Approximation

:f Monotone or Symmetric

Zhao, Nagamochi, Ibaraki (2005)

Questions

What Kind of Approximation Algorithms

Can Be Extended to Optimization Problems

with Submodular Cost or Constraints ?

Cf. Submodular Flow (Edmonds & Giles, 1977)

How Can We Exploit Discrete Convexity

in Design of Approximation Algorithms?

Cf. Ellipsoid Method

(Grötschel, Lovász & Schrijver, 1981)

Submodular Vertex Cover

Graph ),( EVG

Submodular Function

RVf 2:

)(SfFind a Vertex Cover VS Minimizing

2-Approximation Algorithm

Goel, Karande, Tripathi, Wang (FOCS 2009)

Iwata & Nagano (FOCS 2009)

Relaxation Problem

)( 0)( Vvvx

Convex Programming Relaxation (CPR)

)),(( 1)()( Evuevxux

)(ˆ xfMinimize

subject to

CPR has a half-integral optimal solution.


Submodular Cost Set Cover

U).(Xf

NX UN

Find Covering

with Minimum

||max: uUu

N

Rounding Algorithm

-Approximation

Primal-Dual Algorithm

uuN

Submodular Multiway Partition

)( jiVV ji

k

i

iVf1

)(Minimize

subject to kVVV 1

),...,1( kiVs ii

Chekuri & Ene (FOCS 2011)

:f Nonnegative Submodular

Relaxation Problem

),,...,1( 0)( Vvkivxi

Convex Programming Relaxation

k

i

i Vvvx1

)( 1)(

k

i

ixf1

)(ˆMinimize

subject to

Ellipsoid Method

),...,1( 1)( kisx ii

Optimal Solution :

ix

Rounding Scheme

),...,( 1 kAA

3/2-Approximate Solution

:]1,0[

),,...,1( })(|{: kivxvA ii

Chosen Uniformly at Random

Uncross

Return

i

k

iAVU

1:

Symmetric Submodular :f

),,...,( 11 UAAA kk

Rounding Scheme

2-Approximate Solution

:]1,(21

),,...,1( })(|{: kivxvA ii

Chosen Uniformly at Random

Return

i

k

iAVU

1:

Nonnegative Submodular :f

),,...,( 11 UAAA kk

Improvement over the -Approximation

by Zhao, Nagamochi, & Ibaraki (2005)

)1( k

Submodular Function Maximization

Monotone Submodular Functions

Nemhauser, Wolsey, Fisher (1978)

Cardinality Constraint

(1-1/e)-Approximation

Calinescu, Chekuri, Pál, Vondrák (IPCO 2007)

Vondrák (STOC 2008)

Filmes and Ward (FOCS 2012)

Matroid Constraint

(1-1/e)-Approximation

Greedy Approximation Algorithm

}{:

}){(maxarg:

:

1

1

0

jjj

jj

vTT

vTfv

T

1jv1vGreedy Algorithm

)(SfMaximize subject to kS ||

kj ,...,1

1jT

Nemhauser, Wolsey, Fisher (1978)

:f Monotone Submodular Function


),...,1( kj

)(: *Sf:*S

1

1

j

i

ijk

jj

TSu

jjj

j

kTf

TfuTfTf

TSfSf

j

)(

)(}){()(

)()( )

1

\

111

1

**

1*

Optimal Solution

1

1

j

i

i

k

i

ikTf1

)(

)()(: 1 jjj TfTf


),...,1( 0 kjj

j

jkk

11:ˆ

),...,1( kj

kk

k

k

2

1

11

0

1

00

Minimize

k

i

i

1

subject to

e

11

111ˆ

1

kk

i

ik

)(: *Sf

k

i

ki Tf1

)(

Submodular Welfare Problem

(Monotone, Submodular) kff ,...,1Utility Functions

)( jiVV ji

k

i

ii Vf1

)(Maximize

subject to kVVV 1

e)11( -Approximation Vondrák (2008)

Submodular Function Maximization

Nonnegative Submodular Functions

Feige, Mirrokni, Vondrák (FOCS 2007)

2/5-Approximation

Lee, Mirrokni, Nagarajan, Sviridenko (STOC 2009)

1/4-Approximation (Matroid Constraint)

1/5-Approximation (Knapsack Constraints)

Buchbinder, Feldman, Naor, Schwartz (FOCS 2012)

1/2-Approximation

Double-Greedy Algorithm Buchbinder, Feldman, Naor, Schwartz (FOCS 2012)

Initialize: .: ,: ,: DVBA

Invariants: .\ , ABDBA

Iteration: grow A or shrink B.

Output: either A or B that has the greater value.

AB

V

Double-Greedy Algorithm

BDA While

Select ).(\ ADBu

}.0),(}){(max{: AfuAf

}.0),(}){\(max{: BfuBf

with probability ,

}{: uAA

with probability .

}{\: uBB

If ,0 then

}.{: uDD Else

Double-Greedy Algorithm

The expectation )](2)()(E[ ZfBfAf

never decreases in the process.

) Expected changes of

.0)(2 2

at each iteration is at least

)(2)()( ZfBfAf

Approximating Submodular Functions

X

)(Xf

Algorithm Evaluation

Oracle

Function f̂Y

)(ˆ Yf

Algorithm with

for monotone submodular functions

Construct a set function such that

For what function is this possible?

Approximating Submodular Functions

. ),(ˆ)()()(ˆ VXXfnXfXf

,0)( f . 0,)( VXXf

f̂

)log()( nnOn

Goemans, Harvey, Iwata & Mirrokni (SODA 2009)

Ellipsoidal Approximation

:K

)( KxKx

Centrally Symmetric Convex Body

Maximum Volume Inscribed Ellipsoid

(The John Ellipsoid)

)()( AEnKAE

:)(AE

Submodular Load Balancing

?)(maxmin},...,{ 1

jjjVV

Vfm

Xv

jj vpXf )(:)(

:,...,1 mff

Svitkina & Fleischer (FOCS 2008)

Monotone Submodular Functions

Scheduling


Lenstra, Shmoys, Tardos (1990)

)log( nnO -Approximation Algorithm

Learning Submodular Functions

Balcan & Harvey (STOC 2010)

Construct a set function such that

holds for most ( fraction) of

For what function is this possible with

probability in time?

)(ˆ)()()(ˆ XfnXfXf

1

f̂

1 )/1,/1,poly( n

.X

PMAC-learning

Summary

• Submodular Functions Arise Everywhere.

• Discrete Analogue of Convexity.

• Combinatorial Algorithms for Minimization.

• Exploit Special Structures of Problems.

• Approx. Algorithms by Discrete Convexity

• Approx. Algorithms for Maximization

• Learning Submodular Functions

Submodular Optimization and Approximation Algorithm -

Documents

Transcript of Submodular Optimization and Approximation Algorithm -