Internet Routing Example

Acme Routing Company wants to route traffic over the internet from
San Fransisco to New York. It owns some wires that go between San
Francisco, Houston, Chicago and New York. The table below describes
how many kilobytes can be routed on each wire in a second. Figure out
a set of routes that maximizes the amount of traffic that goes from San
Francisco to New York.
Cities Maximum number of kbytes per second
S.F. - Chicago 3
S.F. - Houston 6
Houston - Chicago 2
Chicago - New York 7
Houston - New York 5
One commodity, one source, one sink
 Maximum Flows

• A flow network G = (V, E) is a directed graph in which each edge

(u, v) ∈ E has a nonnegative capacity .
• If (u, v) 6∈ E , we assume that c(u, v) = 0 .
• We distinguish two vertices in a flow network: a source s and a sink
t.

A flow in G is a real-valued function f : V × V → R that satisfies the

following two properties:
Capacity constraint: For all u, v ∈ V , we require 0 ≤ f (u, v) ≤ c(u, v) .
Flow conservation: For all u ∈ V − {s, t} , we require
f (v, u) = f (u, v) .
X X

v∈V v∈V

The value of a flow f is defined as

|f | = f (s, v) − f (v, s) , (1)
X X

v∈V v∈V
 Example

a b
5
4
2

s 1
t
1
3 3

2
c d
 Solutions

2/ 2

a b
2/ 5 2/
4
1/
2
0/ 1
s t
1/ 1
3/ 3 3/ 3

2/ 2
c d
Internet Advertising
 Availability Queries

Given:
• Forecasts on ad impressions for each publisher’s slot
• A reservation request

Compute: The maximum number of reservations that can be satisfied

from the reservation request.
 Example

An advertiser wants:
• 50 placements on sports slots
• 50 placements on afternoon slots
Publisher is offering
• 40 placements on slots that are sports and non-afternoon
• 40 placements on slots that are sports and afternoon
• 40 placements on slots that are non-sports and afternoon

reservations ad placements
sports and 40
afternoon

50 Sports

sports and
40
non−afternoon

50 Afternoon

non−sports and 40
afternoon
 Example is a max flow problem

reservations ad placements
sports and 40
afternoon

50 Sports

sports and
40
non−afternoon

50 Afternoon

non−sports and 40
afternoon

reservations ad placements
sports and 40
inf afternoon

50 Sports
inf
sports and
source 40
non−afternoon
inf sink

50 Afternoon
inf
non−sports and 40
afternoon
 Algorithm: Ford Fulkerson

Greedily send flow from source to sink.

Ford-Fulkerson-Method (G, s, t)
1 initialize flow f to 0
2 while there exists an augmenting path p
3 augment flow f along p
4 return f
For this to work, we need a notion of a residual graph
 Residual Graph

The residual graph is the graph of edges on which it is possible to push

flow from source to sink.
• The residual capacity of (u, v) , is







c(u, v) − f (u, v) if (u, v) ∈ E ,
cf (u, v) = 

f (v, u) if (v, u) ∈ E , (2)
 0 otherwise .




• The residual graph Gf is the graph consisting of edges with positive

residual capacity
 Residual Network

v1 12/12 v3 v1 12 v3
15/ 5
11 /16 20 5
11 15

4
1/4

7/7
s t s t

7
1
3
9
5

4/

5
8/1
3
v2 v4 4/4 8 v2
3
v4
4
11/14 11
(a) (b)

v1 12/12 v3 v1 12 v3
6 19/ 5 1
1 20
11/
11 19
1/4

7/7

s t s t

7
1
3
1
9

9
12/
13
v2 v4 4/4 12
v2
3
v4
4
11/14 11
(c) (d)
 Updating a Flow

• Send flow along the path defined by the residual graph.

• Amount: minimum of capacity of all residual edges in the augmenting
path.
• If a residual edge is a graph edge, then add the flow.
• If a residual edge is a reverse edge, then subtract the flow.
v1 12/12 v3 v1 12 v3
15/ 5
16 20 5
11/
11 15

4
1/4

s 7/7 t s t

7
1
3
9

5
4/

5
8/1
3
v2 v4 4/4 8 v2
3
v4
4
11/14 11
(a) (b)

v1 12/12 v3 v1 12 v3
19/ 1
11 /16 20 5
11 19
1/4

7/7

s t s t

7
1
3
1
9

9
12/
13
v2 v4 4/4 12
v2
3
v4
4
11/14 11
(c) (d)
 s - t Cuts
An s - t cut satsfies
•s∈S , t∈T
• S∪T =V , S∩T =∅

2/ 2

a b
2/ 5 2/
4
1/
2
0/ 1
s t
1/ 1
3/ 3 3/ 3

2/ 2
c d

Capacity of a cut (only forward edges)

c(S, T ) = c(u, v)
X

u∈S,v∈T

Flow crossing a cut (net flow)

(S, T ) = f (u, v) − f (u, v)
X X

u∈S,v∈T u∈T,v∈S
 Properties of cuts and flows

Capacity of a cut (only forward edges)

c(S, T ) = c(u, v)
X

u∈S,v∈T

Flow crossing a cut (net flow)

(S, T ) = f (u, v) − f (u, v)
X X

u∈S,v∈T u∈T,v∈S

• For all cuts (S, T ) and all feasible flows f , f (S, T ) ≤ c(S, T ) (weak
duality).
• For all pairs of cuts (S1, T1) and (S2, T2) , and all feasible flows f ,
f (S1, T1) = f (S2, T2) .
 Examples of cuts

v1 12/12 v3
6 15/
/ 1 20
11

1/4

7/7
s t

9
4/
8/1
3
v2 v4 4/4
11/14
S T
 Max-flow min-cut theorem

If f is a flow in a flow network G = (V, E) with source s and sink t ,

then the following conditions are equivalent:
1. f is a maximum flow in G .
2. The residual network Gf contains no augmenting paths.
3. |f | = c(S, T ) for some cut (S, T ) of G .
Proof
 Ford Fulkerson expanded

F ord − F ulkerson(G, s, t)
1 for each edge (u, v) ∈ E(G)
2 f (u, v) = 0
3 while there exists a path p from s to t in the residual network Gf
4 cf (p) = min{cf (u, v) : (u, v) is in p}
5 for each edge (u, v) in p
6 if (u, v) ∈ E
7 f (u, v) = f (u, v) + cf (p)
8 else f (v, u) = f (v, u) − cf (p)
 Algorithm

Residual graph (Capacities) Graph (Flow/Capacity)

v1 12 v3 v1 4/12 v3
16 20 6 20
4/1
(a) s t s t

7
9
4/
9
13 13
v2 v4 4 v2 v4 4/4
14 4/14

8 8/12
v1 v3 v1 v3
12 20 6 4/2
4 4/1 0
4
4

4/4
(b) s t s t

7
4

9
4/
5

13 10 4/1
v2 v4
4 3
v2 v4 4/4
4 4/14

4 8/12
v1 v3 16 v1 v3
12 6 8/2
8 8/1 0
4 4
4

(c) s t s t

7
4

9
5

4/1
4
v2
10
v4
4 3
v2 v4 4/4
4 4/14
 Algorithm continued

Residual graph (Capacities) Graph (Flow/Capacity)

4 8/12
v1 v3 12 v1 v3
8 15/
8 6 20
8/1
8 8

7/7
(d) s t

4
7
9 s t

9
10 11/
4
v2 v4
4 13
v2 4/4
v4
4 11/14

4 12/12
v1 v3 v1 v3 19/
8 5 6
8 2/1 20
1
8 15

7/7
(e) s t
4

4
7 s t
11
9

9
11/
2
v2
3
v4
4 13
v2 4/4
v4
11 11/14

v1 12 v3
4 1

12 19
(f) s t
4

11
9

2 3 4
v2 v4
11
 Analysis

• 1 iteration of FF takes O(E + V ) time (breadth-first search plus book-

keeping).
• Each iteration sends at least one unit of flow.
• Total time O(f ∗E) .
• This algorithm is only psuedo-polynomial.

00 u 1,0 99 u 1,0 99 u 999

,0 00, ,9 00, ,9 ,99
1,000 000 999 000 999 9
1 1 1
s t s t s 999 t
1

1
,9 99 , 999 ,999
1,0
00, 00
1,0
00, 999 999
0
000 00, 000 1 1 1
v 1,0 v v

(a) (b) (c)

A polynomial algorithm– Shortest Augmenting Path

Algorithm due to Edmonds and Karp, Dinic

Algorithm
• Run Ford-Fulkerson, but always choose the shortest augmenting path
in the residual graph.
• Breadth-first search implements this algorithm.
 Idea for Analysis

Intuition
• Augment along shortest path in residual graph.
• Short paths get “saturated” and disappear.
• Future augmenting paths are along longer paths
• Eventually algorithm terminates because no more paths exist (remem-
ber that no augmenting path can have length greater than V.

This doesn’t quite work, but is close

 Analysis

• Let δ(v) be the shortest path distance from v to t in Gf .

• Lemma: δ(v) is monotonically increasing over time.

Proof:
• Let δ be shortest path distances before an augmentation.
0
• Let δ be shortest path distances after an augmentation.
0
• Suppose, f.p.o.c. that for some v , δ (v) < δ(v) , and v is the minimum
0
vertex with this property (w.r.t δ )
Before

5 4 3 2 1 0

v t

After

4 3 2 1 0

v w t

How could this happen?

 How could this happen?

Before

5 4 3 2 1 0

v t

After

4 3 2 1 0

v w t

0
• (v, w) ∈ Gf but not in Gf . (why?)
• We must have sent flow on (w, v) in the augmenting path
• δ(w) = 6
0
• δ (w) < δ(w) , which contradicts v being the minimum vertex whose
label decreased.

Conclusion: δ(v) never decreases.

 Analysis Continued

• Each δ(v) increases at most V times.

• Total number of times any δ(v) increases is at most V 2 .
• We need to tie this to an augmenting path.
• Problem: A particular augmenting path may not increase any δ(v) .
(How can this happen?)

Solution We need to find some other event that happens on every aug-
menting path and then relate this event to increasing distances.

Conclusion: δ(v) never decreases.

 Analysis Continued

• 0 ≤ δ(v) ≤ V
• δ(v) never decreases.
• Each δ(v) increases at most V times.
• Total number of times some δ(v) increases is at most V 2 .
• We need to tie this to an augmenting path.
• Problem: A particular augmenting path may not increase any δ(v) .
(How can this happen?)

Solution We need to find some other event that happens on every aug-
menting path and then relate this event to increasing distances.

Edge Saturation
• If an augmeting path send cf (v, w) units of flow on edge (v, w) then
we call the push saturating.
• Every augmenting path saturates at least one edge. (Why?)
 Analysis Continued

Edge Saturation
• If an augmeting path send cf (v, w) units of flow on edge (v, w) then
we call the push saturating.
• Every augmenting path saturates at least one edge.

Focus on a particular pair of vertices (v, w)

• Suppose that (v, w) ∈ Gf
• What has to happen between two consecutive saturations of (v, w)
 Events Between 2 edge saturations

5 4
v w

saturate (v,w)

v w

5 6

v w

saturate (w,v)

v w

7 6
v w

saturate (v,w)

• Both v and w have to be relabeled.

• So each edge can be saturated at most V /2 times.
 Putting it together

• Each edge (or its reverse) can be saturated at most V /2 times.

• There are most EV edge saturations.
• Each augmenting causes at least one edge saturation
• Therefore, there are at most EV augmenting paths.
• Recall O(E) time per augmenting path.

A max flow can be found in O(E 2V ) time .

Faster augmenting path based algorithms exist, most based on finding

the augmenting paths more efficiently.
• O(V E log V ) (Sleator and Tarjan)
• O(min E 3/2, V 2/3E log(V 2/E) log C) (Goldberg and Rao)
 Push relabel algorithms

Algorithm due to Goldberg and Tarjan

Ideas
• Maintain variables d(v) which are “distances”, estimates (lower bounds)
on the distance to the sink (or source) in the residual graph
• Push flow over one edge at a time, pushing from a higher distance vertex
to a lower distance vertex.
• Allow excess flow to accumulate at a vertex.
• Maintain a preflow rather than a flow.
Capacity constraint: For all u, v ∈ V , we require 0 ≤ f (u, v) ≤ c(u, v) .
Relaxed Flow conservation: For all u ∈ V − {s, t} , we require
f (v, u) ≥ f (u, v) .
X X

v∈V v∈V

Define excess
e(u) = f (v, u) − f (u, v) (3)
X X

v∈V v∈V
Example
 Comparison with augmenting paths

• Augmenting paths
– Always maintains a flow (feasibility)
– Works towards optimality (maximum flow)
• Push/relabel
– Always maintains a preflow and works towards a flow
– Maintains a superoptimal preflow and moves towards feasibility.
 Basic Operations

• If e(v) > 0 then v is active.

• If cf (v, w) > 0 and d(v) = d(w) + 1 then (v, w) is admissible.

Push from active vertices over admissible edges.

P ush(u, v)
1 // Applies when: u is active, cf (u, v) > 0, and d(u) = d(v) + 1.
2 // Action: Push f (u, v) = min(e(u), cf (u, v)) units of flow from u to v.
3 ∆(u, v) = min(e(u), cf (u, v))
4 if (u, v) ∈ E
5 f (u, v) = f (u, v) + ∆(u, v)
6 else f (v, u) = f (v, u) − ∆(u, v)
7 e(u) = e(u) − ∆(u, v)
8 e(v) = e(v) + ∆(u, v)

Relabel When a vertex has all its outgoing residual edges pointing uphill,
we relabel it.
Relabel(u)
1 // Applies when: u is active and for all v ∈ V such that (u, v) ∈ Ef ,
we have d(u) ≤ d(v).
2 // Action: Increase the label of u.
3 d(u) = 1 + min{d(v) : (u, v) ∈ Ef }
 Generic Push Relabel Algorithm

Initialize − P ref low(G, s)

1 for each vertex v ∈ V (G)
2 d(v) = 0
3 e(v) = 0
4 for each edge (u, v) ∈ E(G)
5 f (u, v) = 0
6 d(s) = |V (G)|
7 for each vertex v ∈ Adj(s)
8 f (s, v) = c(s, v)
9 e(v) = c(s, v)
10 e(s) = e(s) − c(s, v)
Generic − P ush − Relabel(G)
1 Initialize-Preflow(G, s)
2 while there exists an applicable push or relabel operation
3 select an applicable push or relabel operation and perform it
Run Through Example
 Properties of Algorithm (without proofs)

• At any point, any active vertex either has a path to the source or the
sink in the residual graph.
• There is never an s − t path in Gf .
• Let δ(v) be the minimum of distance to sink and V plus distance to
soure in Gf .
• If v is active then d(v) ≤ δ(v) . (inductive proof, similar to shortest
augmenting path, new short paths are not created).
• d(v) ≤ d(w) + 1 for any (v, w) ∈ Gf .
• If e(v) > 0 then either
– v has an outgoing admissible edge
– v can be relabeled (and the label will increase)
• If e(v) = 0 for all v ∈ V − {s, t} , then f is a maximum flow.

Proof of last statement

• f is a flow by the definition of preflow.
• f is maximum because there is no s − t path in Gf .
 Running Time

We will need to bound

1. Number of relabels
2. Time per relabel
3. Number of pushes
4. Time per push
5. Bookkeeping time
 Easy ones:

Relabel(u)
1 // Applies when: u is active and for all v ∈ V such that (u, v) ∈ Ef ,
we have d(u) ≤ d(v).
2 // Action: Increase the label of u.
3 d(u) = 1 + min{d(v) : (u, v) ∈ Ef }

Time per relabel

• Easy bound: O(V )
• Time to relabel each vertex once : O(V 2)
 Easy ones:

Relabel(u)
1 // Applies when: u is active and for all v ∈ V such that (u, v) ∈ Ef ,
we have d(u) ≤ d(v).
2 // Action: Increase the label of u.
3 d(u) = 1 + min{d(v) : (u, v) ∈ Ef }

Time per relabel

• Easy bound: O(V )
• Time to relabel each vertex once : O(V 2)
• Better bound: O(degree(v))
• Time to relabel each vertex once: degree(v) = O(E)
P
v

(This is an example of amortized analysis)

Number of relabels
• Vertex labels are at most 2V , so each vertex is relabelled at most 2V
times.
• Total time spent relabelling = V degree(v) = O(EV )
P
v
 Pushes

P ush(u, v)
1 // Applies when: u is active, cf (u, v) > 0, and d(u) = d(v) + 1.
2 // Action: Push ∆(u, v) = min(e(u), cf (u, v)) units of flow from u to v.
3 ∆(u, v) = min(e(u), cf (u, v))
4 if (u, v) ∈ E
5 f (u, v) = f (u, v) + ∆(u, v)
6 else f (v, u) = f (v, u) − ∆(u, v)
7 e(u) = e(u) − ∆(u, v)
8 e(v) = e(v) + ∆(u, v)

Easy part
• Time per push = O(1) .
• Bookkeeping (can be amortized against other operations with reason-
able data structures and rules for choosing push/relabel operations).

Two types of Pushes

• Saturating push: Sends cf (v, w)) flow on (v, w)
• Non-saturating push: Sends less flow. ( e(v) ).
 Bounding Saturating Pushes

Consider a saturating push on (v, w) . What has to happen before the

next saturating push on (v, w) ?
 Bounding Saturating Pushes

Consider a saturating push on (v, w) . What has to happen before the

next saturating push on (v, w) ?

• w has to be relabelled
• A push on (w, v) must occur
• v has to be relabelled.

Conclusion
• Between any two saturating pushes on (v, w) , v must be relabelled.
• v can be relabelled at most 2V times.
• There are at most 2V saturating pushes on (v, w) .
• The total number of saturating pushes is O(V E) .
 Non-saturating pushes

Issue After a non-saturating push, the graph does not change, nothing
need be relabelled and another non-saturating push can occur on the edge
before anything significant happnes.

Solution As with shortest-augmeting-path, we won’t count directly, but

will bound other operations and related to non-saturating pushes. We will
do so via means of a potential function.

Φ= d(v)
X

v:e(v)>0
 Analysis

Φ= d(v)
X

v:e(v)>0

• After initialiazation Φ ≤ 2V 2.
• At termination Φ = 0 .
• Let R be total increase in Φ due to relabellings.
• Let S be total increase in Φ due to saturating pushes.
• Each non-saturating push decreases Φ by at least 1. (Why?).
 Putting these facts together

• Total decrease in Φ associated with non-saturating pushes is at most

2V 2 − 0 + R + S .
• Each non-saturating push decreases Φ by at least 1. (Why?).
• Number of non-saturating pushes is at most 2V 2 − 0 + R + S .
 Bounding R and S

Φ= d(v)
X

v:e(v)>0
Relabellings

• Each relabelling must increase Φ by at least 1.

• Total increase in Φ associated with relabelling v is at most 2V .
• Total increase associated with all relabellings is at most 2V 2 .

Saturating Pushes
• A saturating push leaves excess at v . It adds excess to w .
– If w already had excess, then Φ is unchanged.
– If w did not have excess, then Φ increases by d(w) , which is at
most 2V .
• There are at most O(EV ) saturating pushes, therefore total increase
due to saturating pushes is O(EV 2) .
 Putting it Together

Number of non-saturating pushes is at most

2V 2 − 0 + R + S = O(V 2 + EV + EV 2) = O(EV 2)
.

Conclusion: Running time is O(EV 2) .

Note: Can be improved by

• Choosing operations more carefully.
• Better data structures to represent flow
• Best running times are close to V E.
• Winner in practice, assuming that one uses two additional heuristic
ideas
– gap heuristic
– global relabellings