Intractability
Intractability
I NTRACTABILITY
                                         ‣ introduction
                                         ‣ P vs. NP
Algorithms F O U R T H   E D I T I O N
                                         ‣ poly-time reductions
                                         ‣ NP-completeness
R OBERT S EDGEWICK | K EVIN W AYNE
                                         ‣ dealing with intractability
https://algs4.cs.princeton.edu
                                                                      2
                                     I NTRACTABILITY
                                     ‣ introduction
                                     ‣ P vs. NP
Algorithms                           ‣ poly-time reductions
                                     ‣ NP-completeness
R OBERT S EDGEWICK | K EVIN W AYNE
                                     ‣ dealing with intractability
https://algs4.cs.princeton.edu
Fundamental questions
A Turing machine
                                                          4
A dif cult problem: integer factorization
Integer factorization. Given an integer x, find a nontrivial factor. or report that no such factor exists
not 1 nor x
                                                                                                                                                5
fi
Another dif cult problem: boolean satis ability
Boolean satisfiability. Given a system of boolean equations, find a satisfying truth assignment.
Applications.
・Automatic verification systems for software.
・Mean field diluted spin glass model in physics.
・Electronic design automation (EDA) for hardware.
・…
                                                                                                                          6
       fi
                                              fi
Another dif cult problem: boolean satis ability
Boolean satisfiability. Given a system of boolean equations, find a satisfying truth assignment.
Ex.
              ¬ x1   or     x2     or            x3             =   true
                x1   or   ¬ x2     or            x3             =   true
              ¬ x1   or   ¬ x2     or    ¬ x3                   =   true
              ¬ x1   or   ¬ x2     or                 or   x4   =   true
                          ¬ x2     or            x3   or   x4   =   true
a SAT instance
                                                                                                   7
       fi
                                            fi
How dif cult can it be?
quantity estimate
Q. Could galactic computer solve satisfiability instance with 1,000 variables using brute-force search?
A. Not even close: 21000 > 10300 >> 1079 ⋅ 1013 ⋅ 1017 = 10109.
                                                                                                                          9
Intractability: quiz 1
                                                                                                     10
     Some computational problems
                                                                                             ¬ x2 or   x3 =           true     x1 = false
                   SAT                given a system of boolean equations,
                                                                                   ¬ x1   or ¬ x2 or ¬ x3 =           true     x2 = true                ?
          (boolean satis ability)           nd a satisfying assignment
                                                                                               x2 or ¬ x3 =           true     x3 = true
                                                                                                  3              0
                ST-CONN                  given a graph and two vertices,
                                                                                                                               0–3–2–4           depth- rst search
           (graph connectivity)            nd a path that connects them
                                                                                          1             2        4
                      ⋮                                  ⋮                                             ⋮                           ⋮                     ⋮
                                                                                                                                                                     11
fi
fi
fi
     fi
              fi
                           fi
Intractable problems
tractable intractable?
                                                                                       12
Intractable problems
                       13
                                     I NTRACTABILITY
                                     ‣ introduction
                                     ‣ P vs. NP
Algorithms                           ‣ poly-time reductions
                                     ‣ NP-completeness
R OBERT S EDGEWICK | K EVIN W AYNE
                                     ‣ dealing with intractability
https://algs4.cs.princeton.edu
Search problems
Search problem. Computational problem for which you can check a solution in poly-time.
         14 7 57395 25 89 67 64 1 292 7       1 93 70 7 72 1
                   instance I                   solution S
Poly-time checking algorithm. Check whether solution is a divisor of x. O(n 2) time via long division
Search problem. Computational problem for which you can check a solution in poly-time.
       ¬ x1   or     x2    or          x3             =   true
         x1   or   ¬ x2    or          x3             =   true           x1 = false
       ¬ x1   or   ¬ x2    or     ¬ x3                =   true           x2 = false
       ¬ x1   or   ¬ x2    or               or   x4   =   true           x3 = true
                   ¬ x2    or          x3   or   x4   =   true           x4 = true
instance I solution S
Poly-time checking algorithm. Plug values of solution into system of equations and check.
                                                                                            O(mn) time
                                                                                                         16
     NP
                            ST-CONN                  given a digraph and two vertices,           check for existence of edges between
                       (graph connectivity)             nd a path that connects them                 consecutive vertices in path
                          BLOCK-CHAIN           given strings s and h, nd a string t whose      compute the hash of the concatenation
                        (hash veri cation)           concatenation with s hashes to h                   and compare with h
⋮ ⋮ ⋮
     Significance. Problems that scientists, engineers, and programmers aspire to solve in practice.
                                                                                                                                        17
fi
fi
fi
       fi
            fi
                 fi
                         fi
Intractability: quiz 2
D. Neither A nor B.
                                                                                                                        18
     P
Definition. P is the class of all search problems that can be solved in poly-time. more accurately, FP
⋮ ⋮ ⋮
NP
                          intractable                                             P = NP
           P           search problems
                       P ≠ NP                                                    P = NP
               brute-force search may be                                    poly-time algorithms for
                   the best we can do                                    FACTOR, SAT, LONGEST-PATH, …
ordinary appreciation
Intuition. Checking a solution seems like it should be way easier than finding it.
                                                                                                                               22
      fi
           fi
                fi
Princeton computer science building
                                      23
Princeton computer science building (closeup)
                   1
                           0         1
                   0                              0     0
                           1          1                           0
                    1                             1                     0
                            0          0                1         0
                     1                             1     1              1
                             0         1                           1
                     0                              0                    0
                             1          1                 0         0
                                                    1                     0
                                                          1         1     1
                              char          ASCII        binary
                                 P           80         1010000
                                 =           61         0111101
                               N             78         1001110
                                 P           80         1010000
                                 ?           63         0111111
                                                                              24
                                     I NTRACTABILITY
                                     ‣ introduction
                                     ‣ P vs. NP
Algorithms                           ‣ poly-time reductions
                                     ‣ NP-completeness
R OBERT S EDGEWICK | K EVIN W AYNE
                                     ‣ dealing with intractability
https://algs4.cs.princeton.edu
    Bird s-eye view
                                                                             26
’
Poly-time reduction
                                    Algorithm
         instance I                                          solution to I
                                      for Y
           (of X)
Algorithm for X
Design algorithms. If X poly-time reduces to Y, and can solve Y efficiently, then can also solve X.
                                                                                                      27
Poly-time reduction
                                     Algorithm
         instance I                                             solution to I
                                       for Y
           (of X)
Algorithm for X
Establish intractability. If SAT poly-time reduces to Y, then Y is intractable. assuming SAT is intractable
                                    Algorithm
         instance I                                              solution to I
                                      for Y
           (of X)
Algorithm for X
X reduces to SAT: X is no harder than SAT. (If I can solve SAT, then I can solve X.)
SAT reduces to X: X is no easier than SAT. (If I can solve X, then I can solve SAT.)
                                                                                       29
Integer linear programming
instance I solution S
          instance I
           (of SAT)                                Algorithm      solution to I
                                                    for ILP
                       yi = 0 ⟹ xi = false
Post-processing:
                       yi = 1 ⟹ xi = true
                                                                                  32
Intractability: quiz 3
                                                                              33
More poly-time reductions from SAT
SAT
PARTITION
                                                                                                                                 34
                                     I NTRACTABILITY
                                     ‣ introduction
                                     ‣ P vs. NP
Algorithms                           ‣ poly-time reductions
                                     ‣ NP-completeness
R OBERT S EDGEWICK | K EVIN W AYNE
                                     ‣ dealing with intractability
https://algs4.cs.princeton.edu
NP-completeness
Two worlds.
NP
                              NPC                      P = NP = NPC
           P
P ≠ NP P = NP
                                                                                             36
Intractability: quiz 4
        I.   X is in NP.
       II.   If X can be solved in poly-time, then P = NP.
      III.   If X cannot be solved in poly-time, then P ≠ NP.
A. I only.
B. II only.
C. I and II only.
Key property. An NP-complete problem can be solved in poly-time if and only if P = NP.
                                                                                         37
Cook–Levin theorem
                                                                                                               38
Implications of Cook–Levin theorem
EXACT-COVER HAMILTON-CYCLE
SUBSET-SUM TSP
PARTITION CLIQUE
EXACT-COVER HAMILTON-CYCLE
SUBSET-SUM TSP
PARTITION CLIQUE
EXACT-COVER HAMILTON-CYCLE
SUBSET-SUM TSP
PARTITION CLIQUE
                              44
Intractability: quiz 5
A program with which of these running times is most likely to be useful in practice?
 A.   10226 n
                                          poly-time
                             (but probably not useful in practice)
 B.   n226
                    n                        exponential time
 C.   1.000000001                    (but probably useful in practice)
Key point. Poly-time is not always a surrogate for useful in practice,       some poly-time algorithms are slow;
though it tends to be true for the algorithms we encounter in the wild.   some exponential-time algorithms are fast!
                                                                                                                       45
Identifying intractable problems
                                                                                              46
Identifying intractable problems
         I “I
           guess I'm nd
              can't  justa to dumb. algorithm.”
                           poly-time              “I can't nd a poly-time algorithm, but neither can all these famous people.”
                                                                                                                                 47
 fi
 fi
Approaches to dealing with intractability
                                                                                                    48
Dealing with intractability:     nd solutions to real-world problem instances
Observations.
・Worst-case inputs may not occur for practical problems.
・Instances that do occur in practice may be easier to solve.
・Reasonable approach: relax the condition of guaranteed poly-time.
Boolean satisfiability.
・Chaff solves real-world instances with 10,000+ variables.
・Princeton senior independent work (!) in 2000.
Traveling salesperson problem.
・Concorde routinely solves large real-world instances.
・85,900-city instance solved in 2006.
                                                                                TSP solution for 13,509 US cities
Integer linear programming.
・CPLEX routinely solves large real-world instances.
・Routinely used in scientific and commercial applications.
                                                                                                                    49
                          fi
Dealing with intractability: approximation algorithms
MAX-CUT: given a graph G, find the cut with maximum number M of crossing edges.
Approximate version: find a large cut.
                                                                                             50
Dealing with intractability: approximation algorithms
3-SAT: given 3-variable equations on n boolean variables, find satisfying truth assignment.
Approximate version: find assignment that satisfies many equations.
                                                                                                         51
      Leveraging intractability: RSA cryptosystem
                                                  multiply
                                                   (easy)
                                                    factor
                                                  (di cult)
                                                                                                                          52
ffi
Leveraging intractability: guiding scienti c inquiry
                                                                             53
                                        fi
Summary
                                                                                       54
Credits
— John Nash
                                                                                         56
fi
                                                         fi