📚 L9: Reductions & Intractability
Lecture Goal
Understand how reductions formalize “problem A is no harder than problem B”, use reductions to prove hardness, and see the chain that connects 3-SAT, Independent Set, Vertex Cover, and Set Cover.
The Big Picture
Reductions are the universal language of hardness. Instead of proving each problem hard from scratch, we show a known-hard problem reduces to it — and hardness “flows” through the reduction chain. This is how we know thousands of problems are all equivalent in difficulty to 3-SAT.
1. What is a Reduction?
🤔 The Core Problem
Instead of learning a new algorithm for every problem, can we transform one problem into another we already know how to solve?
💡 The Reduction Concept
A reduction is a way to solve problem by transforming it into a problem that you already know how to solve.
flowchart LR A["Instance α<br/>(Problem A)"] -->|"Convert"| B["Instance β<br/>(Problem B)"] B -->|"Solve B"| S["Solution B(β)"] S -->|"Translate back"| R["Solution A(α) ✅"] style A fill:#e1f5fe style B fill:#fff9c4 style S fill:#c8e6c9 style R fill:#c8e6c9
Formally, to reduce to :
- Take an instance of problem
- Convert into an instance of problem (this is the reduction step)
- Solve on
- Translate ‘s answer back into an answer for
We write this as: , read as “A reduces to B” or “A is no harder than B”.
📖 Key Terminology
| Term | Definition |
|---|---|
| Instance | A specific input to a problem (sometimes called “encoding”) |
| Reduction | A transformation from instances of to instances of |
| ” reduces to ” — solving gives us a way to solve |
💡 Warm-Up Example: T-SUM reduces to 0-SUM
- 0-SUM: Given array , find such that
- T-SUM: Given array and number , find such that
flowchart LR B["Array B, Target T<br/>(T-SUM instance)"] -->|"Define A[i] = B[i] - T/2"| A["Array A<br/>(0-SUM instance)"] A -->|"Solve 0-SUM"| Sol["Find i,j: A[i] + A[j] = 0"] Sol -->|"Same i,j"| Ans["B[i] + B[j] = T ✅"] style B fill:#e1f5fe style A fill:#fff9c4 style Sol fill:#c8e6c9
Reduction: Given for T-SUM, define .
Then . ✅
⚠️ Common Pitfalls
Pitfall 1: Confusing the direction of the reduction
means you are solving A using B. It does NOT mean is easier than — it means is at least as hard as . Students often read the arrow backwards!
Pitfall 2: Forgetting the translation step
You need both directions: converting AND converting . Forgetting the back-translation is a common error in proofs.
❓ Mock Exam Questions
Q1: Reduction Direction
If we reduce Problem A to Problem B, which problem’s algorithm do we use?
Answer
Answer: We use Problem B’s algorithm
means we transform A-instances into B-instances and solve using B’s algorithm.
Q2: Arrow Meaning
What does mean?
Answer
Answer: A is no harder than B (or equivalently, B is at least as hard as A)
The arrow points from easier to harder. If B is solvable, so is A.
2. p(n)-Time Reductions & Running Time Composition
🧠 Core Concept
A -time reduction from to means:
- The conversion runs in time
- The back-translation runs in time
where is the size of the input (NOT the size of ‘s output).
flowchart TD subgraph Reduction ["Reduction Pipeline"] A["Input α (size n)"] -->|"Convert<br/>O(p(n))"| B["Instance β<br/>(size O(p(n)))"] B -->|"Solve B<br/>T(O(p(n)))"| S["Solution B(β)"] S -->|"Translate back<br/>O(p(n))"| R["Solution A(α)"] end Total["Total Time: T(O(p(n))) + O(p(n))"] style Reduction fill:#e1f5fe style Total fill:#c8e6c9
📐 Running Time Composition
If there is a -time reduction from to , and has a -time algorithm, then can be solved in:
Why? The instance has size , so solving on it takes time. Add the for conversion and back-translation.
📖 Key Terminology
| Term | Definition |
|---|---|
| -time reduction | Reduction where both conversion and back-translation run in time |
| Input size | The size of the original instance — not the size of |
| Running Time Composition | How to combine reduction cost and -solver cost to get total runtime for |
⚠️ Common Pitfalls
Confusing (size of ) with (size of constructed instance)
In the MAT-MULTI → MAT-SQR example: the input size (number of entries in two matrices). The conversion constructs a matrix — work. So it’s an -time reduction, not !
3. Polynomial-Time Reductions & Pseudo-Polynomial Algorithms
🧠 Core Concept
Definition: (polynomial-time reduction) if there is a -time reduction from to where for some constant .
flowchart LR subgraph Implications ["Key Implications"] I1["A ≤_P B and B ∈ P<br/>⇒ A ∈ P"] I2["A ≤_P B and A hard<br/>⇒ B hard"] end Hard["Hard Problem A"] -->|"≤_P"| B["Problem B"] Hard --> I2 B --> I1 style I1 fill:#c8e6c9 style I2 fill:#ffccbc style Hard fill:#ff9999
Key implication:
Contrapositive (crucial for hardness!):
Think of it like a disease spreading upward in difficulty: if is hard and reduces to , then “inherits” the hardness of . 🦠
Why polynomial time? The notion is robust: multiplying two polynomials gives another polynomial. So chaining reductions still gives .
📊 Pseudo-Polynomial Algorithms
An algorithm is pseudo-polynomial if it runs in time polynomial in the numeric value of the input, but exponential in the length (number of bits) of the input.
flowchart LR subgraph KNAPSACK ["KNAPSACK Example"] DP["DP runs in O(nW)"] Input["Input: O(n log W) bits"] W["W = O(2^(log W))"] end DP -->|"Polynomial in W"| Pseudo["Pseudo-polynomial ❌"] Input -->|"True input size"| Exp["O(nW) = O(n · 2^(log W))<br/>Exponential!"] style Pseudo fill:#ffccbc style Exp fill:#ff9999
Classic example: Dynamic programming for KNAPSACK runs in .
- The input size is bits
- But can be as large as
- So is exponential in the input length → pseudo-polynomial, NOT truly polynomial ❌
Compare to FRACTIONAL KNAPSACK with : this IS polynomial ✅
📖 Key Terminology
| Term | Definition |
|---|---|
| polynomial-time reduces to | |
| Polynomial-time algorithm | Runs in for some constant , where = input encoding length |
| Pseudo-polynomial | Polynomial in the numeric value but exponential in the bit length |
| Tractable | Solvable in polynomial time |
| Intractable | No known polynomial-time algorithm (and strong evidence none exists) |
⚠️ Common Pitfalls
Thinking is polynomial for KNAPSACK
is encoded in bits. So the actual input size is , meaning . The runtime is exponential! Always measure polynomial-ness against the encoding length, not the numeric value.
Believing and poly-time poly-time
This is FALSE. The implication only goes the other way: poly-time poly-time. And: hard hard.
❓ Mock Exam Questions
Q3: Implication Direction
If and can be solved in polynomial time, what can we conclude?
Answer
Answer: Nothing about B’s complexity
The reduction only tells us: “if B is easy, then A is easy” (and contrapositively: “if A is hard, then B is hard”). It says nothing about B when A is easy.
Q4: Pseudo-polynomial Classification
The KNAPSACK DP runs in time. Input encoded in bits. What is this?
Answer
Answer: Pseudo-polynomial
It’s polynomial in the numeric value , but , making it exponential in the actual input length.
4. Decision Problems vs Optimisation Problems
🧠 Core Concept
To compare problem hardness uniformly, we use a common language: decision problems.
flowchart LR subgraph Optimisation ["Optimisation Problem"] Opt["Find the BEST solution"] Val["Output: A value/solution"] end subgraph Decision ["Decision Problem"] Dec["Is there a solution<br/>with value ≥/≤ k?"] Ans["Output: YES or NO"] end Opt -->|"Add threshold k"| Dec Dec -->|"If YES for optimal k,<br/>you found the optimum"| Opt style Decision fill:#c8e6c9
| Type | Output | Example |
|---|---|---|
| Optimisation | A value/solution | ”What is the shortest path from to ?” |
| Decision | YES or NO | ”Is there a path from to of length ?” |
Conversion recipe: Given an optimisation problem, create a decision version by introducing a threshold and asking “is there a solution with value ?” (for minimisation) or ”?” (for maximisation).
Why does this work for hardness?
The decision version is no harder than the optimisation version:
- If you can solve optimisation, just check if the optimal value satisfies the threshold.
- So: if the decision version is hard → the optimisation version is also hard.
📖 Key Terminology
| Term | Definition |
|---|---|
| Decision problem | Maps an instance to {YES, NO} |
| YES-instance | An input for which the answer is YES |
| NO-instance | An input for which the answer is NO |
| Threshold | Parameter added to convert optimisation to decision |
⚠️ Common Pitfalls
Thinking the decision problem is always strictly easier
Decision Optimisation (decision is no harder). But in many cases they are equivalent in hardness — if you can solve the decision version for all , you can binary search for the optimum.
5. Reductions Between Decision Problems
🧠 Core Concept
For decision problems, a poly-time reduction is a poly-time transformation such that:
The YES/NO answer is “copied” — no translation needed, just the if-and-only-if.
To prove a valid reduction, show three things:
- ✅ The transformation runs in polynomial time
- ✅ () If is a YES-instance of , then is a YES-instance of
- ✅ () If is a YES-instance of , then is a YES-instance of
Both directions of the if-and-only-if must be proved! 🔁
⚠️ Common Pitfalls
Proving only one direction of the iff
Many students prove () but forget (), or vice versa. Both are required and examiners will look for both.
Confusing with
These are very different statements! means ” is no harder than .” Always be explicit about which direction you’re reducing.
6. The Problem Zoo
🧠 Key Problems
🕸️ INDEPENDENT-SET
Problem: Given a graph and integer , is there a subset with such that no two vertices in are adjacent?
flowchart TD subgraph Graph ["Graph G"] V1["●"] --- V2["●"] V1 --- V3["●"] V2 --- V4["●"] V3 --- V4 V4 --- V5["●"] end IS["Independent Set S<br/>(no edges between members)"] Size["|S| ≥ k?"] style V1 fill:#c8e6c9 style V3 fill:#c8e6c9 style V5 fill:#c8e6c9
Analogy: A group of people at a party where nobody knows each other — they’re “independent.” 🥳
🛡️ VERTEX-COVER
Problem: Given a graph and integer , is there a subset with such that every edge has at least one endpoint in ?
Analogy: Place security cameras at intersections so that every road is watched by at least one camera. 📷
🧩 SET-COVER
Problem: Given integers , , and a collection of subsets of , are there subsets in whose union is ?
Analogy: Choose radio stations so that every city is in at least one station’s broadcast range. 📻
🔮 3-SAT
flowchart LR subgraph Building ["Building Blocks"] Lit["Literal: xᵢ or x̄ᵢ"] Clause["Clause: (x₁ ∨ x₂ ∨ x̄₃)"] CNF["CNF: AND of clauses"] end Problem["Does Φ have a<br/>satisfying assignment?"] Lit --> Clause --> CNF --> Problem style Problem fill:#fff9c4
Building blocks:
- Literal: A Boolean variable or its negation
- Clause: A disjunction (OR) of literals, e.g.,
- CNF formula: A conjunction (AND) of clauses
Problem: Given a CNF formula where every clause has exactly 3 literals, does have a satisfying truth assignment?
Analogy: Can you set switches (True/False) so that in every group of 3 switches, at least one is ON? 💡
📖 Key Terminology Summary
| Problem | Input | YES iff… |
|---|---|---|
| INDEPENDENT-SET | subset of mutually non-adjacent vertices | |
| VERTEX-COVER | subset of vertices touching all edges | |
| SET-COVER | subsets whose union | |
| 3-SAT | CNF formula | truth assignment satisfying all clauses |
⚠️ Common Pitfalls
Confusing Independent Set and Vertex Cover
Independent Set wants vertices with NO edges between them (size ). Vertex Cover wants vertices that TOUCH all edges (size ). Note the complementary nature — this is exactly the key insight in their reduction!
7. The Reduction Chain
The lecture builds the following reduction chain:
flowchart LR SAT["3-SAT"] -->|"≤_P"| IS["INDEPENDENT-SET"] IS -->|"≤_P"| VC["VERTEX-COVER"] VC -->|"≤_P"| SC["SET-COVER"] Hard["All believed HARD"] --> SAT style SAT fill:#ff9999 style IS fill:#ffccbc style VC fill:#ffccbc style SC fill:#ffccbc
By transitivity: . Since 3-SAT is believed to be hard, all problems in this chain are believed to be hard! 😱
7.1 INDEPENDENT-SET VERTEX-COVER
flowchart LR subgraph KeyInsight ["Key Insight"] IS["S is Independent Set"] VC["V \\ S is Vertex Cover"] end IS <--> VC subgraph Reduction ["Reduction"] Input["(G, k) for IS"] Output["(G, n-k) for VC"] end Input -->|"Complement"| Output style KeyInsight fill:#e1f5fe
Key Insight: is an independent set in is a vertex cover.
Reduction: is a YES-instance of INDEPENDENT-SET is a YES-instance of VERTEX-COVER (where ).
Proof ():
Suppose , , is an independent set. For any edge : since is independent, at least one of , so at least one of them is in . Thus is a vertex cover of size . ✅
Proof ():
Suppose , , is a vertex cover. If and , then neither endpoint is in , contradicting being a vertex cover. So is an independent set of size . ✅
7.2 VERTEX-COVER SET-COVER
flowchart TD subgraph Graph ["Graph G = (V, E)"] E1["Edge e₁"] E2["Edge e₂"] E3["..."] end subgraph SetCover ["Set Cover Instance"] U["Universe U = {1, 2, ..., |E|}<br/>(edge indices)"] Sets["For each vertex v:<br/>Sᵥ = {i : eᵢ incident on v}"] K["k' = k"] end Graph -->|"Edges become<br/>universe elements"| SetCover style Graph fill:#e1f5fe style SetCover fill:#fff9c4
Reduction: Given for VERTEX-COVER, construct:
- Universe (label edges )
- For each vertex :
- Set
Intuition: Each vertex “covers” the edges incident to it. Choosing a vertex cover of size corresponds to choosing sets that together cover all edge indices.
Proof ():
If is a vertex cover of size , then every edge has an endpoint in , so . ✅
Proof ():
If () form a set cover, then every edge index is covered, meaning every edge has an endpoint in — a vertex cover of size . ✅
7.3 3-SAT INDEPENDENT-SET
flowchart TD subgraph Construction ["Construction from 3-SAT Φ with m clauses"] C1["Clause 1<br/>(x₁ ∨ x̄₂ ∨ x₃)"] C2["Clause 2<br/>(x̄₁ ∨ x₂ ∨ x₄)"] C3["..."] Cm["Clause m"] end subgraph Graph ["Independent Set Graph"] T1["Triangle: 3 literals<br/>per clause"] T2["Triangle"] Conflict["Conflict edges: connect<br/>xᵢ to x̄ᵢ"] end C1 --> T1 C2 --> T2 Construction -->|"k = m"| Graph style T1 fill:#fff9c4 style T2 fill:#fff9c4 style Conflict fill:#ff9999
Reduction: Given 3-SAT instance with clauses:
- Create 3 vertices per clause (one per literal) — label them with the literal
- Connect the 3 literals in each clause in a triangle (intra-clause edges)
- Connect each literal to all occurrences of its negation in other clauses (conflict edges)
- Set (number of clauses)
Why triangles? An independent set can pick at most 1 vertex from each triangle — forcing us to pick exactly one literal per clause to satisfy.
Why conflict edges? These prevent us from picking from one clause and from another — they can’t both be true!
Proof ():
Given a satisfying assignment, pick one true literal per clause. These vertices form an independent set:
- No two are in the same triangle (one per clause) ✅
- No two are connected by a conflict edge (can’t have and both true) ✅
Proof ():
Given an independent set of size : exactly one vertex from each triangle is in . Set those literals to true. This is consistent (conflict edges prevent and both being chosen), and every clause is satisfied. ✅
⚠️ Common Pitfalls
3-SAT → IS: Forgetting conflict edges
Without conflict edges, you could pick from one triangle and from another — a contradiction. The conflict edges are essential for correctness.
IS → VC: Thinking the complement has size exactly
If , then . The inequality goes the right way — don’t flip it!
VC → Set Cover: The universe is the set of EDGES, not vertices
Students often confuse what the universe represents. It encodes edges, not vertices.
8. NP-Completeness (Preview)
🧠 Core Concept
All the problems seen (3-SAT, INDEPENDENT-SET, VERTEX-COVER, SET-COVER, KNAPSACK, etc.) are NP-complete:
flowchart TD subgraph NPC ["NP-Complete Problems"] SAT["3-SAT"] IS["INDEPENDENT-SET"] VC["VERTEX-COVER"] SC["SET-COVER"] KS["KNAPSACK"] etc["...thousands more"] end Implication["If ANY ONE has a poly-time algorithm,<br/>then ALL of them do<br/>⇒ P = NP"] SAT --> Implication style NPC fill:#ffccbc style Implication fill:#fff9c4
If any one of these NP-complete problems has a polynomial-time algorithm, then all of them do — and we would have .
No polynomial-time algorithm is known for any of them. Most computer scientists believe , meaning these problems are genuinely intractable. 💀
The web of reductions you’ve seen (Dick Karp’s famous 1972 paper!) shows that these seemingly unrelated problems from graph theory, logic, and combinatorics are all equivalent in hardness.
🔗 Connections
| 📚 Lecture | 🔗 Connection |
|---|---|
| L10 | Using reductions to prove NP-completeness |
The Key Insight
Reductions let us stand on the shoulders of giants. We don’t need to prove hardness from scratch — we just show our problem is “at least as hard” as a known-hard problem. The entire NP-completeness web rests on this powerful idea. 🕸️
9. Mock Exam Questions
📝 Multiple Choice
Q1: Reduction Implication
Suppose . Which of the following statements is definitely true?
Answer
Answer: (b) If can be solved in polynomial time, then so can .
means “if is easy, then is easy.” It does NOT imply the reverse.
Q2: KNAPSACK Classification
The dynamic programming algorithm for KNAPSACK runs in time. The input is encoded using bits. Which correctly classifies this algorithm?
Answer
Answer: (c) Pseudo-polynomial, because it is polynomial in the numeric value of but exponential in the bit length of .
, so is exponential in the encoding length.
Q3: Conflict Edges Purpose
In the 3-SAT INDEPENDENT-SET reduction, what is the purpose of connecting a literal vertex to its negation vertex in another clause?
Answer
Answer: (b) To ensure we don’t simultaneously set a variable to both TRUE and FALSE.
Intra-clause triangles enforce “at most one per clause.” Conflict edges enforce consistency of the truth assignment across clauses.
✍️ Open-Ended Practice Problems
P1: SUBSET-SUM ≤_P ZERO-SUBSET-SUM
Let SUBSET-SUM be: Given set and target , does there exist a subset such that ?
Define ZERO-SUBSET-SUM: Given a set of integers , does there exist a non-empty subset such that ?
Show that SUBSET-SUM ZERO-SUBSET-SUM. Clearly describe the reduction and prove correctness in both directions.
Answer
Reduction: Given for SUBSET-SUM, construct for ZERO-SUBSET-SUM.
Runtime: — clearly polynomial time. ✅
Correctness (): If is YES, there exists with . Then has sum . ✅
Correctness (): If is YES for ZERO-SUBSET-SUM, there exists non-empty with sum 0.
- If : Then has sum . ✅
- If : Then has sum 0, meaning (trivially YES for SUBSET-SUM). ✅
P2: INDEPENDENT-SET → VERTEX-COVER
(a) Given a graph with 8 vertices and an independent set of size 5, what is the minimum size of a vertex cover guaranteed by the reduction?
(b) Suppose someone claims: “If and can be solved in polynomial time, then can be solved in polynomial time.” Is this TRUE or FALSE? Justify.
(c) In the VERTEX-COVER SET-COVER reduction, what does the universe represent, and why is it defined this way?
Answer
(a) Vertex cover size . The complement has size at most 3 and forms a vertex cover.
(b) FALSE. means “if is easy, then is easy.” It says nothing about what happens when is easy. Counterexample: Let be any trivial problem (solvable in ), and be any hard problem. We can reduce to , but might not be polynomial-time solvable.
(c) The universe represents the edges of the graph . Each edge is labeled . A vertex cover must “touch” every edge. By encoding edges as universe elements, “covering the universe” corresponds exactly to “every edge being touched by some chosen vertex.”