📚 L9: Reductions & Intractability

Lecture Goal

Understand how reductions formalize “problem A is no harder than problem B”, use reductions to prove hardness, and see the chain that connects 3-SAT, Independent Set, Vertex Cover, and Set Cover.

The Big Picture

Reductions are the universal language of hardness. Instead of proving each problem hard from scratch, we show a known-hard problem reduces to it — and hardness “flows” through the reduction chain. This is how we know thousands of problems are all equivalent in difficulty to 3-SAT.


1. What is a Reduction?

🤔 The Core Problem

Instead of learning a new algorithm for every problem, can we transform one problem into another we already know how to solve?

💡 The Reduction Concept

A reduction is a way to solve problem by transforming it into a problem that you already know how to solve.

flowchart LR
    A["Instance α<br/>(Problem A)"] -->|"Convert"| B["Instance β<br/>(Problem B)"]
    B -->|"Solve B"| S["Solution B(β)"]
    S -->|"Translate back"| R["Solution A(α) ✅"]

    style A fill:#e1f5fe
    style B fill:#fff9c4
    style S fill:#c8e6c9
    style R fill:#c8e6c9

Formally, to reduce to :

  1. Take an instance of problem
  2. Convert into an instance of problem (this is the reduction step)
  3. Solve on
  4. Translate ‘s answer back into an answer for

We write this as: , read as “A reduces to B” or “A is no harder than B”.


📖 Key Terminology

TermDefinition
InstanceA specific input to a problem (sometimes called “encoding”)
ReductionA transformation from instances of to instances of
reduces to ” — solving gives us a way to solve

💡 Warm-Up Example: T-SUM reduces to 0-SUM

  • 0-SUM: Given array , find such that
  • T-SUM: Given array and number , find such that
flowchart LR
    B["Array B, Target T<br/>(T-SUM instance)"] -->|"Define A[i] = B[i] - T/2"| A["Array A<br/>(0-SUM instance)"]
    A -->|"Solve 0-SUM"| Sol["Find i,j: A[i] + A[j] = 0"]
    Sol -->|"Same i,j"| Ans["B[i] + B[j] = T ✅"]

    style B fill:#e1f5fe
    style A fill:#fff9c4
    style Sol fill:#c8e6c9

Reduction: Given for T-SUM, define .

Then . ✅


⚠️ Common Pitfalls

Pitfall 1: Confusing the direction of the reduction

means you are solving A using B. It does NOT mean is easier than — it means is at least as hard as . Students often read the arrow backwards!

Pitfall 2: Forgetting the translation step

You need both directions: converting AND converting . Forgetting the back-translation is a common error in proofs.


❓ Mock Exam Questions

Q1: Reduction Direction

If we reduce Problem A to Problem B, which problem’s algorithm do we use?

Answer

Answer: We use Problem B’s algorithm

means we transform A-instances into B-instances and solve using B’s algorithm.

Q2: Arrow Meaning

What does mean?

Answer

Answer: A is no harder than B (or equivalently, B is at least as hard as A)

The arrow points from easier to harder. If B is solvable, so is A.


2. p(n)-Time Reductions & Running Time Composition

🧠 Core Concept

A -time reduction from to means:

  • The conversion runs in time
  • The back-translation runs in time

where is the size of the input (NOT the size of ‘s output).

flowchart TD
    subgraph Reduction ["Reduction Pipeline"]
        A["Input α (size n)"] -->|"Convert<br/>O(p(n))"| B["Instance β<br/>(size O(p(n)))"]
        B -->|"Solve B<br/>T(O(p(n)))"| S["Solution B(β)"]
        S -->|"Translate back<br/>O(p(n))"| R["Solution A(α)"]
    end

    Total["Total Time: T(O(p(n))) + O(p(n))"]

    style Reduction fill:#e1f5fe
    style Total fill:#c8e6c9

📐 Running Time Composition

If there is a -time reduction from to , and has a -time algorithm, then can be solved in:

Why? The instance has size , so solving on it takes time. Add the for conversion and back-translation.


📖 Key Terminology

TermDefinition
-time reductionReduction where both conversion and back-translation run in time
Input size The size of the original instance not the size of
Running Time CompositionHow to combine reduction cost and -solver cost to get total runtime for

⚠️ Common Pitfalls

Confusing (size of ) with (size of constructed instance)

In the MAT-MULTI → MAT-SQR example: the input size (number of entries in two matrices). The conversion constructs a matrix — work. So it’s an -time reduction, not !


3. Polynomial-Time Reductions & Pseudo-Polynomial Algorithms

🧠 Core Concept

Definition: (polynomial-time reduction) if there is a -time reduction from to where for some constant .

flowchart LR
    subgraph Implications ["Key Implications"]
        I1["A ≤_P B and B ∈ P<br/>⇒ A ∈ P"]
        I2["A ≤_P B and A hard<br/>⇒ B hard"]
    end

    Hard["Hard Problem A"] -->|"≤_P"| B["Problem B"]
    Hard --> I2
    B --> I1

    style I1 fill:#c8e6c9
    style I2 fill:#ffccbc
    style Hard fill:#ff9999

Key implication:

Contrapositive (crucial for hardness!):

Think of it like a disease spreading upward in difficulty: if is hard and reduces to , then “inherits” the hardness of . 🦠

Why polynomial time? The notion is robust: multiplying two polynomials gives another polynomial. So chaining reductions still gives .


📊 Pseudo-Polynomial Algorithms

An algorithm is pseudo-polynomial if it runs in time polynomial in the numeric value of the input, but exponential in the length (number of bits) of the input.

flowchart LR
    subgraph KNAPSACK ["KNAPSACK Example"]
        DP["DP runs in O(nW)"]
        Input["Input: O(n log W) bits"]
        W["W = O(2^(log W))"]
    end

    DP -->|"Polynomial in W"| Pseudo["Pseudo-polynomial ❌"]
    Input -->|"True input size"| Exp["O(nW) = O(n · 2^(log W))<br/>Exponential!"]

    style Pseudo fill:#ffccbc
    style Exp fill:#ff9999

Classic example: Dynamic programming for KNAPSACK runs in .

  • The input size is bits
  • But can be as large as
  • So is exponential in the input length → pseudo-polynomial, NOT truly polynomial ❌

Compare to FRACTIONAL KNAPSACK with : this IS polynomial ✅


📖 Key Terminology

TermDefinition
polynomial-time reduces to
Polynomial-time algorithmRuns in for some constant , where = input encoding length
Pseudo-polynomialPolynomial in the numeric value but exponential in the bit length
TractableSolvable in polynomial time
IntractableNo known polynomial-time algorithm (and strong evidence none exists)

⚠️ Common Pitfalls

Thinking is polynomial for KNAPSACK

is encoded in bits. So the actual input size is , meaning . The runtime is exponential! Always measure polynomial-ness against the encoding length, not the numeric value.

Believing and poly-time poly-time

This is FALSE. The implication only goes the other way: poly-time poly-time. And: hard hard.


❓ Mock Exam Questions

Q3: Implication Direction

If and can be solved in polynomial time, what can we conclude?

Answer

Answer: Nothing about B’s complexity

The reduction only tells us: “if B is easy, then A is easy” (and contrapositively: “if A is hard, then B is hard”). It says nothing about B when A is easy.

Q4: Pseudo-polynomial Classification

The KNAPSACK DP runs in time. Input encoded in bits. What is this?

Answer

Answer: Pseudo-polynomial

It’s polynomial in the numeric value , but , making it exponential in the actual input length.


4. Decision Problems vs Optimisation Problems

🧠 Core Concept

To compare problem hardness uniformly, we use a common language: decision problems.

flowchart LR
    subgraph Optimisation ["Optimisation Problem"]
        Opt["Find the BEST solution"]
        Val["Output: A value/solution"]
    end

    subgraph Decision ["Decision Problem"]
        Dec["Is there a solution<br/>with value ≥/≤ k?"]
        Ans["Output: YES or NO"]
    end

    Opt -->|"Add threshold k"| Dec
    Dec -->|"If YES for optimal k,<br/>you found the optimum"| Opt

    style Decision fill:#c8e6c9
TypeOutputExample
OptimisationA value/solution”What is the shortest path from to ?”
DecisionYES or NO”Is there a path from to of length ?”

Conversion recipe: Given an optimisation problem, create a decision version by introducing a threshold and asking “is there a solution with value ?” (for minimisation) or ”?” (for maximisation).

Why does this work for hardness?
The decision version is no harder than the optimisation version:

  • If you can solve optimisation, just check if the optimal value satisfies the threshold.
  • So: if the decision version is hard → the optimisation version is also hard.

📖 Key Terminology

TermDefinition
Decision problemMaps an instance to {YES, NO}
YES-instanceAn input for which the answer is YES
NO-instanceAn input for which the answer is NO
Threshold Parameter added to convert optimisation to decision

⚠️ Common Pitfalls

Thinking the decision problem is always strictly easier

Decision Optimisation (decision is no harder). But in many cases they are equivalent in hardness — if you can solve the decision version for all , you can binary search for the optimum.


5. Reductions Between Decision Problems

🧠 Core Concept

For decision problems, a poly-time reduction is a poly-time transformation such that:

The YES/NO answer is “copied” — no translation needed, just the if-and-only-if.

To prove a valid reduction, show three things:

  1. ✅ The transformation runs in polynomial time
  2. () If is a YES-instance of , then is a YES-instance of
  3. () If is a YES-instance of , then is a YES-instance of

Both directions of the if-and-only-if must be proved! 🔁


⚠️ Common Pitfalls

Proving only one direction of the iff

Many students prove () but forget (), or vice versa. Both are required and examiners will look for both.

Confusing with

These are very different statements! means ” is no harder than .” Always be explicit about which direction you’re reducing.


6. The Problem Zoo

🧠 Key Problems

🕸️ INDEPENDENT-SET

Problem: Given a graph and integer , is there a subset with such that no two vertices in are adjacent?

flowchart TD
    subgraph Graph ["Graph G"]
        V1["●"] --- V2["●"]
        V1 --- V3["●"]
        V2 --- V4["●"]
        V3 --- V4
        V4 --- V5["●"]
    end

    IS["Independent Set S<br/>(no edges between members)"]
    Size["|S| ≥ k?"]

    style V1 fill:#c8e6c9
    style V3 fill:#c8e6c9
    style V5 fill:#c8e6c9

Analogy: A group of people at a party where nobody knows each other — they’re “independent.” 🥳

🛡️ VERTEX-COVER

Problem: Given a graph and integer , is there a subset with such that every edge has at least one endpoint in ?

Analogy: Place security cameras at intersections so that every road is watched by at least one camera. 📷

🧩 SET-COVER

Problem: Given integers , , and a collection of subsets of , are there subsets in whose union is ?

Analogy: Choose radio stations so that every city is in at least one station’s broadcast range. 📻

🔮 3-SAT

flowchart LR
    subgraph Building ["Building Blocks"]
        Lit["Literal: xᵢ or x̄ᵢ"]
        Clause["Clause: (x₁ ∨ x₂ ∨ x̄₃)"]
        CNF["CNF: AND of clauses"]
    end

    Problem["Does Φ have a<br/>satisfying assignment?"]

    Lit --> Clause --> CNF --> Problem

    style Problem fill:#fff9c4

Building blocks:

  • Literal: A Boolean variable or its negation
  • Clause: A disjunction (OR) of literals, e.g.,
  • CNF formula: A conjunction (AND) of clauses

Problem: Given a CNF formula where every clause has exactly 3 literals, does have a satisfying truth assignment?

Analogy: Can you set switches (True/False) so that in every group of 3 switches, at least one is ON? 💡


📖 Key Terminology Summary

ProblemInputYES iff…
INDEPENDENT-SET subset of mutually non-adjacent vertices
VERTEX-COVER subset of vertices touching all edges
SET-COVER subsets whose union
3-SATCNF formula truth assignment satisfying all clauses

⚠️ Common Pitfalls

Confusing Independent Set and Vertex Cover

Independent Set wants vertices with NO edges between them (size ). Vertex Cover wants vertices that TOUCH all edges (size ). Note the complementary nature — this is exactly the key insight in their reduction!


7. The Reduction Chain

The lecture builds the following reduction chain:

flowchart LR
    SAT["3-SAT"] -->|"≤_P"| IS["INDEPENDENT-SET"]
    IS -->|"≤_P"| VC["VERTEX-COVER"]
    VC -->|"≤_P"| SC["SET-COVER"]

    Hard["All believed HARD"] --> SAT

    style SAT fill:#ff9999
    style IS fill:#ffccbc
    style VC fill:#ffccbc
    style SC fill:#ffccbc

By transitivity: . Since 3-SAT is believed to be hard, all problems in this chain are believed to be hard! 😱


7.1 INDEPENDENT-SET VERTEX-COVER

flowchart LR
    subgraph KeyInsight ["Key Insight"]
        IS["S is Independent Set"]
        VC["V \\ S is Vertex Cover"]
    end

    IS <--> VC

    subgraph Reduction ["Reduction"]
        Input["(G, k) for IS"]
        Output["(G, n-k) for VC"]
    end

    Input -->|"Complement"| Output

    style KeyInsight fill:#e1f5fe

Key Insight: is an independent set in is a vertex cover.

Reduction: is a YES-instance of INDEPENDENT-SET is a YES-instance of VERTEX-COVER (where ).

Proof ():
Suppose , , is an independent set. For any edge : since is independent, at least one of , so at least one of them is in . Thus is a vertex cover of size . ✅

Proof ():
Suppose , , is a vertex cover. If and , then neither endpoint is in , contradicting being a vertex cover. So is an independent set of size . ✅


7.2 VERTEX-COVER SET-COVER

flowchart TD
    subgraph Graph ["Graph G = (V, E)"]
        E1["Edge e₁"]
        E2["Edge e₂"]
        E3["..."]
    end

    subgraph SetCover ["Set Cover Instance"]
        U["Universe U = {1, 2, ..., |E|}<br/>(edge indices)"]
        Sets["For each vertex v:<br/>Sᵥ = {i : eᵢ incident on v}"]
        K["k' = k"]
    end

    Graph -->|"Edges become<br/>universe elements"| SetCover

    style Graph fill:#e1f5fe
    style SetCover fill:#fff9c4

Reduction: Given for VERTEX-COVER, construct:

  • Universe (label edges )
  • For each vertex :
  • Set

Intuition: Each vertex “covers” the edges incident to it. Choosing a vertex cover of size corresponds to choosing sets that together cover all edge indices.

Proof ():
If is a vertex cover of size , then every edge has an endpoint in , so . ✅

Proof ():
If () form a set cover, then every edge index is covered, meaning every edge has an endpoint in — a vertex cover of size . ✅


7.3 3-SAT INDEPENDENT-SET

flowchart TD
    subgraph Construction ["Construction from 3-SAT Φ with m clauses"]
        C1["Clause 1<br/>(x₁ ∨ x̄₂ ∨ x₃)"]
        C2["Clause 2<br/>(x̄₁ ∨ x₂ ∨ x₄)"]
        C3["..."]
        Cm["Clause m"]
    end

    subgraph Graph ["Independent Set Graph"]
        T1["Triangle: 3 literals<br/>per clause"]
        T2["Triangle"]
        Conflict["Conflict edges: connect<br/>xᵢ to x̄ᵢ"]
    end

    C1 --> T1
    C2 --> T2
    Construction -->|"k = m"| Graph

    style T1 fill:#fff9c4
    style T2 fill:#fff9c4
    style Conflict fill:#ff9999

Reduction: Given 3-SAT instance with clauses:

  1. Create 3 vertices per clause (one per literal) — label them with the literal
  2. Connect the 3 literals in each clause in a triangle (intra-clause edges)
  3. Connect each literal to all occurrences of its negation in other clauses (conflict edges)
  4. Set (number of clauses)

Why triangles? An independent set can pick at most 1 vertex from each triangle — forcing us to pick exactly one literal per clause to satisfy.

Why conflict edges? These prevent us from picking from one clause and from another — they can’t both be true!

Proof ():
Given a satisfying assignment, pick one true literal per clause. These vertices form an independent set:

  • No two are in the same triangle (one per clause) ✅
  • No two are connected by a conflict edge (can’t have and both true) ✅

Proof ():
Given an independent set of size : exactly one vertex from each triangle is in . Set those literals to true. This is consistent (conflict edges prevent and both being chosen), and every clause is satisfied. ✅


⚠️ Common Pitfalls

3-SAT → IS: Forgetting conflict edges

Without conflict edges, you could pick from one triangle and from another — a contradiction. The conflict edges are essential for correctness.

IS → VC: Thinking the complement has size exactly

If , then . The inequality goes the right way — don’t flip it!

VC → Set Cover: The universe is the set of EDGES, not vertices

Students often confuse what the universe represents. It encodes edges, not vertices.


8. NP-Completeness (Preview)

🧠 Core Concept

All the problems seen (3-SAT, INDEPENDENT-SET, VERTEX-COVER, SET-COVER, KNAPSACK, etc.) are NP-complete:

flowchart TD
    subgraph NPC ["NP-Complete Problems"]
        SAT["3-SAT"]
        IS["INDEPENDENT-SET"]
        VC["VERTEX-COVER"]
        SC["SET-COVER"]
        KS["KNAPSACK"]
        etc["...thousands more"]
    end

    Implication["If ANY ONE has a poly-time algorithm,<br/>then ALL of them do<br/>⇒ P = NP"]

    SAT --> Implication

    style NPC fill:#ffccbc
    style Implication fill:#fff9c4

If any one of these NP-complete problems has a polynomial-time algorithm, then all of them do — and we would have .

No polynomial-time algorithm is known for any of them. Most computer scientists believe , meaning these problems are genuinely intractable. 💀

The web of reductions you’ve seen (Dick Karp’s famous 1972 paper!) shows that these seemingly unrelated problems from graph theory, logic, and combinatorics are all equivalent in hardness.


🔗 Connections

📚 Lecture🔗 Connection
L10Using reductions to prove NP-completeness

The Key Insight

Reductions let us stand on the shoulders of giants. We don’t need to prove hardness from scratch — we just show our problem is “at least as hard” as a known-hard problem. The entire NP-completeness web rests on this powerful idea. 🕸️


9. Mock Exam Questions

📝 Multiple Choice

Q1: Reduction Implication

Suppose . Which of the following statements is definitely true?

Answer

Answer: (b) If can be solved in polynomial time, then so can .

means “if is easy, then is easy.” It does NOT imply the reverse.

Q2: KNAPSACK Classification

The dynamic programming algorithm for KNAPSACK runs in time. The input is encoded using bits. Which correctly classifies this algorithm?

Answer

Answer: (c) Pseudo-polynomial, because it is polynomial in the numeric value of but exponential in the bit length of .

, so is exponential in the encoding length.

Q3: Conflict Edges Purpose

In the 3-SAT INDEPENDENT-SET reduction, what is the purpose of connecting a literal vertex to its negation vertex in another clause?

Answer

Answer: (b) To ensure we don’t simultaneously set a variable to both TRUE and FALSE.

Intra-clause triangles enforce “at most one per clause.” Conflict edges enforce consistency of the truth assignment across clauses.


✍️ Open-Ended Practice Problems

P1: SUBSET-SUM ≤_P ZERO-SUBSET-SUM

Let SUBSET-SUM be: Given set and target , does there exist a subset such that ?

Define ZERO-SUBSET-SUM: Given a set of integers , does there exist a non-empty subset such that ?

Show that SUBSET-SUM ZERO-SUBSET-SUM. Clearly describe the reduction and prove correctness in both directions.

Answer

Reduction: Given for SUBSET-SUM, construct for ZERO-SUBSET-SUM.

Runtime: — clearly polynomial time. ✅

Correctness (): If is YES, there exists with . Then has sum . ✅

Correctness (): If is YES for ZERO-SUBSET-SUM, there exists non-empty with sum 0.

  • If : Then has sum . ✅
  • If : Then has sum 0, meaning (trivially YES for SUBSET-SUM). ✅

P2: INDEPENDENT-SET → VERTEX-COVER

(a) Given a graph with 8 vertices and an independent set of size 5, what is the minimum size of a vertex cover guaranteed by the reduction?

(b) Suppose someone claims: “If and can be solved in polynomial time, then can be solved in polynomial time.” Is this TRUE or FALSE? Justify.

(c) In the VERTEX-COVER SET-COVER reduction, what does the universe represent, and why is it defined this way?

Answer

(a) Vertex cover size . The complement has size at most 3 and forms a vertex cover.

(b) FALSE. means “if is easy, then is easy.” It says nothing about what happens when is easy. Counterexample: Let be any trivial problem (solvable in ), and be any hard problem. We can reduce to , but might not be polynomial-time solvable.

(c) The universe represents the edges of the graph . Each edge is labeled . A vertex cover must “touch” every edge. By encoding edges as universe elements, “covering the universe” corresponds exactly to “every edge being touched by some chosen vertex.”