Divide and Conquer Optimization in Dynamic Programming

Dynamic programming (DP) is arguably the most important tool in a competitive programmer’s repertoire. There are several optimizations in DP that reduce the time complexity of standard DP procedures by a linear factor or more, such as Knuth’s optimization, Divide and Conquer optimization, the Convex Hull Trick, etc. They are, of paramount importance for advanced competitive programming, such as at the level of olympiads. In this article, we will discover the divide and conquer optimization, NOT to be confused with the divide and conquer algorithm to solve problems. 

Divide and Conquer Optimization Criteria:

The divide and conquer optimization can be used for problems with a dp transition of the following form – 

dp[i][j] = min_1≤k<j (dp[i-1][k-1] + cost[k][j])

Further, the cost function must satisfy the quadrangle inequality (QI), i.e., 

cost(a, c) + cost(b, d) ≤ cost(a, d) + cost(b, c) for all a ≤ b ≤ c ≤ d.

Divide and Conquer Optimization Technique:

The sub-optimal approach to solve any problem with a dynamic programming transition of the form given above would iterate through all possible values of k < j for each transition. Then, if the problem constraints give 1 ≤ i ≤ m and 1 ≤ j ≤ n, the algorithm will take O(mn²) time. 

The key to the optimization is the following:

• Like in Knuth’s optimization, define the function opt(i, j), the minimum (or maximum, doesn’t matter) value of k for which dp[i][j] takes its minimum value. Then, we have the following relation: 

opt[i][j] ≤ opt[i][j+1], where

opt[i][j] = argmin_k<j(dp[i-1][k] + cost[k][j])

Now, suppose we compute opt[i][j] for some i and j. Then, we also know that opt[i][p] ≤ opt[i][j] for all p < j. The sub-optimal solution would involve looping for each j, through all possible values of k for any fixed i. The optimization itself is as follows: 

• Loop through the values of i, and first compute dp[i][j] and opt[i][j] for j = n/2, for the current i.  This is possible as at the time of processing, we know all the values in the dp table for dp[i-1][k] for all k ≤ n, due to the structure of the loop.
• Now, calculate dp[i][n/4] and dp[i][3n/4], knowing that opt[i][n/4] ≤ opt[i][n/2] and opt[i][n/2] ≤ opt[i][3n/4]. 
• We recursively call this solve function, keeping track of the lower and upper bounds for opt[i][j] for some i and the current j. For instance, when calculating dp[i][5n/8], we know that opt[i][n/2] ≤ opt[i][5n/8] ≤ opt[i][3n/4]. 

The algorithm is faster by a linear factor as we don’t have to loop through all values of k, and a logarithmic factor is added due to the recursive nature of this algorithm. The time complexity is thus O(m * n * (log n)).

The generic code for this approach is given below It uses a recursive approach, which is the simplest to implement given the structure of the solution.

Time Complexity: O(M*N²*log₂N).

Space Complexity: O(M*N) as 2d array dp has been created.

Proof of Correctness of Divide and Conquer Optimization:

To prove the correctness of this algorithm, we only need to prove the inequality –

opt[i][j] ≤ opt[i][j+1] 

Follow the below section for proof of correctness:

Assumptions: If cost(i, j) satisfies the Quadrangle Inequality, then dp[i][j] also satisfies the inequality.  
Now, consider the following setup – 

• We have some indices 1 ≤ p ≤ q ≤ j and a separate fixed i. 
• Let dp_k[i][j] = cost[k][i] + dp[k-1][j-1]. 

If we can show that – 

dp_p[i][j] ≥ dp_q[i][j] ⇒ dp_p[i][j+1] ≥ dp_q[i][j+1] 

then setting q = opt[i][j], it will be clear that opt[i][j+1] will be at least as much as opt[i][j], due to the implication of the above inequality for all indices p less than opt[i][j]. This will prove that opt[i][j-1] ≤ opt[i][j].

Prrof:

From the Quadrangle inequality on the dp array we get –

cost(p, j) + cost(q, j+1) ≤ cost(p, j+1) + cost(q, j)
⇒ (dp[i-1, p] + cost(p, j)) + (dp[i-1, q] + cost(q, j+1)) ≤ (dp[i-1, p] + cost(p, j+1)) + (dp[j-1, q] + cost(q, j))
⇒ dp_p[i][j] + dp_q[i][j+1] ≤ dp_p[i][j+1] + dp_q[i][j]
⇒ dp_p[i][j] – dp_q[i][j] ≤ dp_p[i][j+1] – dp_q[i][j+1]

dp_p[i][j] ≥ dp_q[i][j] 
⇒ 0 ≤ dp_p[i][j] – dp_q[i][j] ≤ dp_p[i][j+1] – dp_q[i][j+1] 
⇒ dp_p[i][j+1] ≥ dp_q[i][j+1] 

This completes the proof dp_p[i][j] ≥ dp_q[i][j] ⇒ dp_p[i][j+1] ≥ dp_q[i][j+1]

Examples to Show Working of Divide and Conquer Optimization:

Given an array arr[] of N elements, the task is to divide it into K subarrays, such that the sum of the squares of the subarray sums is minimized.

Examples:

Input: arr []= {1, 3, 2, 6, 7, 4}, K = 3. 
Output: 193
Explanation: The optimal division into subarrays is [1, 3, 2], [6] and [7, 4], 
Giving a total sum of (1 + 3 + 2)² + (6)² + (7 + 4)² = 193.  
This is the minimum possible sum for this particular array and K.

Input: arr[] = {1, 4, 2, 3}, K = 2
Output: 50
Explanation: Divide it into subarrays {1, 4} and {2, 3}.
The sum is (1+4)² + (2 + 3)² = 5² + 5² = 50.
This is the minimum possible sum.

Suboptimal solution: The problem can be solved based on the following idea:

• If first j-1 elements are divided into i-1 groups then the minimum cost of dividing first j elements into i groups is the same as the minimum value among all possible combination of dividing first k-1 (i ≤ k ≤ j) elements into i-1 groups and the cost of the ith group formed by taking elements from k^th to j^th indices.
• Let dp[i][j] be the minimum sum obtainable by dividing the first j elements into i subarrays. 
  So the dp-transition will be – 

dp[i][j] = min_i≤k≤j (dp[i-1][k-1] + cost[k][i])

where cost[k][i] denotes the square of the sum of all elements in the subarray arr[k, k+1 . . . i]

Follow the steps mentioned below for solving the problem:

• The cost function can be calculated in constant time by preprocessing using a prefix sum array:
  • Calculate prefix sum (say stored in pref[] array).
  • So cost(i, j) can be calculated as (pref[j] – pref[i-1]).
• Traverse from i = 1 to M:
  • Traverse from j = i to N:
  • Traverse using k and form the dp[][] table using the above dp observation.
• The value at dp[M-1][N-1] is the required answer. 

Below is the implementation of the above approach. 

193

Time Complexity: O(M * N²)
Auxiliary Space: O(M * N)

Optimal Solution (Using Divide and Conquer Optimization):

This problem follows the quadrangle We can, however, notice that the cost function satisfies the quadrangle inequality

cost(a, c) + cost(b, d) ≤ cost(a, d) + cost(b, c). 

The following is the proof: 

Let sum(p, q) denote the sum of values in range [p, q] (sub-array of arr[[]), and let x = sum(b, c), y = sum(a, c) − sum(b, c), and z = sum(b, d) − sum(b, c). 

Using this notation, the quadrangle inequality becomes 

(x + y)² + (x + z)² ≤ (x + y + z)² + x², 
which is equivalent to 0 ≤ 2yz. 

Since y and z are nonnegative values, this completes the proof. We can thus use the divide and conquer optimization. 

• There is one more layer of optimization in the space complexity that we can do. To calculate the dp[][] states for a certain value of j, we only need the values of the dp state for j-1. 
• Thus, maintaining 2 arrays of length N and swapping them after the dp[][] array has been filled for the current value of j removes a factor of K from the space complexity. 

Note: This optimization can be used for all implementations of the divide and conquer DP speedup.

Follow the steps mentioned below to implement the idea:

• The cost function can be calculated using prefix sum as in the previous approach.
• Now for each fixed value of i (number of subarrays in which the array is divided):
  • Traverse the whole array to find the minimum possible sum for i divisions.
• The value stored in dp[M%2][N-1] is the required answer.

Below is the implementation of the above approach.

193

Time Complexity: O(M * N * logN)
Auxiliary Space: O(N)

O

omkarbajaj000

Follow

Improve