Lecture 5 Dynamic Programming

• Programming => referred to the use of the method to find an optimal program, in the sense of planning or scheduling
• Dynamic => captures the time varying aspect (multi-stage) of the problems

2 key attributes that a problem should have in order for dynamic programming to be applicable

• Optimal Substructure => solution can be obtained by combination of optimal solutions to subproblems, such optimal substructures are usually described recursively
• Overlapping subproblems =>space of subproblems must be small, so the algorithm solving the problem should have same subproblems over and over

Plan

• Memoization and Tabulation
• Money changing problem
• Knapsack Problem
• LCS
• Optimal BST

Fibonacci number

• Divide and Conquer Algorithm

def fib(n):
 if n == 0 or n == 1:
  return 1
 else:
  return fib(n - 1) + fib(n - 2)

• Memoization

table = [0 for i in range(51)]
table[0] = 1
table[1] = 1
def fib(n):
 if table[n] != 0:
  return table[n]
 else:
  table[n] = fib(n - 1) + fib(n - 2)
return table[n]

The technique of using previously computed ( and stored ) values is called memoization.

• Tabulation

Non-recursive implementation

table = [0 for i in range(51)]
def fib(n):
 table[0] = 1
 table[1] = 1
 for i in range(2, n):
  table[i] = table[i - 1] + table[i - 2]
 return table[n]

bottom up approach, performs memoization by a constant factor

In 570 Dynamic programming is tabulation

Memoization => filling up a table recursively in top-down manner

Tabulation => filling up a table non-recursively in bottom up manner

Steps for dynamic programming

• Define subproblems
• Mathematical definition recurrence relation
• write pseudo code how do you fill the the table
• What is the runtime complexity ?

Pseudo polynomial algorithm

• A numerical algorithm runs in pseudo-polynomial time if its run time is polynomial in the numeric value of input but exponential in the length of the input

Longest Common Subsequence

• A subsequence is a subset of elements in the sequence taken in order (with strictly increasing indices).

Motivating examples

• Matching DNA sequences in molecular biology
• Comparing 2 text files (Unix diff) to make sure what changes have been made to the line and display the difference