CSC297.Java, Class 24: Recurrence Relations Revisited

Admin:
* It's probably time to check BishopD's assignments.
* Tell me about your proofs *which you should not be doing at the last moment*.
* We start sorting on Wednesday
* Hw prove:
  * Ignore lower order terms: If f(n) is in O(g(n)) then g(n)+f(n) is in O(g(n))
  * Additivity: If f(n) is in O(g(n)) and h(n) is in O(k(n)) then f(n)+h(n) is in O(g(n) + k(n))  [assume none of these functions take on negative values]
* Hw: Analyze
  * f(n) = c*n + 2*f(n/2)
  * f(n) = c + 2*f(n-1)
  
Overview:
* Proofs
* Recurrence relations, revisited
* Standard recurrences

/Proofs/

Thm: n^2 + 3n is in O(n^2)
Proof:
* Goal is to identify n0 and a d s.t.  d*n^2*(1 + 3/n) <= n^2
* for all n > n0
* Let n0 be 3.
* Let d be 1/2.
* n^2*1/2*(1 + 3/n) <= n^2*1/2*(1+3/3) for all n >= n0
* = n^2 
* QED

Observation
* The goal of this kind of Big-O proof is to identify *a particular* n0 and d
* Why?  Because the statement of Big-O says "There exists n0 and d ..."
* Note that there is a single n0/d pair, but an infinite range of n's.

Ignore constants: d*f(n) is in O(f(n))
* Let n0 be 0, let c be 1/d

Transitivity: If f(n) is in O(g(n)) and g(n) is in O(h(n)), then f(n) is in O(h(n))
* f(n) is in O(g(n)): Exist c1, n1 s.t. |c1*f(n)| <= |g(n)| for all n > n1
* g(n) is in O(h(n)): Exist c2, n2 s.t. |c2*g(n)| <= |h(n)| for all n > n2
* Try one: n0 = 1, c0 = 2
* Fail: Suppose f(n) were 100*n, g(n) were 10*n, and h(n) were n
  * 2*f(n) <= h(n) for all n > 1? No.
* Try again: n0 = , c0 = c1/c2
  * Goal: |c0*f(n)| <= |h(n)| for all n > n0
  * |c0*f(n)| = |c1/c2*f(n)|  = 1/c2*|c1*f(n)| <= |g(n)|/c2  for all n > n1
  * |g(n)|/c2 * c2 = |g(n)|
  * Another failure
* Try again: n0 = max(n1,n2), c0 = c1*c2
  * |c0*f(n)| = |c1*c2*f(n)| 
  * = c2*|c1*f(n)| <= c2*|g(n)|  for all n > n1
  * = |c2*g(n)| for all n > n1
  * <= |h(n)| for all n > n2
  * Success

/Recurrence Relations, Revisited/

* How do you figure out the Big-O for a non-recursive function?
  * Count steps
  * Basic operation: 1
  * Conditional: Cost of test + max cost of the branches
  * Loop: Number of repetitions * cost of each repetition
* How do you figure out the Big-O for a recursive function
  * Similar, but you insert a function for "the cost of this function"
  * The cost of binary search is f(n) = 3 + f(n/2)
  * You then need to figure out what that function is.
* Good computer scientists know a number of common recursively defined functions (at least in terms of Big O)

For all of the following, assume f(1) is 1.

* f(n) = c + f(n/2) 
  * f(n) is in O(log_2(n))
  * Analysis:
    * f(n) = c + f(n/2)
    * f(n) = c + c + f(n/2/2) = c*2 + f(n/2^2)
    * f(n) = c*2 + c + f((n/2^2)/2) = c*3 + f(n/2^3)
    * f(n) = c*k + f(n/2^k) for all 'reasonable' k
    * Suppose f(1) = 1, Let k be log_2(n)
    * f(n) = c*log_2(n) + f(n/2^log_2(n)) 
    * = c*log_2(n) + f(n/n) 
    * = c*log_2(n) + 1
    * in O(log_2(n))

* f(n) = c + f(n-1)
  * Analysis:
    * f(n) = c + c + f(n-2)
    * f(n) = 2*c + f(n-2)
    * f(n) = k*c + f(n-k)
    * When k = n-1, f(n) = (n-1)*c + f(1) = n*c - c + 1 
  * f(n) is in O(n)

* f(n) = c*n + f(n-1)
  * Analysis:
    * f(n) = c*n + c*(n-1) + f(n-2)
    * f(n) = c*(n + n-1) + f(n-2)
    * f(n) = c*(n + n-1) + c(n-2) + f(n-3)
    * f(n) = c*(n + n-1 + n-2) + f(n-3)
    * f(n) = c*(n + n-1 + n-2 + ... + n-(k-1)) + f(n-k)
    * When k = n-1
    * f(n) = c*(n + n-1 + n-2 + ....+ n-((n-1)-1)) + f(1)
    * f(n) = c*(n + n-1 + n-2 + ... + 2) + 1
  * f(n) is in O(n^2)

* f(n) = c*n + f(n/2)
  * Analysis:
    * f(n) = c*n + f(n/2)
    * f(n) = c*n + c*(n/2) + f(n/4)
    * f(n) = c*n + c*(n/2) + c(n/4) + f(n/8)   
    * f(n) = c*(n + n/2 + n/4 + ... n/2^k) + f(n/2^(k+1))
    * Stop when 2^(k+1) = n, , k = log_2(n) [more or less]
    * f(n) = c*n*(1 + 1/2 + 1/4 + .... + 1/n)
  * f(n) is in O(n)

* f(n) = c + 2*f(n/2)
  * Analysis
    * f(n) = c + 2c + 4*f(n/4) = 3*c + 4*f(n/4)
    * f(n) = c + 2c + 4c + 8*f(n/8) = 7*c + 8*f(n/8)
    * f(n) = c + 2c + 4c + 8*f(n/8) = 15*c + 16*f(n/16)
    * f(n) = (k-1)c + k*f(n/k); k is a power of 2
    * Stop when k = n
    * f(n) = (n-1)c + n*1 
  * f(n) is in O(n)