CSC153, Class 20: More Algorithm Analysis

Overview:
* Review: Big-O Notation
* Doing Big-O Analysis
* Dominant terms and other relationships
* Some recurrence relations
* Experimental analysis

Notes:
* Exam 1 due.
* Homework 2 ready.  Two fun Web services.  
  Use higher-order stuff.
* I'll be gone this weekend.  I'll try to answer any email.

----------------------------------------

Does anyone remember what Big-O analysis is?
* A way to examine how many steps a program takes.
  + Approximates 
  + As a function
  + Giving an upper bound
  + Ignores constant multipliers
* Formal definition
  f(n) is in O(g(n)) if and only if
    there exist n0, d > 0 
    such that
    for all n > n0 |f(n)| <= d*g(n)

How do we do Big O analysis?
  + Count each "basic" operation as one step.
  + Repetition of a series of steps
    - Count the cost of the series of steps
    - Count the number of repetitions
    - Multiply
  + Choice between multiple cases
    - Count each case
    - Take the longest
  + Procedure/subroutine call
    - Number of steps in the subroutine
How long does the Mathematical exponentiation algorithm take?
* It depends on how long the functions it depends on take

Some analyses the formal steps help with

Suppose f(n) is in O(g(n)) and h(n) is in O(k(n))

What bound can we give for f(n)+h(n)?
+ Context: 
  I've bounded the first half of my algorithm
  I've bounded the second half of my algorithm
  I do the two halves in sequence
  What's the bound?  
  O(g(n)+k(n))

Proof:
  f(n) is in O(g(n)) implies
    there exist n00, d0 > 0 
    such that
    for all n > n00 |f(n)| <= d0*g(n)
  h(n) is in O(k(n)) implies
    there exist n01, d1 > 0 
    such that
    for all n > n01 |h(n)| <= d1*k(n)

Detour:
  How do we show that (f(n)+h(n)) is in O(g(n)+k(n))
    find n0, d > 0 
    such that
    for all n > n0 |f(n)+h(n)| <= d*(g(n)+h(n))

n0 = max(n00,n01)
d = max(d0,d1)

  |f(n)+h(n)| <= |f(n)| + |h(n)|
              <= d0*g(n) + d1*k(n)  // For n > n0
              <= d*g(n) + d*k(n)
              <= d*(g(n)+k(n))
Quod.Erat.Demonstratum.
+-+
| |
+-+

----------------------------------------

f(n) is in O(g(n)) h(n) is in O(k(n))

f(n)*h(n) is in O(g(n)*h(n))
* Bounding loops

f(n) is in O(g(n)), g(n) is in O(h(n))
  f(n) relative to h(n)?
  + If I wrote, f <= g and g <= h, you'd conclude f <= h
  + Similar here: f(n) is in O(h(n))

f(n) is in O(k*g(n)) where k is a constant greater than 0
  f(n) is in O(g(n))

  Proof: 
    exist n0, dold s.t. f(n) <= dold*k*g(n) for all n > n0
      [By definition of Big O
    let dnew be dold*k
    Now
      f(n) <= dnew*g(n) = dold*k*g(n)
  WE HAVE PROVEN THAT YOU CAN IGNORE CONSTANT MULTIPLIERS

f(n) is in O(g(n)+h(n)) and h(n) is in O(g(n))
  g(n)+h(n) is in O(2*g(n))
  f(n) is in O(g(n))
WHEN YOU HAVE O(TWO FUNCTIONS), YOU CAN THROW AWAY THE SMALLER

  f(n) is O(n^2 + 432n)
  f(n) is O(n^2)

----------------------------------------
So how do we analyze recursive functions?

Let's analyze length

  (define length (lst)
    (if (null? lst) 0
        (+ 1 (length (cdr lst)))))

When there's a test,
  analyze both parts
  add 1 to largest
    computing 0 takes 1 step
    computing (+ 1 (length (cdr lst))) takes
      adding 1 takes one step
      taking the cdr takes one step
      computing (length ...) takes ??? steps

We have a recursive definition of the running time of
length

  t(0) = 2
  t(n) = 3 + t(n-1)

How Sam analyzes them when he forgets the details

  Expand repeatedly until you see a pattern.

  t(n) = 3 + t(n-1)
       = 3 + 3 + t(n-2) = 2*3 + t(n-2)
       = 2*3 + 3 + t(n-3) = 3*3 + t(n-3)
       = 3*3 + 3 + t(n-4) = 4*3 + t(n-4)
       = k*3 + t(n-k)
  Eventually, n-k must be 0 (when n=k)
       = n*3 + t(0) = n*3 + 2 is in O(n)

  Suppose
    t(0) = b
    t(n) = c*t(n-1)

  t(n) = c*t(n-1)
       = c*c * t(n-2) = c^2 * t(n-2)
       = c*c^2 * t(n-3) = c^3 * t(n-3)

  t(n) = b*c^n  = O(c^n) = O(2^n)

  t(n) = c + 2*t(n-1)

-----

  Suppose
    t(0) = 0
    t(n) = n + t(n-1)

 n + n-1 + n-2 .... + 3 + 2 + 1 = n(n+1)/2 is in O(n^2)