CSC297.Java 2003F: Asymptotic Analysis, Continued

Homework:
* Prove
  + n^2 + 3n is in O(n^2)
  + Ignore constants: d*f(n) is in O(f(n))
  + Transitivity: If f(n) is in O(g(n)) and g(n) is in O(h(n)), then f(n) is in O(h(n))
  + Ignore lower order terms: If f(n) is in O(g(n)) then g(n)+f(n) is in O(g(n))

Overview:
* Review
* Best, Worst, Average?
* Analyzing iterative functions to determine Big O
* Analying recursive functions to determine Big O
* One of the proofs

What is Big-O?
* The "order" of a function.
* Formal definition:
  f(n) is in O(g(n)) iff there exists c and n0 > 0 s.t.
    |c*f(n)| <= g(n) for all n > n0

How do computer scientists use Big O?
* As a mechanism for discussing the running time of a procedure or program.
* We say that the running time of the procedure or program is bounded above (ignoring constants) by a particular function and use Big O to note that function.
* We try to find a function that closely models the behavior.

If you believe in transitivity, there are lots of functions you can pick for the big O

  f(n) = n
  f(n) is in O(n)
  f(n) is in O(n^2)
  f(n) is in O(n^1023)
  f(n) is in O(2^n)

Informal agreement that we try to find a "tight" version.

There are variants that require tighter bounds (little-o for lower bound, Theta for exact bound)

When analyzing a procedure, what should you look at?  Best, worst, or average?
* Average: Most of your cases will be around that.
* Worst: Good safeguard.  Keeps clients happier.  Easier.

How do we analyze procedures for their running time?
* We use different strategies for iterative and recursive procedures

Iterative procedures have the following components
* Basic operations
  + All take O(1) (some constant number of steps)
* Conditionals
  + Form: if X then Y else Z
  + Time: Time for X + worst of time of Y or Z
* Sequencing
  + Form: X then Y
  + Time for X plus time for Y
* Repetition
  + Form: while TEST do BODY
  + Time: Worst case amount of time for BODY * Maximum number of repetitions

Consider insertion sort
	Procedure: sort
	Input: lst, a list of N numbers
	Output: sorted, a sorted list of N numbers
	Begin
	  sorted = new List(); 				1
	  while (!lst.isEmpty()) {			n repetitions
	    insertAtCorrectPlace(sorted,lst.car());	n steps
	    lst.deleteFirst();				1
	  }
	  return sorted;				1

	O(n*(n+1) + 2) = O(n^2)

	Procedure: insertAtCorrectPlace
	Input: sorted, a sorted list of N numbers; val, a value
	Output: None, modifies sorted
	Begin
	  Find the correct place for val in sorted. 	n-1
	  Put val in that place.			1

	Procedure Binary Search
	Input: sorted, a sorted array of N numbers; val, a value
	Output: the position of val within sorted or -1 o/w
	Begin
	  If the array is empty, return -1
	  Look at the middle.  If it's val, return the position.	1
	  O/w, if val is less than the middle, search in the left half
	    Split in half
	    Look in the half
	  O/w, if val is greater than the middle, search in the right half

	
	Note that this is recursive.  
	How do we determine the running time of binary search if it depends on the running time of binary search?  
	Solution: Define a function, so that life is more explicit.

	t(n) is the running time of binary search
		n is the size of the array being searched

	t(0) = c
	t(1) = d
	t(n) = 2 + t(n/2)

	The problem of taking functions defined in terms of themselves and rewriting them as functions not defined in terms of themselves is part of the mathematical technique known as "recurrence analysis"

	How does Sam do recurrence analysis?  Expand terms and generalize

	t(n) = 2 + t(n/2)
	     = 2 + 2 + t(n/4) 
	     = 2*2 + t(n/2^2)
	     = 2*2 + 2 + t(n/2^3)
	     = 3*2 + t(n/2^3)
	     = 3*2 + 2 + t(n/2^4)
	     = 4*2 + t(n/2^4)
	     = k*2 + t(n/2^k)
		if we find a k0 s.t. n = 2^k0
	     = k0*2 + d

		log2n = log2(2^k0) ... log2n = k0

	     = log2n * 2 + d