Class 31: Algorithm Analysis (3)

Held: Wednesday, March 16, 2005

Summary: Today we conclude our initial explorations of algorithm analysis with some further explorations of the notation and some experiments.

Related Pages:

• EBoard.
• Lab.
• Sample Code.

Overview:

• Review
• Lab
• The role of experimentation
• Using the notation: Eliminating constants
• Using the notation: Simplifying running times

Review

• f(n) is in O(g(n)) iff
  • there exists a number n₀
  • there exists a number d > 0
  • for all but finitely many n > n₀, abs(f(n)) <= abs(d*g(n))
• This notation provides a convenient mechanism for talking about running time without worrying about too many potentially useless details.
• We determine the asymptotic running time of an algorithm by "counting steps" for the worst case
  • Sequence of operations: Add 'em up
  • Assignment: 1
  • Method call: Cost of that method
  • Conditional: Worst cost of two branches
  • Loop: Number of repetitions times cost of body

Lab

• Do the lab.
• Be prepared to reflect in the last five or ten minutes.

Experimental Analysis

• Particularly as you start to analyze algorithms, it may be helpful to analyze them experimentally as well as abstractly.
• What do I mean when I say analyze them experimentally? I mean that we can time them on a variety of inputs and graph the results.
• If the experimental and the abstract results match, we can be fairly confident in the abstract results.
• If they don't, we may need to reflect and try again.
  • Our analysis may be correct and our implementation incorrect.
  • Our analysis may be correct and our data may all be outliers.
  • Our analysis may be incorrect.
  • ...
• Note that some analyses can be very difficult.

Eliminating Constants

• One of the nice things about asymptotic analysis is that it makes constants unimportant because they can be hidden in the d.
• If f(n) is 100n seconds and g(n) is 0.5n seconds, then
  • f(n) is in O(g(n)) [let d be 200]
  • g(n) is in f(n) [let d be 1]
• If f(n) is 100n seconds and g(n) is n² seconds, then f(n) is in O(g(n)) [let n₀ be 100 and d be 1]
• However, g(n) is not in O(f(n)). Why not?
  • Suppose there were an n₀ and a d.
  • Consider what happens for n = 101d.
  • d*f(n) = d*100*101*d = d*d*100*101.
  • However, g(n) = d*d*101*101, which is even larger.
  • If n₀ is greater than 101d, we'll still have this problem [proof left to reader].
• Since constants can be eliminated, we normally don't write them.
  • That is, we say that the running time of an algorithm is O(n) or O(n²) or ....

Relating Functions

• Note that some of the counting can be hard, so we may also need to estimate throughout the process.
• We can often express the running time of an algorithm as a composition of other functions.
  • For example, we might have a primary control structure which repeats its contents O(h(n)) times and within that control structure, we have a subalgorithm that takes O(g(n)) time.
  • Similarly, we might break our algorithm up into two independent algorithms, one of which takes O(h(n)) time and one of which takes O(g(n)) time.
  • We might also look at other relationships.
• Here are some questions we might ask ourselves about composition of functions.
  • If f(n) is in O(h(n)) and g(n) is O(k(n)), then what can we say about f(n) + g(n)?
  • If f(n) is in O(h(n)) and g(n) is O(k(n)), then what can we say about f(n) * g(n)?
• We might also ask more general questions about Big-O notation.
  • If f(n) is in O(g(n)) and g(n) is O(h(n)) then what can we say about f(n) and h(n)?
  • If f(n) is in O(kg(n)) and k is a constant, then what else can we say about f(n) and g(n)?
  • If f(n) is in O(g(n) + h(n)) and g(n) is in O(h(n)), then what simpler thing can we say about f(n)?