Fundamentals of Computer Science II (CSC-152 97F)

[News] [Basics] [Syllabus] [Outlines] [Assignments] [Examples] [Readings] [Bailey Docs]

# Outline of Class 18: Algorithm Analysis

## Miscellaneous

• Questions (general and specific) on chapter 4?
• Questions on assignment four?
• Start reading chapter 5, on sorting.
• Don't forget the talk by Johnny Wong on Wednesday.

## Introduction to asymptotic analysis

• In comparing two or more implementations of the same class (or same method), one way to choose which to use might be how fast (or "efficient") the algorithm is.
• Another might be how much memory or other resources the algorithm consumed.
• Suppose we wanted to tell our users how quickly (or slowly) our methods ran. What is a good way to represent such information?
• Surpringly, we often can't provide an exact number. Why not?
• Many methods have conditionals, so different input data will give different run times (and it may be difficult or excessive to precisely explain which data yield which run times).
• Not all of the "built-in" operations of a language take the same amount of time. For example, addition is quicker than multiplication, and integer addition is quicker than floating point addition.
• The same operation may take different times on different machines.
• Noting this problem, computer scientists developed a technique often called asymptotic analysis in which we talk about the general behavior of an algorithm in terms of the size of the input, but don't want to get too much into the details.
• We say that a function f(n) is O(g(n)), which is read "big O of g of n" if
• there exists a number n0
• there exists a number d > 0
• for all n > n0, abs(f(n)) <= abs(d*g(n))
• For algorithms,
• n is the "size" of the input (e.g., the number of elements currently in the vector)
• f(n) is the running time of the algorithm
• Some people say "is in big O of g of n", rather than "is big O of g(n)".
• One thing that we observe with asymptotic analysis are that "constants matter much less than exponents". For example, if algorithm A took 1000*n seconds and algorithm B took 1*n*n seconds, for small n, B is much better, but once n gets sufficiently large (i.e., greater than 1000) A is much better.
• This means that we normally write the f(n)'s without leading constants.
• An algorithm that takes f(1) steps is a "constant time" algorithm. It takes more-or-less the same time for every input, and is independent of the size of the input.

### Analysis in practice

• So, how do you estimate the running time of an algorithm? Usually, by looking at the steps in the algorithm, and considering how long each step takes (most steps take O(1) steps, but some take O(n) or more).
• Once we have a rough estimate, we attempt to verify that estimate through more formal analysis.
• Let's guess and then analyze a few simple problems (note that our estimates will be based, in part, on the algorithms we develop).
• Finding the smallest element in a Vector of comparable elements.
• Finding the largest element in a Vector of comparable elements.
• Finding the average of all the elements in a Vector of numbers.
• Finding the median of all the elements in a Vector of numbers.
• Removing an element from a vector?
• Putting the largest element in a Vector at the end of the Vector. What if we were only allowed to swap subseqeuent elements?
• Computing the nth fibonacci number.

### Composition and big-O notation

• We can also do some analysis by considering our algorithm as the composition of algorithms, and use prior analysis of those algorithms.
• If f(n) is O(g(n)) and g(n) is O(h(n)) then what can we say about f(n) and h(n)?
• If f(n) is O(kg(n)) and k is a "constant", then what else can we say about f(n) and g(n)?
• If f(n) is O(g(n) + h(n)) and g(n) is O(h(n)), then what simpler thing can we say about f(n)?
• If f(n) is O(g(n)) and h(n) is O(k(n)), then what can we say about f(n) + h(n)?
• If f(n) is O(g(n)) and h(n) is O(k(n)), then what can we say about f(n) * h(n)?

### Alternate notations

• There are a number of concepts like big-O notation.
• We say that a function f(n) is Omega(g(n)), which is read "Omega of g of n" if
• there exists a number n0
• there exists a number c > 0
• for all n > n0, abs(f(n)) >= abs(d*g(n))
• We say that a function f(n) is theta(g(n)), which is read "theta of g of n" if
• there exists a number n0
• there exists a number c > 0
• there exists a number d > 0
• for all n > n0, c*g(n) <= abs(f(n)) <= abs(d*g(n))
• theta notation gives a more precise running time, as it provides both a lower and upper bound on the running time.

Outlines: prev next

[News] [Basics] [Syllabus] [Outlines] [Assignments] [Examples] [Readings] [Bailey Docs]

Disclaimer Often, these pages were created "on the fly" with little, if any, proofreading. Any or all of the information on the pages may be incorrect. Please contact me if you notice errors.