Time and Space Complexity
Time and Space Complexity
Goals: This laboratory exercises introduces some principles of
algorithm effectiveness, including the amount of time and memory required
for the algorithm. Big-O notation is introduced to provide an informal
measure of the time or space required by an algorithm.
In considering the solution to a problem, it is natural to ask how
effective that solution might be. Also, when comparing solutions, one
might wonder if one solution were better than another. Altogether, one
might use many criteria to evaluate such solutions, including:
-
Accuracy: Of course, any program should produce correct answers.
(If we were satisfied with wrong results, it is trivial to produce many
such answers very quickly.) However, it may not be immediately clear just
how accurate results should be in a specific instance. For example, one
algorithm may be simple to program and may run quickly, but it only may be
accurate to 5 decimal places. A second algorithm may be more complex and
much slower, but may give 15 place accuracy. If 5-place accuracy is
adequate for a specific application, the first algorithm is the better
choice. However, if 10 or 12 place accuracy is required, the slower
algorithm must be used.
-
Efficiency: Efficiency can be measured in many ways: programmer time,
algorithm execution time, memory used, and so on. If a program is to be
used once, then programmer time may be a major consideration, and a simple
algorithm might be preferred. If a program is to be used many times,
however, then it may be worth spending more development time with a complex
algorithm, so the procedure will run very quickly.
-
Use of Memory: One algorithm may require more computer memory in
which to execute. If space is a scarce resource, then the amount of space
an algorithm requires should be taken into consideration when comparing
algorithms.
-
Ease of Modification: It is common practice to modify old programs
to solve new problems. A very obscure algorithm that is difficult to
understand, therefore. is usually less desirable than one which can be
easily read and modified.
For this laboratory exercise, we focus on algorithm execution time and the
use of memory.
Algorithm Execution Time
In determining algorithm execution time, we may proceed in several ways:
-
We may time how long an algorithm takes on a specific machine.
-
We may analyze the algorithm at a detailed level to determine how many
instructions a computer must execute to solve the problem.
-
We may analyze the algorithm at a high level to approximate time factors,
independent of a specific machine.
Each of these approaches has advantages, but each also has drawbacks.
Execution times on a specific machine normally depend upon details of the
machine and on the specific data used. Timings may vary from data set to
data set and from machine to machine, so experiments from one machine and
one data set may not be very helpful in general.
The analysis of instructions may take into account the nature of the data
-- for example, one might consider what happens in a worst case. Also,
such analysis commonly is based on the size of the data being processed --
the number of items or how large or small the data are. This is sometimes
called a microanalysis of program execution. Once again, however,
the specific instructions may vary from machine to machine, and detailed
conclusions from one machine may not apply to another.
A high-level analysis may identify types of activities performed, without
considering exact timings of instructions. This is sometimes called a
macroanalysis of program execution. This can give a helpful overall
assessment of an algorithm, based on the size of the data. However, such
an analysis cannot show fine variations among algorithms or machines.
For many purposes, it turns out than a high-level analysis provides
adequate information to compare algorithms. For the most part, we follow
that approach here.
-
Reread Section 2.3.1 of the textbook.
Since that section focuses upon the processing of one or two numbers, the
analysis is based on the size of those numbers. Further, for the examples
in that procedure, within each procedure, only a few steps (a comparison
and a few arithmetic operations) are performed before the procedure is
called recursively. Said another way, the number of machine instructions
performed within a procedure is limited (perhaps under 10) before the
procedure is called again. Thus, the total number of steps performed by an
algorithm is roughly proportional to the number of procedure calls.
Applying this idea in more detail, the text concludes that the number of
instructions required for procedure mult%1 is proportion to
b, while the number of operations for procedure mult%2 is
proportion to log b.
-
Explain in your own words why this conclusion seems justified. (Be sure to
ask the instructor if you have questions.)
A macroanalysis ignores proportionality constants from a microanalysis; as
differences from machine to machine tend to change the proportionality
constant, not the nature of the main terms. Informally, the book suggests
that the order of an algorithm is the amount of time required to execute an
algorithm, ignoring the proportionality constants. Thus, procedure
mult%1 has order b -- written mult%1 has O(b).
Similarly, mult%2 has O(log b).
-
Solve exercises 2-15 through 2-19 from the textbook.
Section 2.3.2 discusses the amount of space required by an algorithm.
Again, the analysis can focus on a specific machine, or one may proceed at
a higher level.
-
Reread Section 2.3.2 of the textbook.
-
Define in your own words what is meant by tail recursion.
-
Solve exercises 2-21 through 2-23 from the textbook.
Work to be turned in:
-
Explanation from parts 2 and 5 of this section of the lab.
-
Solutions of the exercises described in parts 4 and 6 of this lab.
Note: When one of the above exercises requires you to write a
program, be sure to run and test the program following the format
specified for the Supplemental Problems.
This document is available on the World Wide Web as
http://www.math.grin.edu/~walker/courses/153/lab-complexity.html
created January 14, 1998
last revised January 14, 1998