Sorting a collection of values -- arranging them in a fixed order, usually alphabetical or numerical -- is one of the commonest computing applications. When the number of values is even moderately large, sorting is such a tiresome, error-prone, and time-consuming process for human beings that the programmer should automate it whenever possible. For this reason, computer scientists have studied this application with extreme care and thoroughness.
One of the clear results of their investigations is that no one algorithm for sorting is best in all cases. Which approach is best depends on whether one is sorting a small collection or a large one, on whether the individual elements occupy a lot of storage (so that moving them around in memory is time-consuming), on how easy it is to compare two elements to figure out which one should precede the other, and so on. In this course we'll be looking at two of the most generally useful algorithms for sorting: the insertion sort, which is the subject of today's reading, and the merge sort, which we'll talk about in the next reading.
Imagine first that we're given a collection of values and a rule for
arranging them. The values might actually be stored either in a list or in
a vector; let's assume first that they are in a list. The rule typically
takes the form of a predicate of arity 2 that can be applied to any two
values in the set to determine whether the first of them could precede the
second when the values have been sorted. (For example, if one wants to
sort a set of real numbers into ascending numerical order, the rule should
be the predicate <=; if one wants to sort a set of strings into
alphabetical order, ignoring case, the rule should be string-ci<=?,
and so on.)
The insertion sort works by taking the values one by one and inserting each
one into a new list that it constructs, constantly maintaining the
condition that the elements of the new list are in the desired order with
respect to one another. Clearly, this condition will not be maintained if
each element is added to the new list at the beginning, using
cons; instead, the insertion sort adds each element at a
carefully selected position within the new list, placing the new element
after each previously placed element that precedes it according
to the given precedence rule, but before every such element that
it precedes. The following procedure, insert, adds a new
element to a list in exactly this way. For the moment, we'll assume that
the elements of the list are real numbers and than we want to sort them
into ascending order; <= is therefore used as the ordering
predicate.
;;; insert: add a given real number to a given list of real numbers in
;;; ascending order, returning a new list, also in ascending order
;;; Givens:
;;; NEW-ELEMENT, a real number
;;; LS, a list of real numbers
;;; Result:
;;; EXTENDED, a list of real numbers
;;; Precondition:
;;; LS is in ascending order (that is, each element other than the first
;;; is greater than or equal to the one before it).
;;; Postconditions:
;;; (1) The elements of EXTENDED are exactly the elements of LS together
;;; with NEW-ELEMENT.
;;; (2) EXTENDED is in ascending order.
(define insert
(lambda (new-element ls)
(cond ((null? ls) (list new-element))
((<= new-element (car ls)) (cons new-element ls))
(else (cons (car ls) (insert new-element (cdr ls)))))))
In English: If the list into which the new element is to be inserted is empty, return a list containing only the new element. If the new element can precede the first element of the existing list, then, since the existing list is assumed to be sorted already, it must also be able to precede every element of the existing list, so attach the new element onto the front of the existing list and return the result. Otherwise, we haven't yet found the place, so issue a recursive call to insert the new element into the cdr of the current list, then reattach its car at the beginning of the result.
Now let's return to the overall process of sorting an entire list. The insertion sort algorithm simply takes up the elements of the list to be sorted one by one and inserts each one into a new list, initially empty:
;;; insertion-sort: arrange a given list of real numbers in ascending order
;; Given:
;; UNSORTED, a list of real numbers.
;; Result:
;; SORTED, a list of real numbers.
;; Preconditions:
;; None.
;; Postconditions:
;; (1) The elements of SORTED are exactly the elements of UNSORTED.
;; (2) SORTED is in ascending order.
(define insertion-sort
(lambda (unsorted)
(if (null? unsorted)
null
(insert (car unsorted) (insertion-sort (cdr unsorted))))))
By writing the specific predicate <= into the definition of insert, we restricted the preceding version of insertion-sort so
that it applies only to lists of real numbers and always returns a list in
ascending numerical order. Let's go back now and lift that restriction.
According to the original specification, insertion-sort should take
two arguments, the list ls and a predicate may-precede? that
compares elements of that list. Since in many applications the nature of
the desired ordering is known before the particular list to be ordered and
is constant over many applications to different lists, it makes sense to
tweak the interface to the sorting algorithm so that it takes these
arguments separately. (The logician Haskell B. Curry was one of the first
to reduce multiple-argument functions to single-argument ones by using this
technique, and in his honor the interface-tweaking operation is often
called ``currying.'')
As a first draft, we might try this:
(define insertion-sort ;; Be careful: This version doesn't quite work.
(lambda (may-precede?)
(letrec ((sorter (lambda (unsorted)
(if (null? unsorted)
null
(insert (car unsorted)
(sorter (cdr unsorted)))))))
sorter)))
The problem is that the actual use of the ordering predicate may-precede? is not in the body of insertion-sort, but inside the
insert procedure, which is entirely separate. Even if we went back
to the definition of the insert procedure and changed <= to
may-precede?, the sort still wouldn't work, because inside the insert procedure the identifier may-precede? wouldn't be bound to
anything.
What we'd really like to do is pick up the entire definition of insert and put it inside the definition of insertion-sort, at a
point where the identifier may-precede? has been bound:
;;; insertion-sort: given a binary ordering predicate, construct a
;;; procedure that takes a list and arranges its elements to be consistent
;;; with the ordering
;; Given:
;; MAY-PRECEDE?, a binary predicate
;; Result:
;; SORTER, a procedure.
;; Precondition:
;; MAY-PRECEDE? expresses an ordering relation (that is, it is connected
;; and transitive).
;; Postconditions:
;; Given any list UNSORTED of elements that meet any preconditions that
;; MAY-PRECEDE? imposes on either of its arguments, SORTER returns a
;; list SORTED such that
;;
;; (1) The elements of SORTED are exactly the elements of UNSORTED.
;; (2) SORTED is in ascending order.
(define insertion-sort
(lambda (may-precede?)
;; insert: add a given value to a given ordered list, returning a new
;; list, also ordered
;; Givens:
;; NEW-ELEMENT, a value
;; LS, a list
;; Result:
;; EXTENDED, a list
;; Precondition:
;; LS is in order by MAY-PRECEDE? (that is, each element except the
;; last bears MAY-PRECEDE? to the one after it).
;; Postconditions:
;; (1) The elements of EXTENDED are exactly the elements of LS
;; together with NEW-ELEMENT.
;; (2) EXTENDED is in order by MAY-PRECEDE?.
(define insert
(lambda (new-element ls)
(cond ((null? ls) (list new-element))
((may-precede? new-element (car ls)) (cons new-element ls))
(else (cons (car ls) (insert new-element (cdr ls)))))))
(letrec ((sorter (lambda (unsorted)
(if (null? unsorted)
null
(insert (car unsorted)
(sorter (cdr unsorted)))))))
sorter)))
It is a pleasant surprise to discover that it is legal to do exactly this
in Scheme. The identifiers that are introduced through such embedded
definitions are local, and behave as though they were bound by means of a
letrec-expression; indeed, the Scheme standard specifies that the
preceding code must be semantically identical to the following version
using letrec:
(define insertion-sort
(lambda (may-precede?)
(letrec ((insert
(lambda (new-element ls)
(cond ((null? ls) (list new-element))
((may-precede? new-element (car ls))
(cons new-element ls))
(else
(cons (car ls) (insert new-element (cdr ls))))))))
(letrec ((sorter (lambda (unsorted)
(if (null? unsorted)
null
(insert (car unsorted)
(sorter (cdr unsorted)))))))
sorter))))
However, a lot of people find the version that uses an embedded definition
more readable. One can even convert both letrec-expressions
into internal definitions:
(define insertion-sort
(lambda (may-precede?)
(define insert
(lambda (new-element ls)
(cond ((null? ls) (list new-element))
((may-precede? new-element (car ls)) (cons new-element ls))
(else (cons (car ls) (insert new-element (cdr ls)))))))
(define sorter
(lambda (unsorted)
(if (null? unsorted)
null
(insert (car unsorted)
(sorter (cdr unsorted))))))
sorter))
Disciplined programmers use embedded definitions only for procedures.
Beginners are sometimes tempted to use them instead of let or
let* to create local names for intermediate results in a
computation, but this is usually a mistake. Unlike top-level definitions,
embedded definitions are not sequential -- the bindings are
mutually recursive and simultaneous, as in letrec-expressions,
not successive, as in let*-expressions.
Finally, let's consider the rather different case in which the values that we want to arrange are presented as a vector and the goal of the sorting algorithm is to overwrite the old arrangement of those values with a new, sorted arrangement of the same values.
Instead of constructing a new vector, we partition the original vector into two subvectors: a sorted subvector, in which all of the elements are in the correct order relative to one another, and an unsorted subvector in which the elements are still in their original positions. The two subvectors are not actually separated; instead, we just keep track of a boundary between them inside the original vector. Items to the left of the boundary are in the sorted subvector; items to its right, in the unsorted one. Initially the boundary is at the left end of the vector. The plan is to shift it, one position at a time, to the right end. When it arrives, the entire vector has been sorted.
Here's the plan for the main algorithm, then. Once again, we use currying so that the ordering rule can be provided before the vector.
;;; insertion-sort!: given a binary ordering predicate, construct a
;;; procedure that takes a vector and destructively rearranges its elements
;;; to be consistent with the ordering
;; Given:
;; MAY-PRECEDE?, a binary predicate
;; Result:
;; SORTER!, a procedure.
;; Precondition:
;; MAY-PRECEDE? expresses an ordering relation (that is, it is connected
;; and transitive).
;; Postconditions:
;; Given any vector VEC of elements that meet any preconditions that
;; MAY-PRECEDE? imposes on either of its arguments, SORTER! destructively
;; modifies VEC so that the following conditions are met:
;;
;; (1) The elements of VEC are the same as in its initial state.
;; (2) VEC is in order by MAY-PRECEDE?
;;
;; SORTER! does not return any particular value; it is invoked only for
;; its side effect.
(define insertion-sort!
(lambda (may-precede?)
;; The definition of the INSERT! procedure goes here.
(lambda (vec)
(let ((len (vector-length vec)))
(do ((boundary 0 (+ boundary 1)))
((= boundary len))
(insert! (vector-ref vec boundary) vec boundary))))))
The insert! procedure takes three arguments: an element to be
inserted into the sorted part of the vector, the vector itself, and the
current boundary position. The new element can be inserted at any position
up to and including the current boundary position, but it must be placed in
the correct order relative to elements to the left of that boundary. This
means that any elements that should follow the new one should be shifted
one position to the right in order to make room for the new one. (Elements
that precede the new one can keep their current positions.)
;;; insert!: place a given value into a given vector, preserving ordering
;;; within an initial segment of the vector
;; Givens:
;; NEW-ELEMENT, a value
;; VEC, a vector
;; BOUNDARY, an exact integer
;; Results:
;; None.
;; Precondition:
;; BOUNDARY is not negative and is less than the length of VEC.
;; Postconditions:
;; (1) The elements of VEC in positions less than or equal to BOUNDARY
;; are NEW-ELEMENT and the values that were initially in positions
;; less than BOUNDARY in VEC.
;; (2) The elements in positions 0 through BOUNDARY, inclusive, in VEC
;; are ordered by MAY-PRECEDE?.
;; (3) The elements of VEC in positions greater than BOUNDARY are the
;; same values that initially occupied those positions.
(define insert!
(lambda (new-element vec boundary)
(do ((candidate boundary (- candidate 1)))
((or (zero? candidate)
(may-precede? (vector-ref vec (- candidate 1))
new-element))
(vector-set! vec candidate new-element))
(vector-set! vec candidate (vector-ref vec (- candidate 1))))))
In English: Starting at the boundary and working from right to left,
examine each position in turn as a candidate for the position at which
new-element should be inserted. If the position number is 0
(so that we've reached the left end of the vector), or if the element just
to the left of the current position can precede new-element,
stop and put new-element in the current candidate position.
Otherwise, fill in the current candidate position by copying the element
just to its left into it and proceed to the next iteration, in which the
position of the element just copied will be overwritten one way or the
other.
This document is available on the World Wide Web as
http://www.cs.grinnell.edu/~stone/courses/scheme/readings/sorting-by-insertion.xhtml
created November 21, 1997
last revised November 4, 2003