Sarah M. Luebke and Hilary A. Mason
Department of Mathematics and Computer Science
Grinnell College
Grinnell, Iowa 50112
{luebke,masonh}@ac.grin.edu
The World-Wide Web [Berners-Lee et al. 1994] is revolutionizing the ways in which we share information. In particular, it is affecting the ways in which we teach and learn. Unfortunately, while the Web is often seen as making students more active learners, most students can only interact with pages passively, by reading and clicking links. In a more active settings, students would be able to annotate pages with notes (for individual or public use), glosses, questions, answers, and discussions directly to web pages. In this paper, we describe a system we have developed to support such annotations.
There are other annotation tools (or web-based systems that include annotations) available, such as ComMentor [Röscheisen 1995], CoNote [Davis 1996], HyperWave [Maurer 1996], and WebCT [Goldberg 1998]. We distinguish our system from by supporting annotations of both local and remote pages; permitting annotations at both author-defined and reader-defined positions on the page; supporting annotations on changing pages; and providing multiple protection levels for annotations.
We developed this annotation system to serve multiple purposes. First, an annotation system would provide a way for a student to take personal notes. Given the increasing number of course pages available, and the increased use of computers in the classroom, it may be possible for students to take online notes on course pages during class. For example, [Rebelsky 1998] reports that in a computer-equipped classroom in which daily course outlines were available online during class, some students opened the daily outline in an HTML editor to add their own notes. Second, an annotation system is also an appropriate way to share notes. For example, one might post questions about an assignment or suggest followup readings. Third, students can use sequences of notes to provide a discussion (similar to a typical threaded bulletin-board, but with focus on a particular point on a web page).
For an annotation system to be successful, it must not only accommodate the needs, preferences, and habits of its users, but also predict and shape them. A student or instructor new to hypermedia authoring and usage may understand the ability to add notes to a page, but may not think about other issues, such as protection (who should be able to see an annotation), selection of annotation points (where annotations should go in a document), or appearance and positioning of annotations. We report on these design issues and the user testing that guided our design choices in the subsequent sections.
There are three trends for protection of annotations: annotations are either private (accessible only to the user), group/semi-private (restricted to a group of users), or public (accessible to anyone reading the page). Many systems permit only one type of annotation. For example, CoNote provides only group annotations. Similarly, systems treat the different types of annotations quite differently. For example, WebCT [Goldberg 1997] separates annotations, which are private, from discussions, which are semi-private, using a different interface for the two.
We chose to provide all three types of annotations with a consistent interface. When a user creates a new annotation, (s)he may choose to make it private, group, or public. Because we could foresee different groups of people using the same page (e.g., faculty, teaching assistants, and students might all have discussions on a course page), group protection also includes a note as to which groups may view the annotation. When browsing annotations, readers can see the headers of all the annotations, but may only view the bodies of annotations for which they have appropriate permissions.
We faced a more difficult problem faced when discussing at which points in the document annotations should or could appear. Often, annotations are treated as page-level notes not associated with any particular part of the page. However, discussions with faculty and students suggested that there is need for more specific placement of annotations. For example, a question about a particular problem in a homework assignment best belongs near that problem. More importantly, an instructor interested in having students gloss a work (i.e., add commentary) will not be satisfied with annotations that are so coarse-grained.
A finer-grained ideal would be to permit users to select the text to be annotated. Such a solution is used in the Multivalent Document system's markup system [Phelps and Wilensky 1997]. Unfortunately, standard HTML browsers will not easily support this choice. One alternative, used in the Group Annotation Transducer [Schickler et al. 1997], is to allow users to specify the text that will be tagged. Unfortunately, entering the appropriate text requires more effort on the part of the user and also suggests difficulties if the page changes or if the same phrase appears multiple times. Both of these fine-grained solutions may lead to annotations at awkward, confusing, or otherwise inappropriate points on the page.
Another alternative is to allow authors to specify the annotation points in a document, as in CoNote [Davis 1996]. The disadvantages of such an alternative is that it places all the burden on the author, and provides little freedom for the annotator. An advantage is that it permits annotations to remain through changes of the document (something particularly difficult in phrase-based annotations).
We chose a hybrid solution, in which annotation points can be added by the author, but are also generated by the system for "natural" points in the document. These points are currently set to be section headings, which often designate a logical break from one thought to another. For typical documents, section headings are fine-grained enough to permit annotation at a reasonable level but coarse-grained enough that the document is not over-croweded with annotation points. We use heuristics based on approximate text matching to determine the (re-)placement of annotation points in changed documents.
A similarly interesting problem was where to put the annotations. A number of alternatives suggested themselves (many of which have been used in other systems). One might make links to the annotations and bring them up in the same window when clicked. Such a solution is easy to develop, but makes it more difficult for the novice web user to view both text and annotations. One might make links to the annotations and bring them up in the same window. One might insert the annotation text into the same page. However, this can make the original text more difficult to read, as the annotation interrupts the flow of the text. One might use popup windows, in which the annotation appears over the annotated text when one moves the mouse over that text or clicks on it.
We decided that this was not an issue we could approach in the abstract, after the system was constructed. Hence, we surveyed potential users about the strengths and weaknesses of a variety of designs. We developed eight different prototype interfaces for an annotation system and surveyed students with a variety of computer background. The examples varied in appearance and technical design issues. One example displayed annotations in a separate pop-up window. Another brought the user to a new page which displayed the annotation and the corresponding paragraph (the annotated text). In a third design, the annotations were visible on the page, within the document. For another the annotations were in the margin. The colors and icons changed from example to example, while the text and annotations were constant. Since we worried about interrupting the flow of text, a few examples permitted the users to turn the annotation feature on or off. We did not test entry into the system (for entering account and password) which is taken from on Project Clio [Becker and McLaughlin, 1998].
Most participants preferred a pop-up window to view and add annotations, since they did not want to disrupt the page they were on. Annotations directly on the web page seemed to add clutter and make the page less readable. Also, the less obtrusive the annotation buttons were, the better. At the same time, many found the ability to hide and show the annotation buttons (or the annotations themselves) confusing.
Based on this feedback, we developed a hybrid of the eight used in the survey. We chose to use two small buttons to indicate the annotation points. One button is used to add annotations and appears at every annotation point in the document. The second button is used to indicate the presence of annotations at that particular point in the document, and is only used if there are annotations for that annotation point.
If one clicks on the "view annotations" button, a separate window appears showing a collapsible tree of annotations corresponding to that annotation point. There is a tree of annotations because we permit readers to write followups to other annotations, and found a tree the most natural way to represent the many discussions that may come from one annotation point.
If there is only one annotation available for a particular annotation point, that annotation is used as the "alt text" for the annotation button. In modern browsers, such as Netscape Navigator 4.0, this provides a quick pop-up view of the annotation. If multiple annotations are available for a particular annotation point, the number of annotations is used as the "alt text".
When the user selects the "add annotation" button or the "followup" button, a new window appears. The annotator can enter a name, title, and the annotation. Annotators can also select permissions: self, selected groups, everyone on system, world. Finally, an annotator can choose to make an annotation anonymous (so that, for example, students can post comments without fear of looking foolish).
After the new interface was complete and the underlying system implemented (so that subjects could add their own annotations and not just view existing annotations), a second group of subjects was brought in to test the new interface. This group of subjects included some of the first group, but also some new subjects.
While participants were generally satisfied with the structure of the implemented annotation system (e.g., the pop-up windows), many identified some technical problems that would not be obvious in a prototype. For example, many pointed out that the windows should auto-reload, so they can see the changes they made immediately. It was also noted that a close button was needed in the annotation window. Finally, many subjects needed more ready access to a help system.
We have developed and begun testing of a system that we expect will enhance the effectiveness of course pages by permitting students to add notes and questions to web pages in and out of the classroom. Our initial user-testing suggests that this will be a usable system, but we do not yet know how much or how it will be used. During Fall 1998, the system will be used in a few select courses. Based on feedback from that initial course trial, we expect to make extensions and modifications to the system.
We graciously acknowledge Professor Sam Rebelsky for his guidance throughout this project, Raphen Becker and Kevin McLaughlin, whose work made made our project possible, and our wonderful survey participants for giving us very helpful comments. This work was supported by Grinnell College, the Grinnell College Noyce Science Summer Research Fund, and the Robert N. Noyce Faculty Study Grant.