Summer 2009 Research with Samuel A. Rebelsky: Media Scripting and Interactive Application Scripting

This document provides an overview of the summer projects I have available for Grinnell students and the expectations I have of my summer students.

This page may be found online at http://www.cs.grinnell.edu/~rebelsky/Department/samr-summer-2009.html The application guidelines can be found online at http://www.cs.grinnell.edu/~rebelsky/Department/glimmer-app-2009.pdf.

Project Background

In this project, we are exploring scripting. What do we mean by scripting? There are clearly many ways in which computer scientists and computer programmers use the term, such as (1) writing small programs that handle administrative tasks (the original goal of Perl); (2) writing programs in PHP that build Web pages or in ECMAScript to (a.k.a. JavaScript) to make pages interactive; and (3) Visual Basic scripts used in Microsoft applications. We are focusing on something similar to the last interpretation, but in a more interactive context. In particular, we are considering how to support the kinds of small programs that novices might write in the context of working with a computer application, such as a graphics program or a word processor.

This project has evolved from some investigations of the applicability of the Scheme programming language (and, more generally, the functional programming paradigm) to a variety of image-based computations that I began in summer 2006. In summer 2007, we began to look at ways to bridge this project with various open-source applications, particularly the GIMP and InkScape, and to support better scripting of these applications in Scheme.

This past year, to deal with deficiencies in the scripting console in the GIMP, Soren Berg and I have been building a new user interface for interactively scripting the GIMP. We've ended up generalizing most of what we were doing, making the interface support multiple applications and multiple scripting languages. Hence, the direction of the project has moved from describing images functionally to investigating how best to support and encourage interactive scripting of applications. By interactive, we mean that the environment should support and encourage non-programmers to intermingle scripting with their regular use of the application. (We call the interactive-scripting approach IAScript and the media-focused version MediaScript.)

There are three directions in which we might expand IAScript and MediaScript this summer: We can add support for more languages, we can add support for more applications, and we can develop a large suite of illustrative examples that demonstrate the power of interactive scripting. MAP students and MIP (Mentored Introductory Project; 299) students will work on projects of their choice. MIP students will work primarily on developing the suite of illustrative examples. MAP students work in teams of two or three on language and application extensions. It's not clear how long all of these extensions will take, so some students may work on a single subproject all summer while others may work on multiple related subprojects.

Illustrative Examples

The team of MIP students working on illustrative examples will focus on the original application domains we have been exploring for scripting (raster graphics and vector graphics) and on the Scheme programming language. I expect that the first examples these students will work on will be extensions of our current examples:

• creating color washes each of whose pixels are computed based on the position of the pixel;
• creating new image filters on the fly;
• updating the drawing context (e.g., brush, color) while drawing; and
• employing the power of randomness.

The MIP students will consider the core GIMP procedural operations and consider appropriate ways to wrap those operations to make them more usable by novice programmers.

The MIP students will work through a classic text on using computers in art, Visual Modeling with Logo, finding ways to reframe and extend the examples using interactive scripting in Scheme.

The MIP students will explore tutorial guides for the GIMP and for InkScape, finding natural ways to turn long sequences of manual actions into scripting commands.

The MIP students will also work on the design of an online repository of illustrative examples. That is, they will consider ways to present, to tag, and otherwise to provide convenient access to the examples.

Language Extensions

Support for a language in IAScript/MediaScript requires a number of steps:

• Wrapping the interpreter in such a way that it fits the IAScript model.
• Creating appropriate user-interface facilities, such as a formatter/indenter for the language.
• Extending the language to support the IAScript/MediaScript applications (currently InkScape and GIMP)
• Developing some appropriate examples that demonstrate the benefit of scripting in the language.

By the time this summer rolls around, we should have support for three language dialects:

• We currently support Basic PLT Scheme (and MzScheme)
• We will soon have support for Python
• We will soon have support for TinyScheme

There are a number of languages that we would like to support. I expect we'll focus on one or two of these this summer

• We would like to add support for
  • ECMAScript
  • Tcl (and maybe Tk)
  • ML
  • Others you might suggest

Application Extensions

Support for a language requires as many things, and is likely to be more complex.

• Convert the IAScript user interface to an application plugin. (This may involve designing a plugin architecture for the application.)
• Determine appropriate application-specific operations.
• Implement those operations for Scheme.
• Implement those operations for Python.
• Implement those operations for other IAScript languages.
• Developing some appropriate examples that demonstrate the benefit of scripting in the application.

By the time this summer rolls around, we should have support for two applications:

• We currently support the GIMP
• We have an earlier, experimental version, that supports InkScape.
• We anticipate a new version of the InkScape plugin.

I am open to suggestions for other applications to script. Two that come to mind are

• Audacity
• Open Office

It may also be appropriate to pick domains, rather than applications, and look at how to best support interactive scripting for the domain. Right now, one domain seems particularly appropriate:

• Bioinformatics

It may also be appropriate to look at variants of MediaScript that better support interactive image creation using particular libraries:

• The ImageMagick image processing library
• The OpenGL 3D graphics library

MAP students should expect to do a significant amount of C programming, as IAScript and many open source applications (particularly the GIMP) are implemented in C. In most cases, I will expect MAP applicants to be familiar with C and memory management in C.

IAScript relies on the GTK+2.0 user interface toolkit. It also uses the GObject objects in C library. I will not expect students to be familiar with these libraries, but they will need to learn them.

Scheme remains the core IAScript language. I expect all applicants (both MAP and MIP applicants) to be comfortable with Scheme and with basic higher-order concepts, such as map.

We use autoconf and automake to manage compilation. I do not expect that students will know these tools before they begin, but they will need to learn them quickly.

We use SVN to manage our code repository. Once again, I do not expect that students will know how to use SVN, but I do expect them to learn quickly.

There will, of course, be other technologies that we use. Particular technologies are likely to depend on the project.

Distributing Our Work

Note that any usable software written in this project will be distributed under an open source model. IAScript and MediaScript have a two-paragraph BSD license, and I expect to continue to use this form.

Approximate Schedule

The spring portion of this schedule mimics the schedule officially approved by the division. However, I will certainly understand if students accept a position with me and then later choose to take a position elsewhere. In that case, I will notify students on my waiting list.

The particular ten weeks we use for summer research are currently under discussion. I would like to start on May 19, but other issues may make that impossible. Hence, the schedule may move back a week.

• Friday, 20 February 2009: Application forms due. You must submit the division-wide form in hardcopy to the division office and your responses to the Glimmer form via email to me.
• Friday, 6 March 2009: Initial selections announced (provided that the institution has approved funding).
• Friday, 13 March 2009: Preliminary acceptances/rejections due.
• Week of 9 April 2007 First meeting.
• Unspecified other dates Additional meetings.
• Monday, 18 May 2009: Commencement.
• Tuesday, 19 May 2009: Summer research begins.
• Week of June 29: Sam unavailable.
• Some other week: Sam unavailable.
• Friday, 24 July 2009: Summer research concludes.

Expectations

I have very high expectations of my summer research students. Among other things, I expect my students to begin their summer research during spring semester and continue their summer research into fall semester (and sometimes beyond). By applying for summer research you are agreeing to meet these expectations if I take you on as a research student. You are unlikely to receive explicit credit or compensation for work in the spring and fall.

As some students from the past may have reported, I expect my students to be self-reliant. While I do my best to be around, I expect you to be able to do many things on your own or with a small group.

Spring

• Topic Preparation. You are expected to begin your background research during the spring. In particular, you must identify at least four papers on related projects. Sources to consider include the ACM Digital Library (particularly proceedings of the SIGGRAPH Conference and perhaps the conference on LISP and Functional Programming). You are also encouraged to use the web and, once you've identified potentially useful resources, to consult with the librarians about using Science Citations.
• Skill Preparation. If your project will require a programming language (e.g., C++), data language (e.g., SVG), or library (e.g., QuickTime For Java) that you do not yet know, you are expected to begin studying that language. You need not master the language or libraries, but should develop some basic familiarity.

Summer

During the summer, you are expected to work full-time on the project (40-50 hours per week for ten weeks). Most of the work will be done in my research laboratory, during normal hours (8/9 to 4/5). The work will include scheduled daily group meetings.

• Topic Preparation. For the first week of summer research, you will continue your preparation from the Spring, developing a survey of the state of the art in whatever project you've decided to undertake. You should prepare a short survey paper. On the first day of the second week, you will give a public presentation of your work.

• Core Research. For the next eight weeks of the summer, you will work on your project, using what you've learned during preparation for guidance. Some of this time may be spent developing skills. We will have a full-group meeting every day at 10:00 a.m. Each group will present at least once per week at that meeting.

• Writing. For the last week of the summer (and, preferably, as you do your work), you will work on a five-to-ten page paper describing your work and placing it in the context of related work. Your paper should meet the highest standards of writing at Grinnell. If you work as part of a small group, the group need only prepare one paper. In most cases, you will be required to submit a version of this paper to a conference or journal. (I may provide significant assistance in developing the submitted version, in which case I will probably ask to be listed as co-author.)

Fall and Beyond

I realize that some students may be unavailable in the fall, due to a study-abroad program or similar opportunity. Those students will do what they can of this work (e.g., they might present work in the spring, rather than in the fall).

• Poster Presentation. You will create a poster describing your work and present it at the Grinnell Science Poster Seminar (typically during parents' weekend).
• Internal Public Presentation. You will give a twenty-five or fifty minute presentation on your work as part of the Computer Science Thursday Extras series.
• External Conference Presentation. If your work is submitted and accepted to a conference, and there is funding available for you to attend the conference, you are expected to attend and present your work.
• External Pew Presentation. You must submit your work to the Pew Midstates Science and Mathematics Consortium Fall Symposium on Undergraduate Research in the Physical and Mathematical Sciences. You must attend the symposium (including non-cs talks and present your work (in poster or talk form) if your work is accepted. You must give at least one practice talk before going to the conference.

Priorities

There are, unfortunately, a number of conflicting issues I must consider as I choose who to accept for summer research. While one goal is certainly to enhance my research, summer research also helps students, the department, and the institution. My approximate MAP priorities are as follows, although I will certainly take some students from lower categories before taking every student from a higher category.

1. Students in the class of 2010 on track to receive honors. One of the requirements for honors is that students receive external evaluation of their work. Research in CS is one way to complete that goal. (High scores on the Putnam and Modeling competition are others.) Students also need a 3.4 overall GPA, a 3.5 in CS, an extra 300-level Math or CS course, 213 and (211 or Physics 220), and probably some other things I've forgotten. I'd like to make sure that students who meet all the other requirements also meet the research requirement. Because international students are not eligible for REUs, I am more likely to take international students who meet the other honors requirements.
2. Students whose background shows they will be particularly productive members of the project team.
3. Students in the class of 2011 likely to participate in the project for multiple summers. I'd like at least one student who will be in category 1 the following year.

Because women and domestic students of color are under-represented in computer science, I will strive for some equity in the mix of students I select. I hope this will encourage more members of these groups at Grinnell to participate in computer science and, as they go on to successful careers, that they will further encourage others.

I many years, I have imitated the early research experience program in Physics and will reserve a few slots for first-year women and students of color who have expressed an interest in and aptitude for computer science. It is likely that these will be the only first-year students I take for paid positions.

Evaluating Applicants

Applicants should fill out the divisional summer research application, which is available from the science division office and at http://www.cs.grinnell.edu/~rebelsky/Department/Sci_Div_Summer_2009_Res_App.pdf and http://www.cs.grinnell.edu/~rebelsky/Department/Sci_Div_Summer_2009_Res_App.doc.

In addition, applicants must prepare responses to the Glimmer application form, available at http://www.cs.grinnell.edu/~rebelsky/Department/glimmer-app-2009.pdf.

Finally, applicants must meet with me for a fifteen minute interview.

I am likely to distribute the essays (project description for upper-division students, essay questions for first-year students) from your applications anonymously to a group of trusted advisors. We will each evaluate each person's essay or essays on a 1-4 scale.

After evaluating the various parts of the application, I will score each applicant using something like the sum of the following:

• 3 x average essay rank
• 2 x GPA in CS (I will use your Math GPA if you have no CS GPA)
• 1 x count of certain positive characteristics (e.g., being a rising senior, eligibility for honors, volunteering for Glimmer, being abused by the college in a particularly horrific way)
• -1 x count of certain negative characteristics (e.g., having previously completed a MAP)

I may discuss applicants with colleagues before and after the interview. While the interview and those discussions are not directly incorporated in the score, they provide additional information. In particular, I will not necessarily use the rankings as an absolute order. For example, a very poor interview may disqualify you, no matter how good your overall score is, and I may bump up a person with an excellent interview or who past experience suggests will fit well in my research team. I may also prioritize students with certain skill sets.