The Flashlight Project and an Introductory Writing Course Sequence:

Susanmarie Harrington

Indiana University-Purdue University, Indianapolis

In the fall of 1995, the Indiana University-Purdue University English Department entered its tenth year of computer-assisted composition instruction. The department has had a long history of involvement with computers and composition r esearch and practice; it served as one of the beta testing sites for earlier versions of the Daedalus Integrated Writing Environment, one of the earliest software programs developed specifically for writing classes, and its faculty includes some pioneerin g computers and composition researchers. The school's computer facilities have been consistently expanded and upgraded, and increasing numbers of introductory writing courses integrate some level of computer time into their regular offerings.

After ten years, the department's writing program administrators (a collaborative committee which I chair) began to wonder whether the computer-assisted writing courses were victims of their own success. That is, after many years of ex perience, and a series of technological upgrades, computer-assisted instruction had ceased to seem extraordinary, and had lost its position within the writing program as a new practice needing consistent evaluation and orientation. Other—quite legitimate —needs within the writing program were receiving more attention from faculty and administrators. Of course, it was possible that the seeming lack of attention to computer-assisted instruction merely reflected success; perhaps teachers and students in tho se rooms were engaged in exciting teaching and learning practices. As administrators we decided to investigate the activities in the computer-assisted classrooms, to evaluate the ways in which faculty were trained and supported in their uses of technolog y, and to evaluate the ways in which students were learning with technology. Since roughly half the sections in our introductory writing sequence meet in the computer classrooms for half their class meetings, we began a study that would allow us to comp are computer-assisted classes to traditional classes and also allow us to describe the activities in computer-assisted classrooms.

The Flashlight Project offered us a way to carry out this investigation, through two different instruments. The Flashlight Current Student Inventory (beta test version) was used to gather information from writing students in three different courses (basic writing, first year composition, and a second-semester course in professional and business writing, and the Flashlight Cost Analysis Manual (which we used with the assistance of IUPUI's Economic Model Office)was used to create an expansive model of costs associated with writing instruction. This multi-pronged research effort probed the relationship between students’ technology use, teaching and learning, and offered the department’s writing program administrators a valuable context for future curriculum and policy planning. Our Flashlight participation is notable not only for the research results these efforts produced—he did learn many interesting things about the differences, and lack of differences, between students in computer classrooms and students in traditional class rooms—but for the process of research, reflection, and action that the Flashlight project stimulated.

Indiana University-Purdue University, Indianapolis (IUPUI) is a large, urban campus in the heart of Indianapolis. Home to the Indiana University professional schools (in law, medicine, dentistry, and social work), it serves over 28,000 students in central Indiana, earning degrees from a range of Purdue and Indiana University schools. Historically, its student body has been older (the average age of undergraduates is 28), part-time, and commuting. In the past few years, more and more students in traditional college age groups have been choosing to attend IUPUI, and in 1996-97 the proportion of full-time students pulled equal to that of part-time students for the first time in the campus’ 26 year history. Despite these changes, nontra ditional students remain the prominent feature of the undergraduate student body, and the vast majority of students work at least 20 hours per week in addition to their class load. Furthermore, many students transfer into and out of IUPUI, so the student body itself is not stable.

The Department of English offers a variety of writing classes to meet the needs of this diverse student body. All students must complete a three credit first year composition course. Each year, between 35-50% of entering students are j udged not yet ready for that course, and they are required to take a basic writing course. Most schools at IUPUI require students to have an additional three credits of writing instruction beyond the first year composition course, and the department offers two courses as a second semester option. These courses stress research in either academic or professional contexts (see figure 1).

About half the sections of these writing courses meet in a computer classroom for half their class sessions. In order to give as many students as possible access to the computer classrooms, we schedule classes in pairs, allowing each class in the pair access to the room for one of the two weekly meetings. The sections that meet in the computer classrooms differ from the sections that meet in traditional classrooms in some significant ways. Computer-as sisted sections have 23 students each, while traditional sections have 29. This difference arises purely from practical matters and not from pedagogical theory; only 23 computers fit in the computer classrooms, and class size in traditional classes has been raised by the school in the face of budgetary constraints. Computer-assisted sections meeting in the computer rooms have access to a wide array of software. The classrooms are connected to the campus backbone, so that students can access the library ca talog and selected databases from class; they also have access to the World Wide Web and e-mail connections. Several commercial word processors are available; most classes use WordPerfect 6.0 for Windows. The one piece of software bought specifically for writing classrooms is the Daedalus Integrated Writing Environment (DIWE), which is actually a suite of programs that run under a single interface. Various components of DIWE make it possible for students to compose text in a rudimentary word processor (Daedalus Write), to hold synchronous conferences (Interchange), to respond to work in progress (Respond), to begin the writing process (Invent), or send mail within the class (Daedalus Mail). Students’ computers, arranged in rows, face the teacher’s computer, which is connected to a projector; students can share their files via network connections, and the teacher’s station can project a file for the whole class to view.

Space limitations affect the courses which are scheduled into the two computer classrooms we control. We offer more than 60 sections of first year composition each semester, and about half of them are able to meet in computer classroom s. At the time of this study, all on-campus sections (approximately 10 per semester) of business and professional writing met in the computer classrooms. This meant that only two or three (of about 20) sections of basic writing classes and none of the academic research writing sections were able to use the computer classroom (see figure 2). Classes that met twice a week had access to the computer room once a week; classes that met only once a week had two classrooms assigned to them, so that they coul d meet in either room as the instructor and students desired. That the computer-assisted classes are smaller than traditional classes, and that different courses have access to computer rooms at different rates, raises questions about the data generated by this research. It is possible, even probable, that some of the factors that appear to be related to technology use here are inter-related with factors stemming from the course curriculum or class size. In acting upon these findings, then, we have tre ated them with caution, realizing that the information gathered is simply one piece of a large and complex puzzle.

Despite the environmental differences between computer-assisted and traditional sections, many similarities across sections exist. All classes, whether meeting in a traditional classroom or computer classroom, shared the same course goals and syllabi. Decisions about what to do in class each day were up to individual instructors, but a standardized portfolio for each class was used for end-of-semester evaluation. The standardized portfolios lead to a great deal of standardization a mong classroom activities.

As we began to reflect on the standardization of our courses, and the literature on computers and composition which offers a tantalizing array of arguments about why emerging electronic literacies are different from traditional print li teracies, we began to question that standardization, both in theory and in practice. We began to wonder whether this standardized approach really was standardized: were students in the computer classrooms learning, and being taught, in the same ways? An d did students in the computer classes feel that students were helping them learn? Did they share our teacherly assumptions about the value of time spent in those computer classrooms?

In order to explore these questions, we used the beta test version of the Current Student Inventory item bank to draft a questionnaire that would be distributed to students in the three different writing courses. The questionnaire asked students about their teaching and learning practices, experiences with instructional technology, satisfaction with course, satisfaction with technology training, course outcomes, and demographic information. We focused our questions on teaching and learning practices relating to student effort and engagement, and connections with other people. Since a large part of our curriculum encourages students to see writing as a social practice requiring commitment and ethical engagement, we selected questi ons that would allow us to gauge the impact of technology and curriculum on this dimension of writing. In retrospect, the survey (with more than 150 questions) given to our students was too long and cumbersome; we were working with researchers from sever al different units on campus, all of whom considered this project a way to answer important questions. The survey constructed tried to cover too much ground than we would advise others to cover in a single survey.

We distributed surveys to a random sampling of basic writing, first year composition, and business and professional writing sections, taking account of the variety of times and places courses are scheduled: on- and off-campus, daytime, evening, and weekends. A total of 421 surveys were returned, divided more-or-less equally between computer-assisted and traditional sections. Because the surveys were completed during a class meeting for all of the sample sections, all students present at those classes returned surveys. We have no way of knowing how many students were absent the day that surveys were distributed, but we have no reason to suspect that rates of absence differ between sections. We should note that business and professional writing students are under-represented in the traditional section sample because so few sections of their course meet in traditional classrooms, and basic writing courses are over-represented in the traditional section sample because so few of their sect ions meet in the computer classroom.

In addition to surveying students with the Current Student Inventory, we used the Cost Analysis Manual to investigate the faculty effort devoted to computer-assisted classes and traditional classes. The Cost Analysis Manual provides an expansive model for determining the costs (or savings) of different instructional techniques. The model analyzes obvious costs (such as the price of computers for the computer classrooms) as well as less obvious ones (such as the cost of the classroom s pace, and the use of faculty time). We used the Cost Analysis Manual to evaluate the total costs of offering writing instruction in computer-assisted and traditional sections. While it might seem counterintuitive to explore the question of whether a com puter classroom could be more cost-effective than a traditional classroom, the Cost Analysis Manual addresses this question in a sophisticated way. Although the computers themselves will always cost more than traditional desks, if students in the compute r classes can learn more material more quickly, or are retained at higher rates, the classes could be said to be more cost-effective. Furthermore, if faculty are spending more or less effort teaching in different formats, the cost of faculty time may var y.

Perhaps the most striking finding of this research effort is that there are very few significant differences between computer-assisted sections and traditional sections. There are few differences in the students in each type of section , there are few differences in overall faculty effort, there are few differences in the teaching and learning practices in each type of section. On the whole, students in both sections reported great satisfaction with their courses and their teachers, an d on the whole, students in both sections reported that they were engaging in a wide variety of collaborative learning practices.

Students in both sections reported remarkably similar relationships to technology. Both groups of students reported equivalent measures of technological expertise (rating themselves, overall, middling competent with a variety of techno logies ranging from answering machines to computer programming). Students in the computer-assisted sections reported slightly less proficiency with word processing, so it is possible that they registered for computer-assisted sections in order to improve that skill, although student interviews suggest that students are not aware of the differences in instruction between sections at registration. Both types of students reported having used a computer for close to two years before starting the semester; b oth groups reported similar overall GPAs before the start of the semester, and both groups were disinclined toward technology anxiety (when asked to agree or disagree with the statement that technology anxiety is hurting their course performance, both gro ups reported that they disagreed.). These groups of students are similar in age, and in prior e-mail experience.

These groups of students also reported remarkably similar learning practices. They are equally likely to review additional readings outside of class, equally likely to revise assignments, equally likely to study with other students, to help other students with class assignments, and to communicate with their instructors about their coursework. They spend about the same amount of time revising their work.

This similarity in survey results mirrors the perceptions of instructors, who have never reported noticeable differences between students in computer-assisted classes and traditional ones. These similarities are likely produced by a nu mber of factors. All introductory writing courses are required, and students take them without knowing which instructor they will have and apparently without regard for mode of instruction (the computer sections are listed in the time schedule ahead of th e traditional sections, which may influence enrollment). Students appear to sign up for sections of these required courses randomly (most likely based on time preferences). In class, students pursue the same curriculum; we have not made any efforts to d istinguish the curriculum in the computer sections from the curriculum in the traditional sections.

All students reported spending a significant portion of time engaged in collaborative activities, no matter what type of writing course they were in. As illustrated in Table 1, significant numbers of students spend between 1 and 5 hour s each week engaged in the work of writing and the work of talking to others about that writing. Given that students reported spending an average of nearly 6 hours per week studying for their writing course (with median response of 4 hours and a mode of 3 hours), reports of 1-5 hours spent on any given activity represent a large portion of the students’ time given to study. Our writing curriculum emphasizes conversation and collaboration, and we were not surprised to see these activities reflected in st udents’ reports of how they spent their time out of class each week.

The decision to put as many classes as possible into the computer classrooms means that computer-assisted classes spend about half their time in a traditional classroom, so the differences in instructional format are not as great as the y would be if the computer-assisted classes met only in computer classrooms. Because the courses are offered in so many sections, taught often by part-time faculty, they use a common syllabus developed by the writing program administrators (with the part icipation of part-time faculty). This common syllabus is used in all sections, computer-assisted and traditional, and the program has always made an effort to minimize differences between the computer-assisted sections and the traditional sections, seeki ng to offer all students the same high-quality experience. As a result of the Flashlight inquiry, we are reconsidering whether the implies to standardize across sections affects the possibilities for the computer-assisted sections, and whether we should explore ways in which the computer-assisted sections could achieve program goals by different means. At the time the data were collected, however, the common syllabi were a powerful influence on student experiences; given that there were no significant differences between curricula, it is not surprising that student learning practices were so similar across sections.

Despite these general similarities among students across courses, the Current Student Inventory questionnaire did reveal three ways in which students in computer-assisted classrooms differed from their counterparts in traditional classr ooms. These differences enable us to make some inferences about the ways in which students value their computer use. First, across all three courses, students in the computer sections reported that, because their course requires the use of electronic c ommunication, they are more likely to put thought into their comments (this difference is statistically significant at the .05 level). While the students did not report spending any more time on their assignments, and they did not report engaging in very different patterns of consulting with their teachers or peers, they did feel that the thinking efforts expended were greater than they would be in a course that did not require instructional technology. If instructional technology increases students’ pe rceptions that they are engaging with course material, this is an encouraging finding. Engaging students is the first step in teaching them.

The first difference, then, involves a difference in which computer-assisted instruction appeared to enhance student learning. The two other differences identified involve ways in which such instruction appears to detract from student learning.

Even while students in the computer-assisted sections reported putting more thought into their comments, they were less likely to think that writing assignments encouraged their creativity. This finding was not statistically significan t (.08), but is of concern because of the importance of creativity in a writing class. The relationship between students’ perceptions of creativity and students’ perceptions of effort is one that bears further study; a writing curriculum seeks to help st udents understand the relationships among their own creativity and imagination and demands placed on them by the audience and purpose for their writing assignment. In interpreting the findings about differences between students with respect to effort exp ended, creativity, and time spent learning technology, we must remember that students in the computer-assisted sections number among them a much higher proportion of students in the second-semester course in business and professional writing. On the whol e, students in that course reported that the writing assignments encouraged creativity at a much lower rate than students in the other two courses; in fact, as students move through the writing curriculum, they report less and less creativity is encourage d. Since the curriculum offers increasingly formal constraints as it develops, it is not surprising that students see this as a disincentive to be creative. The students in the computer-assisted sections thus include a higher proportion of more advanced students (i.e. sophomores or juniors rather than first year students), and this could be related to their finding that they put more thought into their writing assignments. However, given the overwhelming similarities between the two groups of students, it is likely that some of these responses are due to instructional methods.

The final set of differences between the two groups of students relates to the time spent using technology. Students in the computer-assisted sections are more likely to feel that, compared to courses which did not require technology, they waste time sorting through their e-mail to find important messages (significant at .03) , and more likely to feel that they are spending too much time learning technology (significant at .05). Of course, it is not entirely surprising that increased technology use brings with it costs in terms of time and effort; students who are not required to use technology in particular ways are not spending time learning technology in class. But to the extent that time spent learning technology detracts from ti me spent learning course material, or frustrates students as they become accustomed to classroom routines, it becomes an issue that teachers must learn to manage better.

Time spent learning technology proved to be a factor for teachers in the computer-assisted classrooms (see below) as well the students there. Both groups reported spending more time—from the students’ point of view, perhaps too much ti me—learning to use technology. Such efforts are unavoidable, although it is possible to change attitudes towards that time spent. But the use of the technology requires time invested in training and orientation; the challenge for teachers is to make tha t training time also learning time related to course goals, and the challenge for administrators to make the faculty development continually generative.

As we move into a world which increasingly demands computer skills from college graduates, students who are unable to acquire technical skills are increasingly at risk. The Current Student Inventory data allowed us to look at the relat ionship between students with low levels of technical skills and their more techno-comfortable peers. Students used a 10 point scale to rate their levels of technical skill (10 representing "expert user") with a range of technologies (from answ ering machines to the World Wide Web). Students who rated themselves an average of 3 or lower on most technologies were designated as having low technical competence for purposes of our analysis; the mean for most technologies was a self-reported rating between 5.2 and 7.8, although some technical skills (such as preparing multimedia papers or authoring web pages) received low ratings from most students (means of 2.2 and 2.1, respectively).

Only 62 students reported their technical competence at very low levels (3 or lower), overall, which was encouraging news about our students’ technical preparation. We might expect that these students would be more likely to enroll in traditional classrooms than computer-assisted classrooms. In fact, this was not the case (perhaps because students do not consciously register for one format or the other; 27 students registered for computer-assisted sections while 35 for traditional). Nor were these students different in any demographic categories; the students with low technical skills included women and racial/ethnic minorities in the same proportion as the total sample. (Interestingly, students who rated themselves highest in techn ical skills, especially for more "hard-core" technological skills like programming or using Java, were overwhelmingly male, but the low-skills students were equally divided between men and women.)

These students with low skills differed from their peers in terms of computer access. They ere much less likely to own or use a computer. More distressingly, their GPAs tend to be lower, another sign that they may be at relative acade mic risk.

The relationship of these students to the classroom environment also differs in some very real ways. In a series of questions that asked students whether they spent too much time learning various forms of technology (e-mail, word proces sing), these students consistently reported that they were disadvantaged because of their low skills, the time spent learning the technology, and their poor levels of access to technology (all of these findings are significant at the .00 level). Perhaps because of their poor access to technology, they spend fewer hours per week using e-mail and word processing for their writing courses, even though they work, on average, 4 fewer hours per week. Such a finding seems common-sensical, of course; there is l ittle reason to expect a student with very poor typing skills to find class activities involving rapid typing very satisfying. But given that students with low technical skills are so few in number, they can easily become an invisible group, scattered amo ng sections. Teachers must be more vigilant about identifying these students early, and providing assistance early and often.

In class, these students were more withdrawn then their peers in a variety of ways. When these students enroll in computer sections, they are less likely to communicate with friends who attend other institutions to ask for help (signif icant to .09), and they are less likely to communicate with experts outside the university (significant to .09). They tend to be less likely to discuss course material with their instructors; although this finding was significant only to .13, it struck us as noteworthy given that only 27 students in the study fell into this category. With such small numbers, we interpreted the results generously.

The other prong of our research effort involved efforts to estimate the costs of faculty time spent on teaching in computer and traditional classrooms. The Cost Analysis Manual investigation allowed us to find out how faculty reported spending their time, which offered another perspective on student-faculty interaction. Faculty teaching first-year composition in both traditional and computer-assisted classrooms reported spending time on the same types of activities—responding to stude nts’ work, preparing for class, holding office hours, answering phone calls, and teaching—but the teachers in the computer-assisted classrooms reported spending significantly less time on certain types of activities. They reported spending 22% fewer hour s preparing for class, 13% fewer hours teaching, and 15% fewer hours responding to students’ work than did their colleagues teaching in traditional classrooms. They also reported spending more time in office hours, advising, and orientation than their co lleagues, but on the whole devoted 6% fewer hours to their teaching duties. On the whole, a 6% difference may not amount to much over the course of a year, but the difference in types of activities is intriguing. What about teaching in our computer-assi sted classrooms created situations in which more time was spent in personal interactions with students and less time in written responses to students’ work? And why would computer classes be spending less time in class, given that all sections met for th e same time periods? These differences were discovered in a very small group of respondents—about half of the part-time faculty responded to the faculty effort survey—and one must be cautious interpreting the results.

We have used the information from both parts of the Flashlight survey to develop a set of queries for a new committee that is spearheading the English department’s involvement in the national Epiphany Project, funded by Annenberg/CPB. The Epiphany Project is designed to foster curriculum and faculty development. The Flashlight results provide us with some baseline data for these efforts, and will be used to shape approaches to teaching with technology in our core writing courses. The queries that are guiding our work as we respond to this information are:

· Should there be more curricular differences between the computer-assisted sections and traditional sections?

· Are students with low technical proficiency at the start of the semester not forming close relationships with their instructors?

· How can training for faculty and students obviate differences in prior technical expertise?

· Can we untangle the influence of instructional technology from the influence of the course curriculum?

As noted above, many dimensions of our program encourage similarities among writing sections, and given that these are sections of the same course, similarity is essential for consistency and standards. The common syllabi and common po rtfolio expectations naturally create common practices among sections; furthermore the fact that the computer sections meet in the computer classroom only once a week, and in a traditional classroom once a week, mitigates the actual differences in class e nvironment. However, it would appear that the potential of the networked computer classrooms may not be fully realized, in that the curriculum and portfolio expectations do not take into account the electronic writing students may do. As graduates are i ncreasingly expected to possess an array of technical skills, and the calls for computer use in schools become increasingly common, students need to be encouraged to explore the potential of electronic literacy. In the curriculum revision that has been o ngoing since the end of our participation in Flashlight, we have considered the ways in which students can use some of the work they do in the computer classrooms to demonstrate their growing proficiency with the course goals. A wider variety of portfoli o possibilities should open the door for the electronic writing to be one part of the final portfolios. The e-mail and synchronous conferencing that students do should be part of the work they can draw on to demonstrate their having met course standards, and we anticipate that greater curricular openness on our part with encourage students to think about electronic literacy more broadly.

Of course, we must also consider that these findings suggest that our traditional classrooms are not the stuffy, teacher-centered classrooms that are often opposed to vigorous, on-line environments. The survey data suggest that faculty and students are often in contact with each other, and that all our students take collaboration seriously. This is an element of all our writing classes that we seek to preserve as we move into a revised curriculum for the coming academic year. These f indings, overall, reflect a community of writers at work, engaged in collaborative activities, seeking out instructors outside of class, and sharing work with each other. That these practices happen in all sections is a happy finding.

However, we remain concerned about the students who report low technical proficiency, especially given our urban mission. As an institution that is committed to serving the widest range of students in central Indiana, we offer ways to help underprepared students succeed in college. Thus, we are working on ways to identify these students at the start of each year, and one of the tasks that the department’s Epiphany committee will tackle in the fall is the development of teaching strate gies that can help these students manage some of the technological problems they have (such as failure to gain campus access to computers). We may want to target particular forms of instruction, or resources, for these students.

The starting point will be the training given to teachers and to the student consultants who assist teachers during each session in a computer classroom. Our training programs will support individualized technical instruction, making b etter use of the course consultants who are available during each class period in the computer classrooms. Faculty reported that neither they nor their consultants were familiar enough with the software in the computer rooms (campus-level changes led to frequent changes in our own rooms). If consultants are better trained on the software in use in writing classes, and we make stronger efforts to help students with their technical problems early, the poor learning practices (resistance to contacting inst ructors outside of class, e.g.) that are associated with low technical proficiency should vanish. We need to provide teaching environments in which students with low technical skills can get the help they need, so that close relationships with their inst ructors do not suffer.

Faculty training is an important element of this effort, and we are re-designing our faculty orientation to computer-assisted instruction in order to ensure that faculty get help not only in learning the software and hardware, but also in how to best introduce students to the technology. We are planning to incorporate training for consultants with training for our instructors, giving the consultants a much broader knowledge base and a sense of how the technology fits with the course pe dagogy. This should enable us to target the students with low proficiency for better, more individualized, instruction. Both faculty and consultants need to be able to help students with technical matters in class; faculty need to be familiar with the general workings of software they are asking students to use. Faculty orientation that stresses such pedagogical issues will meet faculty needs more directly, thus decreasing faculty resistance to the added orientations they must attend for these teachin g assignments.

As a final note, having completed a broad study of three very different writing courses, we must step back to consider the ways in which technology can play a very different role in a more advanced course in business and professional wr iting or a developmental course in basic writing. In part because of the project’s findings, we moved to re-schedule classes so that all sections of our other second-semester course (academic research-based writing) could also meet in the computer classr oom (which, unfortunately, pushed some introductory composition sections out of the computer rooms). The fit between curriculum and technology in that course was as pressing as the fit between curriculum and technology in the business and professional wr iting course. In the coming year, as curriculum evaluation takes place on a more formal basis, we will be evaluating the relationship between instructional technology, student attitudes about assignments, and portfolios.

On the whole, our involvement with the Flashlight project demonstrated that our students are largely satisfied with their courses. Our students range from enthusiastic to neutral about the value of their courses and the place of techn ology in their courses. This baseline data allows us to proceed with curriculum revision with clearer insights into students’ reactions to their writing courses and clearer senses of how students spend their time during a semester. The economic analysis and survey data have provided a very broad picture of the issues facing the introductory writing program, and serve as an excellent jumping-off point for future work.