I-CANS Logo -- Click here to return to the I-CANS home page
Chapter 1 Chapter 2 Chapter 3 Chapter 4 Chapter 5 Chapter 6 Chapter 7 Chapter 8

Previous Section | Chapter 2 Table of Contents | Next Section

Along the Way: Ongoing Assessment Tools
from Making Meaning Making Change, by Elsa Roberts Auerbach

One of the most important ways to develop student participation in curriculum development and evaluation is by structuring ritualized procedures, built in on a regular daily, weekly or monthly basis, for collecting and reflecting on student work. These activities give participants immediate feedback about the effectiveness of learning and teaching, allowing adjustments to be made along the way; in addition, they provide a basis for reviewing progress at the end of the cycle.

In-class goal-setting activities
Early classes can focus on tasks designed to elicit student conceptions about language learning, student-teacher roles, and student and teacher goals for the class. Many of the activities described in Chapter 4 can be used in this way. In addition, "The English Class" (Unit I, Lesson 3 of ESL for Action, Auerbach and Wallerstein, 1987) includes activities to generate discussion about reasons for studying English, attitudes toward language learning, and a model for ongoing evaluation of each lesson. Andy developed a lesson (Nash, 1989) based on group responses to "It's easy to learn when_____; It's hard to learn when_____." From this, a set of class resolutions emerged as the basis for evaluation.

Action evaluations
Once students have addressed a problem through individual or group action (like testifying at a funding hearing, dealing with issues of family or classroom dynamics), they can reflect on what happened with dialogue questions like: What happened? How did you feel about it? Why did it happen this way? What might you do differently next time? What new issues have arisen from this action?

Charts/checklists
Students can make charts reflecting what they can and can't do, do and don't like, want and don't want to learn. These can be done individually or as a group, daily, weekly or monthly. East End Literacy (1990) and Nunan (1988) include a range of checklist formats that can be adapted for particular groups or students. The
following questions are adapted from a format suggested by Nunan (1988, p. 134):

This week I studied__________.

This week I learned__________.

This week I liked____________.

This week I didn't like:________.

This week I used my English in these places:

This week I spoke with these people:

This week I had difficulties with_______.

I would like to know/ work on_________.

My learning and practicing plans for next week are_________.

Individual student journals
Journals can be used for assessment in a number of ways. Balliro (1989) suggests building in 15 minutes at the end of each class in which the teacher and students each write journal entries. Students can reflect on their own learning, assess their progress using English, and report on accomplishments, writing about reactions to classroom experiences, interactions using English outside of class, family interactions, or anything else that is on their minds. As they develop, journals provide concrete evidence of students' progress. Teachers can evaluate them in terms of criteria like range of topics/ content, elaboration of ideas (including use of details, examples, depth of analysis, emotional force, etc.), length of entries, grammatical development (specific forms like tense markers, fragments, etc.) and coherence as well as in terms of students' own perspective on their learning. Students can use them for self-evaluation by reading and responding to the finished products, noting changes and areas needing work.

The group journal
Sauve (1987) describes a group process whereby everyone contributes reflections at the end of each class in response to questions like, "What happened today? What did we do today? What did we learn today?" This provides a sense of the differing perspectives in the group, forces the group to name what they have done, and encourages collective responsibility. It can be done as an LEA activity; journals can be collected as a class history.

The posted journal
Charo's class used an evaluation procedure that involved posting a sheet of newsprint in class with the word "Accomplishments" at the t op and two columns, one called "In Class" and the other called "Out of
Class". Whenever anyone had something to report that they felt good about, they wrote it on the list.

Class newsletters
Andy developed a class newsletter in which she summarized the activities of the week as a vehicle for reflecting on learning and discussing accomplishments. She included points covered in the lessons (grammar, readings, etc.), reports on class discussions, attendance, and accounts of individual students' problems or achievements. The newsletter served an number of functions, from reviewing lesson content, to becoming a reading text, to catalyzing action about class issues (like attendance), to documenting the progress of the class and becoming an evaluation tool. See "Our Class: A Weekly Literacy Ritual" in Talking Shop (Nash, et al., in press).

Portfolios
Student writing can be collected in individual portfolios that include everything from informal free-writing assignments to all the drafts of each piece from the beginning to the end of the cycle. These become records of development that both teachers and students evaluate periodically or at the end of the cycle. Students then see concrete representations of their own growth and they can be asked to comment on changes they note. Teachers can look for development of spelling, grammar, coherence, organization, elaboration of ideas, etc.

 

ANECDOTES

Hemmingdinger (1988) identifies anecdotes as an important tool for legitimating the many ways that literacy changes students' lives. It defines an anecdote as "an account of someone, describing what you noticed about the student in the beginning and how the student has changed since then" (p. 128). These stories describe changes that don't show up in direct paper-and-pencil-assessment procedures including affective changes in self-confidence, openness, group participation, ability to make a living, etc. In a participatory approach, they go beyond personal changes to include the ability to use literacy to address social problems: to work with others to make changes in family and community life.

Anecdotes serve two functions. First, they provide valuable feedback to the learner; because of this, they should be written with the learner in mind, using language that is accessible and content that can be shared. Second, they are a means of reporting changes to others in a systematic format. To accomplish these goals, East End Literacy suggests that anecdotes have two components, one that is descriptive and another that is analytical. The former tells the story of the incident indicating change, comparing the student before and after. The latter entails labeling or categorizing changes to provide a schema for documenting them. This is Not a Test: A Kit for New Readers (East End Literacy, 1990) describes in detail how to identify, keep track of, and summarize changes. It suggests a format in which the description is written on the right hand side of the page with corresponding categories of changes listed alongside them in the left margin. Categories of change can be summarized at the end of a collection of anecdotes with short phrases providing examples of each category.

 

LOOKING BACK: END OF CYCLE EVALUATIONS

The following activities for the end of the cycle invite students and teachers to reflect on what they have and haven't accomplished; again students should have the option of doing these in either English or their first language.

Peer interviews
Students can interview each other using questions they have generated collectively. These may be framed in terms of initial goals or more general questions like "What are the most important things you learned in this class? What can you do now that you couldn't do before? What changes have you made since you began this class? What did you like most about this class? What should be changed about the class?" Students can report each other's answers to the whole group and compare impressions.

Student-teacher conferences
Students and teachers can use the same questions they started with at intake, comparing before and after responses.

Review and repetition of earlier tasks
Students can review their portfolios, journals, and coursework to see changes. They can repeat reading and writing sample tasks and compare results; this can be done individually, with peers, or with the teacher.

Student self-evaluations
Students can use chart, checklist or narrative writing formats to evaluate their own learning. If they used checklists to identify goals, interests, and needs earlier, they can come back to these and determine whether their goals have been met and what they still need to work on. See Nunan (1988), pp. 131-134).

Class evaluations
Students can be invited to provide feedback about the class either during or at the end of a cycle. Because students are often reluctant to express negative feelings or criticism, questions should be impersonal; students can be asked to write anonymous evaluations or work in groups so that no individual's ideas are identified. In addition, it helps to have specific questions about what participants disliked or would change. For a beginning class, Ann used questions like "How do you usually feel in class? Is the class too easy or difficult for you? What could improve the class?" At a higher level, she asked, "What kind of atmosphere did you expect to find in classes before you came? What did you find that you didn't expect to find? What didn't you find that you expected to find? In what situations did you use what you learned in the class? What were the games you learned the most from?

Program evaluation
Students from various classes can come together to discuss programmatic issues like class structure, content, use of the native language, child care, scheduling, class size, groupings, and funding concerns. Ann Cason details the processes and benefits of this kind of evaluation in Talking Shop ("All-Program Evaluations").

TEACHER RESEARCH: ONGOING DOCUMENTATION PROCEDURES

Finally, teachers can document their own curriculum development processes through the following:

  • Retroactive lesson plans
  • Teachers' journals
  • Tape recordings
  • Monthly reports
  • Minutes of meetings

The cornerstone of qualitative evaluation is documenting what is happening in the classroom as it happens. As Johnston (1989,p. 509) says,

"Central to this approach is the teacher's ability to know the students, and to notice and record their development in a variety of areas...the ability to set the conditions for and to notice patterns of activity and changes in those patterns is at the heart of the teacher's evaluative skill.

This kind of evaluation provides the context within which to understand students' progress, the basis for curriculum decision-making, and a record of changes that can become data for further analysis. Thus, teachers become researchers of their own classrooms.

Both the process and the product of documenting curriculum development serve important functions. Since documentation is done in an open-ended, descriptive waycollecting and recording data without predetermining what to look for (as in ethnographic research)the process itself becomes a vehicle for listening to and valuing the unexpected. It enables the teacher to stand back from the immediate moment and reflect on it, which, in turn, may lead to new insights about patterns and issues. Just by writing or talking about what is happening in their classes, teachers gain new understandings of why it is happening and what to do next.

Very often, however, insights about what is happening don't come until much later. It may not be clear how to use information as it is being gathered. Thus, the ongoing accounts can serve a retroactive function, becoming "data" for future reflection; they provide invaluable information as recorded histories of class cycles, student progress, and teacher thinking. Teachers in our project, for example, collected teaching materials, student writings, notes, and journal-like descriptions of particular teaching cycles. While they were in the middle of the cycles, they had one perspective on them but when they wrote about them later, their perspectives changed: "Putting our experiences down on paper, we have been forced to reflect on them as we may not have previously done. In this way, we've learned not only from each others' writings but from our own as well" (Nash et al., in press). Likewise, it was the cumulative, detailed documentation of day to day activities, discussions, and student work that provided the data for the analysis which has emerged in writing the curriculum guide. There are many forms of documentation ranging from very structured and schematic to open-ended and flexible. Some that we have used are described on the next pages".

Retroactive lesson plan forms
A uniform format for keeping track of activities and student reactions to them can be used on a daily or weekly basis. We started with a documentation chart with columns for noting where themes came from (catalysts), how they were developed (tools / activities), how language work was incorporated, what new issues emerged, and teacher's reflections. Although teachers in our project felt it was artificial and inhibited rich description, others might find it useful, particularly in moving from a traditional to a participatory approach.

Teachers' journals
Teachers in our project felt that journals were a more organic way of documenting day to day classroom interactions. They are more open-ended and personal. Journal entries, taken together, become a kind of history of the development of the curriculum as well as a record of particular events. While they at first seem time-consuming, the payoff is worth while, as Lucille Fandel (one of my graduate students at the time) writes here (personal communication, May, 1990):

Taking time to keep a journal after every class made a qualitative difference in my teaching, helped that lessons flow better. Before, I'd do a lesson plan based on how much we'd "covered" of my previous plan. With my journaling I've found that I remember significant things that happen (often very fast, in a fleeting way) in class. This way I can pick up on them. They are the "stuff" of people's lives / of our personal interactions / of their evaluations of the class or of their own progress. Journaling has helped me see more clearly where the "flow" is going.

Family literacy teachers kept journals in which they described activities, student reactions, issues, or concerns that arose for them, their own reactions or reflections about the interactions, and ideas for future lessons.

Tape recording
Taping classroom interactions can provide raw data for future analysis. Madeline used tape recording for the purpose of monitoring small group discussions to get a sense of how one group was doing while she was working with another. Listening to the tapes revealed both interesting student issues and areas for language work. She shared the tapes with the whole class in the form of transcriptions. Thus, students could themselves reflect on the interactions, exploring both the content and linguistic aspects together. In this way, the documentation fed into instruction and students were involved in the analysis.

We also used tapes to compare reactions to materials and activities. When teachers used the same code to generate dialogue about homework, Andy taped each class on the day it was used and transcribed the tapes as a basis for teachers to analyze and compare responses. Since each group reacted differently depending on how the lesson was introduced and who the students were, the transcription enabled teachers to see concretely how the context shaped dialogue.

Monthly reports
Summaries of the month's activities provide an overview of activities, accomplishments, and issues for groups. The process of writing these reports can be a framework for reflection, a time for teachers to think about what they've done and where they're going. The reports can also provide a place to communicate problems, needs, and concerns to program administration. Depending on the point in program development, these might address questions about recruitment, intake, students, classroom activities, critical incidents or anecdotes, new insights, ideas, or issues (what you've learned about your own teaching, reflections, goals, challenges). It is also helpful to include examples of materials, student work, journal entries, and retroactive lesson plans.

Minutes of meetings
There are several limitations to these forms of documentation done by individual teachers. First, they take more time than many teachers have. Second, while each may reflect what's happening in particular classes, separately they may be fragmented and
fall short of reflecting the broader picture for a project. Third, because by nature they are the result of an individual process, they preclude the kind of insight that comes from collective analysis. Thus, the dialogue that takes place in teacher meetings and the recording of that dialogue through minutes are central to the curriculum development and evaluation process. The meetings provide a framework for the development of a "community of knowledgeable peers" (Balliro, 1989) and a context for the program-based practitioner research called for by Lytle (1988). The minutes provide a detailed, sequential documentation of what transpires in this process; they become the thread that ties together individual accounts.

 

PUTTING IT ALL TOGETHER: A CONTEXT-SPECIFIC APPROACH TO EVALUATION

Although the system of evaluation proposed here isn't neatly packaged into a ready-made sequence we hope it will be useful as programs construct their own evaluation schemes. Specifically, we hope that its rationale and principles can help to challenge demands for standardized testing and to justify qualitative approaches to evaluation, that the "toolkit" can be a resource as programs select and adapt tools for their own contexts, and that the documentation processes will help teachers gather information that is useful for their own curriculum development as well as for reporting to others (administrators and funders). Most importantly, however, the results of this kind of evaluation will help to develop the field of adult education. As more and more practitioners document, analyze, and share what they're doing, our collective understanding of what does and doesn't work will grow so that research that comes from inside the classroom will become the basis for constructing and extending our knowledge about adult learning.

Back to Top | Next Section