"Expert" on Writing Assessment

Becoming an "Expert" on Writing Assessment
by studying Chapter 6 from Authentic Assessment for English Language Learners: Practical Approaches for Teachers by J. Michael O'Malley and Lorraine Valdez Pierce. pages 135-161

Focus: The Authentic Assessment of Writing -- [What does it mean when assessment is authentic?]
Chapter Overview:
- overview of the nature of writing in school
[My thoughts: We ask them to do many kinds of writing. Most seems to be written to the teacher, though there is a trend to having students write for other audiences as well. Writing is done to prove learning, as well as to obtain learning/understanding. Many teachers expect conventions to be handled correctly on almost any assignment. Teachers are also trying to let students have more choice in selecting topics for their writing. Because of heavy student loads, teachers may not have much time to respond to student writing.]
- the nature of the writing task [My thoughts: There are similarities between the task of a native speaker and the task of a language learner, though there may be vast differences in degree of effort required. All are looking for the right words in the right order to express the right ideas. The English language learners, however, are more often looking entirely away from their own background knowledge for those words, and for the order in which they should be presented.]
- types of authentic assessment in writing
- scoring rubrics
- self and peer assessment
- instructional uses of assessment results
- interactions between instruction and writing assessment

On the Nature of Writing in School
Teacher judgment is important.
Writing is judged on
-- substantive information contained in the message
-- clarity of the message
-- mechanics of writing

Teacher centered approach:
-- Teacher defines topic.
-- Teacher establishes criteria.
-- Teacher grades the writing.
-- Students write in isolation from reading and other literacy activities.
-- Teachers tend to overemphasize mechanics and grammar.

New Views on Writing (1996)
THE WRITER
Writing = a personal act -- writers take ideas or prompts -- transform them into "self-initiated" topics -- draws on background knowledge, mental processes -- leading to new insights.
The writer needs to combine PURPOSE or PROMPT with his or her own UNIQUE APPROACH TO WRITING.

Kinds of Knowledge Needed for Writing based on a a shared experience:
1. Knowledge of the content
--- accessing memory of prior knowledge, experiences, and generating ideas

2. Procedural knowledge to organize the content
--- grouping and sequencing ideas to match the purpose of the writing
--- creating goals or a plan for organization

3. Knowledge of discourse structures, syntactic forms, and conventions
For example, is the student writing a persuasive piece, a description, or a explanation?

4. Procedural knowledge for integrating all the other types of knowledge.
"Students need extensive opportunities for writing in which all the types of knowledge are combined as they compose a message for a purpose with a particular audience." p. 137

Implications for Writing Assessment with ELL Students:

1. Writing assessment should evaluate more than just mechanics and grammar. [Amen!]
2. Writing assessment should capture some of the processes and complexity involved in writing so that teachers can know in which aspects of the writing process students are having difficulty. p. 137
3. Writing assessment should look at the context in which the writing occurs.

PURPOSES AND TYPES OF WRITING
Purpose determines the nature of the writing.
purpose and genre --
We need to help students gain control over various genres.

PURPOSE
Three purposes in writing:
1) informative
2) expressive/narrative
3) persuasive

-- Student writing ability may vary depending on the purpose.
-- Student writing ability may vary among pieces with the same purpose, depending on prior knowledge or interest.

Key point: ASSESS ACROSS A WIDE VARIETY OF PURPOSES AND PROMPTS.


Genres/Types of Writing include biographies, essays, stories, journal entries, newspapers, advertisements, and so on.


Writing Instruction
Research has shown the weakness of a traditional approach to writing instruction. What students learn about vocabulary, spelling, mechanics, and grammar don't transfer well to new writings when students have learned each in relative isolation.

Process Writing
-- personal interest
-- writing shared with peers, who ask questions, comment, encourage
-- student-teacher conferences
-- editing and revision is done after receiving feedback
-- interactive learning
-- emphasis on process
-- focus on meaning (making meaning?)

-- teachers model
3 stages in the process:
1) prewriting
2) writing [We call it drafting.] [This text includes revision and editing in the writing stage.]
3) postwriting [We call it revision, editing, and publishing.]

-- Conferencing is important in which the teacher meets with the student and they discuss. reflect on the process they have followed. [metacognition] Assessment can take place during conferencing.

Writing Across the Curriculum
"Student who write about topics tend to understand them better." p. 139
"Student write to learn rather than learn to write."
Students use writing to
-- manipulate information
-- consolidate prior learning
-- prepare for future learnign
-- extend or reformulate prior learning

Authentic Assessment of Writing

Two important components in the authentic assessment of writing are
-- the nature of the task
-- the scoring criteria

Overview of this section--
-- guidelines for constructing writing tasks
-- examples of types of scoring criteria
-- ways to review students' developmental level of writing and the processes they use
-- getting students to rate their own interest in and awareness of writing
-- getting students to rate the quality of their own writing [Teaching them 6-Traits Assessment?]
-- suggestions for peer assessment of writing

The PROMPT includes
-- the question or statement students will address
-- the conditions under which they will write
-- amount of time -- resources available -- how much of the "process" will be used -- platform to be used

Make sure your prompts reflect classroom instruction.
Provide ample time for students to complete the writing task.
May specify purpose and audience.

-- Give students a say in whether you will offer them a single prompt or multiple prompts from which they may choose.
-- Let them help generate prompts.
-- Choice among prompts can be helpful, but students may spend too much time trying to decide among prompts

Provide students with practice for the types of prompts they will meet on tests [including state tests] and in other classrooms.

[Your prompts should be designed to get the students started, giving them enough to get them going, but not giving them so much as to restrict their unique voices. ]

An effective prompt will do the following:

-- invite the desired type of writing or genre
-- engage the thinking, problem-solving composing, and text-making processes central to the type of writing
-- be challenging for many students and accessible to all [a fine balancing act!]
-- provide equitable opportunities for all students to respond
-- produce interesting, not just proficient writing
-- be liked by many students

-- tell the students what will be valued in their writing (criteria for grading)
[The prompts on GoMyAccess are set up this way, with a list of criteria at the end of each prompt.]

Integrated Language Assessment
-- includes reading, speaking, and writing
[I've used prompts such as these while teaching at the junior high, high school, and college level. Another step that could be included is having the students help create the rubric. This also fits with the trend toward using many short pieces of text in teaching rather than fewer longer texts.]

Day Activity Description
1 Reading -- Students read background materials on topic/take notes.
2 Discussion -- Students discuss readings in small groups/modify notes.
optional -- [Collaborate to Create Rubric] [Teacher and students collaborate to create or to revise the rubric to be used for grading .]
3 Draft 1 -- Students write first draft.
4 Review Rubric and/or Receive Feedback -- Students are given the rubric and discuss its applications. [Time for peer review and teacher conferences]
5 Edit and Revise -- Students edit and revise to produce the final product.

Types of Scoring
-- scoring should be defined before the rest of the assignment is developed.

HOLISTIC SCORING
uses a variety of criteria to produce a single score
Rationale: the total quality of the written text is more than the sum of its components.
[This is described as six-traits scoring.]
See also the "Holistic Scoring Rubric for Writing Assessment with ELL Students" page 143.
This one looks at different developmental levels . Actually, 6-Traits Writing also looks at student development along a continuum.
Find information on Six-Traits Scoring at http://www.nwrel.org/assessment/scoring.php?d=9

PRIMARY TRAIT SCORING
This variation of holistic scoring focuses on one or a couple of traits in an assignment, rather than looking at all traits. Or the scoring could focus on some aspect of content. Other features are ignored in the scoring.

ANALYTIC SCORING
Each component is scored separately. Different aspects of the writing may be given varied weight in the scoring. The total score is a sum of the weighted component scores. Measures degree of control over each aspect of writing. See page 145.

Stages of Writing Development
See pages 146 through 147.


The level of writing shown by a student will depend in part on the prompt. This may depend on the familiarity of the topic or the level of interest in the topic.

Monitoring Student Progress in Process Writing
-- direct observation of writing
-- conferencing

Strategies for Process Writing
See the Process Writing Checklist 6.4.
[Note to myself: Keep a portfolio of writing samples and assessment for ELL students.]

Writing Conferences
Questions depend on
-- the writer
-- the purpose of the writing
-- the instructional focus for the student or assignment
Issues: time, ways to keep records, what to do with the other students
See page 150 for sample questions.

Written Summaries
Summarizing includes
-- deleting minor details and redundant information
-- combining similar details
-- selecting or composing main idea sentences (Casazza 19920

Students can practice with T-Lists or graphic organizers.
Students can write summariies in dialogue journals and learning logs.
See figure 6.5. for a scoring guide for summarizing.

Self-Assessment in Writing
-- Thinking , reflection, metacognition --

Dialogue Journals
- student writes on his or her choice of topics TO the teacher
- teacher reads and responds -- not to correct student language, but modeling correct language use
-- could be done in a notebook or on the computer
-- five to ten minutes at the end of class, or at another convenient time
-- teacher could keep anecdotal records concerning topic initiation, elaboration, variety, use of different genres, expression of interests and attitudes, awareness of the process of writing.

Learning Logs [We've done something like this called "exit slips."]
Students make entries during the last 5 minutes of learning, answering such questions as
-- What did I learn today?
-- What strategies or approaches worked best for me in learning?
-- What was hard to understand?
-- What do I do to understand better?

Questions to assess learning logs:
-- Does the student define or use new vocabulary?
-- Does the student use content vocabulary appropriately?
-- Does the student identify a range of strategies that worked in learning?
-- Does the student have reasonable plans for improving his or her learning?

Surveys of Interest and Awareness
Students are asked to indicate their attitudes toward writing and to gauge their improvement as writers. See figure 6.6 on page 154.

See the post "Computer Lab Pre-Assessment." I have used this assignment as a beginning year "survey of interest and awareness" for my regular English classes. The assignment was designed to find out about students' interests and awareness, as well as their ability to use the computers for writing.

Writing Strategies
See Figure 6.7 on page 155 for a "Self-Assessment of Writing Strategies" for ELL.
Strategies in the Writing Process:
Before Writing--
-- topic choice and review, organization [gathering ideas, information]
During Writing --
-- advance the writing toward meeting the original purpose
-- used to overcome obstacles or enhance communication through examples, clarification, and drawings.
After Writing --
-- reread, edit, revise to ensure it meets the original purpose
Over a number of written products with varying purposes and genres, students should use many of these strategies.

Writing Checklist -- SELF-ASSESSMENT
Students need to understand --
1) the activity and what proficiency it requires
2) the steps needed to attain proficiency
3) how they are progressing
Students need to check their own writing using rubrics, with descriptors put into their own words.
See figure 5.8 on page 157.

Peer Assessment in Writing
Student writing conferences
Suggestions:
Make each student responsible for finding out how his or her paper can be improved.
Have students pair up and read their papers to each other. [I like to do this in three's.)
Answer these three questions:
-- What did you like about the paper?
-- What facts or ideas could be added to the paper?
-- What changes could be made to improve the paper?
Students could use a scoring rubric to rate each other's papers.
Vary the procedure.
See Figure 6.9 on page 158.

Using Writing Assessment in Instruction
"Writing assessment can be used most effectively with instruction when the criteria for scoring written projects are clear to the students and when the students see an obvious relationship between what they have written and the scores they received."
"Students see the connection between their writing and the scores more clearly with analytic scoring rubrics than with holistic scoring." -- Analytic scoring provides specific feedback.

Students should be familiar with the scoring rubric.

** Students who benefited most from being familiarized with scoring rubrics were ELL and learning disabled students.

Anchor papers (papers representing each of the six points on a scoring rubric -- or as in the 6-traits training I've had, at least papers represent point 1, 3, and 6) can be shown to students It is particularly useful to show a student the next highest level on the rubric, then allow them to revise their papers to match what they see in the anchor paper. (GoMYAccess does this for students, with anchor papers available online.)

Best Practices -- to Maximize the Interaction between Writing Assessment and Instruction:
1. Select prompts that are appropriate for the students.
-- invite the desired type of writing
-- engage the type of thinking and problem solving you want to access
-- are both challenging and accessible to all students
-- are sensitive to cultural backgrounds of students
-- produce interesting writing
-- are enjoyed by the students
(Discuss with the students whether they would prefer to select the topic for the prompt from a list of options or to suggest prompts themselves.)

2. Select rubrics students can use.
-- keep the rubric uncomplicated [I have a great 6-traits rubric for middle school/junior high kids.]
-- try out holistic, analytic, and primary traits scoring

3. Share the Rubrics with students.
-- let them use the rubric on their own writing and to use it to evaluate the writing of their peers. -- let it become a natural part of their editing process as they review their own work. [Though they said you should use various types of rubrics, this sounds as if you should have a basic rubric that is used most of the time.]

4. Identify benchmark papers.
-- These are papers you have rated high on the components of your scoring rubric.
[By the way, if you have not taught writing, look for 6-Traits materials such as Vicki Spandel's book Creating Writers or Ruth Culham's 6 + 1 Traits of Writing. Both books provide rubrics and benchmark papers, as well as what you need to know to teach 6-Traits writing.]
-- Use models from both ELL students and native English speakers, if possible.
[I am wondering if there are six traits materials -- rubrics, benchmark papers -- already adapted to ELL students. I just e-mailed Great Source asking about this.]

5. Review how students write not just what they write.
-- Use checklists that focus on Process Writing and students' writing strategies.
-- Don't, however, ignore essential elements of good writing.

6. Provide time and instructional support for self-assessment and peer assessment.
-- Make time and provide guidance on self- and peer assessment.
-- The teacher doesn't have to rate every paper himself.

7. Introduce self-assessment gradually.
-- Students may(?) need scaffolded support in learning to edit and revise their own work.
-- Model the editing process [and revising process].
-- Or introduce one part of the rubric at a time.
-- Help students generate criteria for good writing.
-- Display charts of writing criteria.

8. Use conferencing to discuss writing with students.
-- Ask students key questions about their writing processes.
-- Provide students with personalized feedback.
-- Inquire about the effectiveness of instructional materials or approaches you have tried.

Hint: You could give students rubrics first in their native language.

WRITING IS PRESENTED AS A PROCESS THAT MUST BE UNDERSTOOD AND APPLIED RATHER THAN SIMPLY A PRODUCT TO BE EVALUATED. p. 160

GGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGG

Writing process image found at http://writingportfolio.net/res/siwp.jpg

Image of girl writing found at

at: www.readwritethink.org/.../grades6_8/FoundPoems/