Makalah Test and Evaluation Assessing Reading and Writing

Makalah
Test and Evaluation

Assessing Reading and Writing

 







Presented by:

1.  Aldo Novanda (1255

2.  Elviyasa Gaberia Siregar (12551127)

3.  Putri Liana Fransiska

Education Department Of English Study Program

State College For Islamic Studies (STAIN) Curup

2014

Assessing Reading

            In foreign language learning,reading is likewise a skill that teachers simply effect learners to acquire.Basic,beginning level textbooks in a foreign language presuppose a student’s reading ability if only because it’s a book that is the medium.Many formal test use the wrritten word as a stimulus for test-takers response;even oral interviews may require reading performance for certain tasks.Reading,arguably the most essential skill for success in all educational contexts,remains a skill of paramount importance as we create assessment of general language ability.

There are two primary hurdles must be cleared by learners to become efficient readers:

1)      They need to be able to master fundamental bottom-up strategies for processing separate letters,words,and phrases as well as top-down,conceptually driven strategies for comprehension.

2)      As part of that top-down approach,second language readers must develop appropriate content and formal schemata-background information and cultural experience to carry out those interpretations effectively.

The assessment of reading ability does not end with the measurement of comprehension.It is also important,especially in formative classroom assessment,to assess the strategies that readers use-or fail tro use-to achive ultimate comprehension of a text.

Genres of Reading

Each genre of written text has its own set of governing rules and conventions.A reader must be able to anticipate those conventions to process meaning efficiently.With an extraordinary number of genres present in any literate culture,the reader’s ability to process texts must be very sophisticated.Consider the following abridged list of common genres,which ultimately form part of the specifications for assessments of reading ability:

1.      Academic reading

·         Generas interest articles

·         Technical reports

·         Reference material

·         Essays,papers

·         Test directions

·         Editorials and opinion writing

2.      Job-related reading

·         Messages

·         Letters

·         Memos

·         Reports

·         Schedules,label,sign,announcements

·         Forms,aplications,questionnaires

·         Financial documents

·         Directories

·         Manual,directions

3.      Personal reading

·         Newspapers and megazines

·         Letters,emails,greeting cards,invitations

·         Messages,notes,lists,blogs

·         Schedules

·         Recipes,menus,maps

·         Advertisments

·         Novel,short stories

·         Financial documents

·         Forms,quetionnaires,medical reports,immigration documents

·         Comic strips,cartoon

The genre of a texts enables readers to apply certain schemata that assists them in extracting appropriate meaning.Efficient readers also must know what their purpose is in reading a text,the strategies for accomplishing that purpose and how to retain the information.

Microskills,Macroskills,And Strategies For Reading

The following macroskills represent the spectrum of possibilities for objectives in the assessment of reading comprehension.

Microskills 1.      Discriminate among the distinctive graphemes and orthographic patterns of English 2.      Retain chunks of language of different lengths in short-term memory 3.      Process writing at an efficient rate of speed to suit the purpose 4.      Recognize a core of words and interpret word order patterns and their significance 5.      Recognizen grammatical word classes,patterns,rules,and elliptical forms 6.      Recognize that a particular meaning may be expressed in different grammatical forms 7.      Recognize cohesive devices in written discourse and their role in signaling the relationship between and among clauses

Macroskills 8.      Recognize the rhetorical coventions of written discourse and their significance for interpretation 9.      Recognize the communicative functions of written textts,according to form and purpose 10.  Infer context tha is not explicit by activating schemata 11.  From described events,idea,infer links and connections between events,deduce causes and effects,and detect such relations as main idea,supporting idea,new information,given information,generalization,and exemplification 12.  Distinguish between literal and implied meaning 13.  Detect culturally specific references and interprete them in a context of the appropriate cultural schemata 14.  Develo and use battery of reading strategies.

Some principal strategies for reading comprehension

1.      Identify your purpose in reading atext

2.      Apply spelling rules and conventions for bottom up decoding

3.      Use lexycal analysis to determine meaning

4.      Guess at meaning when you are not certain

5.      Skim the text for the gist and main ideas

6.      Scan the texts for specific information

7.      Use silent reading techniques for rapid processing

8.      Use marginal notes,outlines,charts,or semantic maps for understanding and retainning information

9.      Distinguish between literal and implied meanings

10.  Use discourse markers to process relationship

Types of Reading

In the case of reading,variety of performsnce is derived more from the multiplicity of types of texts than from the variety of overt types of performance.These are various assessment tasks:

1.      Perceptive: involveattenting to the components of larger stretches of dicourse

2.      Selective:to ascertain one’s reading recognition of lexical,grammatical,or discourse features of language,within a very short stretch of language certain typical tasks are used: picture-cuedbtasks,matching,true/false,multiple-choice,etc.

3.      Interactive: included among interactive reading types are stretches of language of several paragraphs to one page or more in which the reader must,in a psycholinguistic sense,interact with the text.The focus of an interactive tasks is to identify relevant features within texts of moderatelyshort length with the objective of retaining the information that is processed.

4.       Extensive: it applies to texts of more than a page,up to and including professional articles,essays,technical reports,short stories,and books.The purposes are to tap into into a learner’s global understanding of a text,as opposed to asking test-takers to’zoom in’ on small details.

Designing Assessment Tasks: Perceptive Reading

Such tasks of perception are often referred to as literacy tasks,implying that the learner is in the early stages of becoming ‘literate’.Some learners are already literate in their own native language,but in other cases the second language may be the first language that they have ever learned to read.

Reading Aloud  (S,R)

The test-taker see separate letters,words,and/or short sentences and reads them aloud,one by one,in the presence of an administrator.Because the assessment is of reading comprehension,any recognizable oral approximation of the target response is considered correct.

Written Response (R,W)

            Test-taker’s task is to reproduce the probeing  writing.Because of the transfer across different skills here,evaluation of the test-taker’s response must be carefully treated.

Multiple-Choice

Other formats of multiple-choice include same/different,circle the answer,true/false,choose the letter,and matching.

Picture-Cued Items

Test-takers are shown a picture,such as the following,along with a written text and are give one of a number of possible tasks to perform.With the sam picture,the test-taker might read sentences,choose true/false possibility,or matching to the picture.Finally,test-taker might see a word or phrase then directed to choose one of four pictures that is being described,thus requiring him or her to transfer from a verbal to a nonverbal mode.

Designing Assessment Tasks: Selective Reading

The category includes what many incorrectly think of a testing ‘vocabulary and grammar’.

Multiple-Choice (for Form-Focused Criteria)

Multiple-choice format become popular method of testing a reading knowledge of vocabulary and grammar because of its practicality,easy to administer,and can be scored quickly.

Matching Tasks

At this selective level of reading,the test-taker’s task is simply to respond correctly,which makes matching an appropriate format.The most frequently appearing criterion in matching procedures is vocabulary.To add a communicative quality to matching,the first numbered list is sometimes a set of sentences with blanks and a list of of words to choose from.

Editing Tasks

            Editig for grammatical or rhetorical errors is a widely used test method for assessing linguistic competence in reading.The TOEFL tets and many other tests employ this technique with the argument that it not only focuses on grammar but also introduces a simulation of the authentic task of editing,or discerning errors in written passages.ex: multiple choice grammar editing task.

Picture-Cued Tasks

Several types of picture-cued methods are commonly used.

1.      Test-takers read a sentence or passage and choose one of four pictures that is being described

2.      Test-takers read a series of sentences or definitions, each describing a labeled part of a picture or diagram. Their task is to identify each labeled item.

Gap-Filling Tasks

Many of the multiple-choice tasks described above can be converted into gap-filling, or "fill-in-the-blank,"'items in which the test-taker's response is to write a word or phrase. An extension of simple gap-fIlling tasks is to create sentence completion items where test-takers read part of a sentence and then complete it by writing a phrase.

DESIGNING ASSESSMENT TASKS: INTERACTIVE READING

Ø  Cloze Test

In written language, a sentence with a word left out showed have enough context that a reader can close that gap with a calculated guess, using linguistic expectancies (formal schemata), background experience(content schemata), and some strategic competence. Based on tIus assumption, cloze tests were developed for native language readers and defended as an appropriate gauge of reading al?ility. Some research (Oller, 1973, 1976,1979) on second language acquisition vigorously defends cloze testing as an integrative measure not only of reading ability but also of other language abilities. It was argued that the ability to make coherent guesses in cloze gaps also taps into the ability to listen, speak, and write.With the decline of zeal for the search for the ideal integrative test in recent years, cloze testing has returned to a more appropriate status as one of a number ofassessment procedures available for te~ting reading ability.Cloze tests are usually a minimum of two paragraphs in length in order to account fof' discourse expectancies. They can be constructed relatively easily as long as the specifications for choosing deletions and for scoring are clearly defmed.. Typically every seventh word (plus or minus two) is deleted (known as Itxed-ratio \/ deletion), but many cloze test designers instead use a rational deletion procedure _11, of choosing deletions according to the grammatical or discourse functions of the words. Rational deletion also allows the designer to avoid deleting words that would be difficult to predict from the context. For example, in the sentence "Everyone in the crowd enjoyed the gorgeous sunset," the seventh word i~ gorgeous, bUrt learners could easily substitute other appropriate adjectives. Traditionilly, cloze passages have between 30 and 50 blanks to fill, but a passage with as few as half a dozen blanks can legitimately be labeled a cloze test.

Two approaches to the scoring of cloze tests are commonly used. The exact word method gives credit to test-takers only if they insert the exact word that was originally deleted. The second method, appropriate word scoring, credits the testtaker for supplying any word that is grammatically correct and that makes good sense in the context. In the sentence above about the "gorgeous sunset," the testtakers would get credit for supplying beautiful, amaZing, and spectacular. The choice between the two methods of scoring is one of practicality/reliability vs. face validity. In the exact word approach, scoring can be done quickly (especially if the procedure uses a multiple-choice technique) and reliably. The second approach takes more time because the teacher must determine whether each response is indeed appropriate, but students-will-perceive-the test-as being-fairer~-theywon't-get "marked off" for appropriate, grammatically correct responses.

Impromptu Reading Plus Comprehension Questions

If cloze testing is the most-researched procedure for assessing reading, the traditional "Read a passage and answer son1e questions" technique is undoubtedly the oldest and the most common. Virtually every proficiency test uses the format, and one would rarely consider assessing reading without some component of the assessment involving impromptu reading and responding to questions.

Short-Answer Tasks

Multiple-choice items are difficult to construct and validate, and classroom teachers rarely have tinle in their busy schedules to design such a test. A popular alternative to multiple-choice questions following reading passages is the age..old short-anSwer format. A reading passage is presented, and the test-taker reads questions that must be answered in a sentence or two. Questions might cov~r the same specifications indicated above for theTOEFL reading, but be worded in question form.

Editing (Longer Texts)

Several advantages are gained in the longer format.

First, authenticity is increased.The likelihood that students in English classrooms will read connected prose of a page or two is greater than the likelihood of their encountering the contrived format ofunconnected sentences. Second, the task simulates proofreading one's own essay, where it is imperative to find and correct errors. And third, if the test is connected to a specific curriculum (such as placement into one of several writing courses), the test designer. can draw up specifications for a number ofgrammatical and rhetorical categories that match the content of the courses. Content validity is thereby supported, and along with it the face validity of a task in which students are willing to invest.

Scanning

Scanning is a strategy used by all readers to fmd relevant information in a text. Assessment ofscanning is carried out by presenting test-takers with a text (prose or something in a chart or graph format) and requiring rapid identification of relevant bits of information. Possible stimuli include:

,r'

•                     a one-to two-page news article,

•                     an essay,

•                     a chapter in a textbook,

•                     a technical report,

•                     a table or chartdepicting some research fmdings,

•                     a menu, and

•                     an application form ..

Among the variety of scanning objectives (for each of the genres named above),the test-taker must locate:

•                     a date, name, or place in an article;

•                     the setting for a narrative or story;

•                     the principal divisions of a chapter;

•                     the principal research fmding in a technical report;

•                     a res1.I,lt reported in a specified cell in a table;

•                     the cost of an item on a menu; and

•                     specified data needed to fill out an application.

Ordering Tasks

Students always enjoy the activity of receivirtg little strips of paper, each with a sentence on it, anti assembling them irito a story, sometimes called the "strip story" technique. Variations on this can serve :as an assessment of overall global understanding of a story and of the cohesive devices that signal the order of events or ideas.

Information Transfer: Reading Charts, Maps, Graphs, Diagrams

Every educated person must be able to comprehend charts, maps, graphs, calendars, diagrams, and the like. Converting such nonverbal input into comprehensible intake requires not only an understanding of the graphic and verbal conventions of the medium but also a linguistic ability to int~rpret that information to someone else. Reading a map implies understanding the conventions of map graphics, but it is often accompanied by telling someone where to turn, how far to go, etc. Scanning a menu requires an ability to understand the structure of most menus as well as the capacity to give an order when the time comes. Interpreting the numbers on a stock market report involves the interaction of understanding the numbers and of conveying that understanding to others.

All of these media presuppose the reader's appropriate schemata for interpreting them and often are accompanied by oral or written discourse in order to convey, clarify, question, argue, and debate, among other linguistic functions. Virtually every language curriculum, from rock-bottom beginning levels to high-advanced, utilizes this nonverbal, visuaVsymbolic dimension. It is therefore imperative that assessment procedures include measures of comprehension of nonverbal media.To comprehend information in this medium (hereafter referred to simply as "'graphics"), learners must be able to

·         comprehend specific conventions of the various types of graphics;

·         comprehend labels, headings, numbers, and symbols;

·         comprehend the possible relationships among elements of the graphic; and

·         make inferences that are not presented overtly.

The act of comprehending graphics includes the linguistic' performance of oral or written interpretations, comments, questions, etc. This implies a process of information transfer from one skill to another: in this case, from reading verbal.

DESIGNING ASSESSMENT TASKS: EXTENSIVE READING

Extensive reading involves somewhat longer texts than we have been dealing with up to this point. Journal articles, technical reports, longer essays, short stories, and books fall into this category. The reason for placing such reading into a separate category is that reading of this type of discourse almost always involves a focus on meaning using mostly top-down processing, with only occasional use of a targeted bottom-up strategy. Also, because of the extent of such reading, formal assessment is unlikely to be contained within the time constraints of a typical formal testing framework, whi,ch presents a unique challenge for assessment purposes.

Another complication in assessing extensive, re,acting is that the expected response from the reader is likely to involve as much written (or sometimes oral) performance as reading. For example, in asking test-takers to respond to an article or story, one could argue that a greater emphasis is placed on writing than on reading. This is no reason to sweep extensive reading assessment under the rug; teachers should not shrink from the assessment of this highly sophisticated skill.

Before examining,a few tasks that have proved to be useful in assessing extensive reading, it is essential to note that a number of the tasks described in previous categories can apply here. Among them are

•                     impromptu reading plus comprehension questions,

•                     short-answer tasks,

•                     editing,

•                     scanning, • ordering,

•                     information transfer, and

•                     interpretation (discussed under graphics),

Skimming Tasks

Skimming is the process of rapid coverage of reading matter to determine its gist or main idea. It is a prediction strategy used to give a reader a sense of the topic and purpose of a text, the organization of the text, the perspective or point of view of the writer, its ease or difficulty, and/or its usefulness to the reader.Of course skimming can apply to texts of less than one page, so it would be wise not to confine this type of task just to extensive texts.

Summarizing and Responding

One of the most common means of assessirtg extensive reading is to ask the testtaker to write a summary of the text. The task that is given to stu(1ents can be very simply worded:

Directions for summarizing

Write a summary of the text. Your summary should be about one paragraph in length (100-150 words) and should include your understanding of the main idea and supporting ideas.

Evaluating summaries is difficult: Do you give test-takers a certain number of points for targeting the main idea and its supporting ideas? Do you use a full/partial!no-credit point system? Do you give a holistic score? Imao (2001) used four criteria for the evaluation of a summary:

1.                  Expresses accurately the main idea and supporting ideas.

2.      Is written in the student's own words; occasional vocabulary from the original text is acceptable.

3.      Is logically organized.

4.      Displays facility in the use of language to clearly express ideas in the text.

As you can readily see, a strict adherence to the1critetjon ;ofassessing reading, and reading only, implies consideration of only the frrst factor; the other three pertain to writing performance. The first criterion is nevertheless a crucial factor; otherwise the reader-writer could pass all three of the other criteria with virtually no understanding of the text itself. Evaluation of the reading comprehension criterion will of necessity renlain somewhat subjective because the teacher will need to determine degrees of fulfillment of the objective.

Of further interest in assessing extensive reading is the technique of asking a student to respond to a text. The two tasks should not be confused with each other: summarizing requires a synopsis or overview of the text, while responding asks the reader to provide his or her own opinion on the text as a whole or on some statement or issue within it

Note-Taking and Outlining

Finally, a reader's comprehension of extensive texts may be assessed through an evaluation of a process of note-taldng and/or outlining. Because of the difficulty of controlling the conditions and time frame for both these techniques, they rest firmly in the category of informal assessment.Their utility is in the strategic training that learners gain in retaining information through marginal notes that highlight key information or organizational outlines that put supporting ideas into a visually manageable framework. A teacher perhaps aone and one conferences-with students, can use student notes/outlines as indicators of the presence or absence of effective reading strategies, and thereby point the learners in positive directions.

Assessing Writing

The ability to write has become an indispensable skill in our global literate community. Writing skill, at least at rudimentary levels, is a necessary condition for achieving employment in many walks of life and is simply taken for granted in literate cultures.

In the field of second language teaching, only a half-century ago experts were saying that writing was primarily a convention for recording speech and for reinforcing grammatical and lexical features of language. Now we understand the uniqueness ofwriting as a skill with its own features and conventions. We also fully understand the difficulty of learning to write "welltl in any language, even in our own native language. Every educated child in developed countries learns the rudiments of writing in his or her native language, but very few learn to express themselves clearly with logical, well-developed organization that accomplishes an intended purpose. And yet we expect second language learners to write coherent essays with artfully chosen rhetorical and discourse devices!

GENRES OF WRITTEN LANGUAGE

1. Academic writing

Papers and general subject reports essays, compositions academically focused journals short-answer test responses technical reports (e.g., lab reports) theses, dissertations

2. Job-related writing

Messages, letters/emails memos, schedules, labels, signs advertisements, announcements manuals

3. Personal writing

Letters, emails, greeting cards, invitations messages, notes calendar entries, shopping lists, reminders financial forms, questionnaires, medical reports, immigration documents diaries, personal journals fiction.

TYPES OF WRITING PERFORMANCE

1.      Imit.ative. To produce written language, the learner must attain skills in the funqamental, basic tasks of.writing letters, words, punctuation, and very brief sentences. This category includes the ability to spell correctly and to perceive phoneme-grapheme correspondences in the English spelling system. It is a level at which learners are trying to master the mechanics ofwriting. At this stage, form is the primaryifnot exclusive focus, while context and meaning are ofsecondary concern.

2.      Intensive (controlled). Beyond the fundamentals of imitative writing are skills in producing appropriate vocabulary within a context, collocations and idioms, and correct grammatical features up to'the length of a: sentence. Meanipgimd. context are of some importance in determining correctness and appropriateness, but most assessment tasks are more 'concerned with a focus on form, and lare rather strictly controlled by the test design.

3.      Responsive. Here, assessment tasks require learners to perform at a limited discourse level, connecting sentences into a paragraph and creating a logically connected sequence of two or three paragraphs. Tasks respond to pedagogical directives, lists of criteria, outlines, and other guidelines. Genres of writing include brief narratives and desriptions, short reports, lab reports, summaries, brief responses to reading, and interpretations of charts or graphs. Under specified conditions, the writer begins to 'exercise some freedom of choice among alternative forms of expression of ideas. The writer has mastered the fundamentals of sentence-level grammar and is more focused on the discourse conventions that will achieve the objectives ofthe written text. Form-focused attention is mostly at the discourse level, with a strong emphasis on context and meaning.

4.      Extensive. Extensive writing implies successful management of all the processes and strategies of writing for all purposes, up to the length of an essay, a term paper, a major research project report, or even a thesis. Writers focus on achieving a purpose, organizing and developing ideas logically, using details to support or illustrate ideas, demonstrating syntactic and lexical variety, and in many cases, engaging in the process of mUltiple drafts to achieve a fmal product. Focus on grammatical form is limited to occasional editing or proofreading of a draft.

MICRO-AND MACROSKII.IS OF WRITING

Microskills

1.      Produce graphemes and orthographic patterns of English.

2.      Produce writing at an efficient rate of speed to suit the purpose.

3.      Produce an acceptable core of words and use appropriate word order patterns.

4.      Use acceptable grammatical systems (e.g., tense, agreement, pluralization), patterns, and rules.

5.      Express a particular meaning in different grammatical forms.

6.      Use cohesive devices in written discourse. Macroskills

7.      Use the rhetorical forms and conventions of written discourse.

8.      Appropriately accomplish the communicative functions of written texts according to form and purpose.

9.      Convey links and connections between events, and communicate such relations as main idea, supporting idea, new information, given information, generalization, and exemplification.

10.  Distinguish between literal and implied meanings when writing.

11.  Correctly convey culturally specific references in the context of the written text.

12.  Develop and use a battery of writing strategies, such as accurately assessing the audience's interpretation, using prewriting devices, writing with fluency in the first drafts, using paraphrases and synonyms, soliciting peer and instructor feedback, and using feedback for revising and editing.

DESIGNING ASSESSMENT TASKS: IMITATIVE WRITING

With the recent worldwide emphasis on teaching English at young ages, it is tempting to assume that every English learner knows how to handwrite the Roman alphabet. Such is not the case. Many beginning-level English learners, from young children to older adults, need basic training in and assessment of imitative writing: the rudiments offorming letters, words, and simple sentences.We examine this level of writing fIrst.

Tasks in [Hand] Writing Letters, Words, and Punctuation

First, a comment should be made on the increasing use of personal and laptop computers and handheld instruments for creating written symbols. Handwriting has the potential of becoming a lost art as even very young children are more and more likely to use a keyboard to produce writing. Making the shapes of letters and other symbols is now more a question oflearning typing skills than of training the muscles of the hands to use a pen or pencil. Nevertheless, for all practical purposes, hand

writing remains a skill of paramount importance within the larger domain of lan

guage assessment.

A limited variety of types of tasks are commonly used to assess a person's ability to produce written letters and symbols. A few of the more common types are described here:'

1.                  Copying. There is nothing innovative or modern about directing a test-taker to copy letters or words. The test-taker will see something like the following: Handwriting letters, words, and punctuation marks,The test-taker reads: Copy the following words in the spaces given: bit bet bat but Oh? Oh! bin din gin pin HelloI John.

2.                  Listening cloze selection tasks. These tasks combine dictation with a written script that has a relatively frequent deletion ratio (every fourth or ftfth word, perhaps). The test sheet provides a list of missing words from which the test-taker must select. The purpose at this stage is not to test spelling but to give practice in writing. To increase the difficulty, the list ofwords can be deleted, but then spelling might become an obstacle. Probes look like this:

Test-takers hear: Write the missing word in each blank. Below the story is a list of words to choose from.

Have you ever visited San Francisco? It is a very nice city. It is cool in the summer and warm in the winter. I like the cable cars and bridges.

Test-takers see:

Have………ever visited San Francisco? It is…… very nice…... It is……in…..summer and…..in the winter. I…….the cable cars……bridges.

3.                  Picture-cued tasks. Familiar pictures are displayed, and test-takers are told to write the word that the picture represents. Assuming no ambiguity in identifying the picture (cat, hat, chair, table, etc.), no reliance is made on aural comprehension for successful completion of the task.

4.                  Form completion tasks.A variation on pictures is the use of a simple form that asks for name, address, phone number, and other data. Assu~g,of course, that prior classroom instruction has focused on filling out such forms, this task becomes an appropriate assessment ofsimple tasks such as writing one's name and address.

5.                  Converting numbers and abbreviations to words.

This task can serve as a reasonably reliable method to stimulate handwritten English. It lacks authenticity, however, in that people rarely write out such numbers (except in writing checks), and it is more ofa reading.task (recognizing numbers) than a writing task. If you plan to use such a method, be sure to specify exactly what the criterion is, and then proceed with some caution.

Spelling Tasks and Detecting Phoneme-Grapheme Corrrespondences

A number of task types are in popular use to assess the ability to spell words correctly and to process phoneme-grapheme correspondences.

1.      Spelling tests. In a traditional, old-fashioned spelling test, the teacher dictates a simple list ofwords, one word at a time,followed by the word in a sentence,repeated again, with a pause for test-takers to write the word. Scoringemphasizes correct spelling.You can help to control for listening errors by choosing words that the students have encountered before-words that they have spoken or heard in their class.

2.      Picture-cued tasks. Pictures are displayed with the objective of focusing on familiar words whose spelling may be unpredictable. Items are chosen according to the objectives of the assessment, but this format is an opportunity to present some challenging words and word pairs: boot/book, read/reed,bit/bite, etc.

3.      Multiple-choice techniques. Presenting words and phrases in the form of a multiple-choice task risks crossing over into the domain of assessing reading, but if the items have a follow-up writing component, they can serve as formative reinforcement of spelling conventions. They might be more challenging with the addition of homonyms .Here are some examples.

Multiple-choice reading-writing spelling tasks

Test-takers read:

Choose the word with the correct spelling to fit the sentence, then write the word in the space provided.

1. He washed his hands with _______

A. soap .8. sope

C. sop

D. soup

2. I tried to stop the car, but the _______ didn't work.

A. braicks

B. brecks

C. brakes

D. bracks

3. The doorbell rang, but when I went to the door, no one was _______

A. their

B. there

C. they're

D. thair

4. Matching phonetic symbols.If students have become familiar with the phonetic alphabet, they could be shown phonetic symbols and asked to write the correctly spelled word alphabetically. This works best with letters that do not have one-to-one correspondence with the phonetic symbol (e.g., lrel and a).In the sample below, the answers, which ofcourse do not appear on the test sheet, are included in brackets for your reference.

DESIGNING ASSESSMENT TASKS: INTENSIVE (CONTROLLED) WRITING

This next level of writing is what second language teacher training manuals have for decades called controlled writing. It may also be thought of as form-focused writing, grammar writing, or simply guided writing. A good deal of writing at this level is display writing as opposed to real writing: students produce language to display their competence in grammar, vocabulary, or sentence formation, and not necessarily to convey meaning for an authentic purpose. The traditional grammar/vocabulary test has plenty ofdisplay writing in it, since the response mode demonstrates only the test-taker's ability to combine or use words correctly. No new information is passed on from one person to the other.

Dictation and Dicto-Comp

Dictation is simply the rendition in writing of what one hears aurally, so it could be classified as an imitative type of writing, especially since a proportion of the testtaker's performance centers on correct spelling. Also, because the test-taker must listen to stretches of discourse and in the process insert punctuation, dictation of a paragraph or more can arguably be classified as a controlled or intensive form of writing.

A form of controlled writing related to dictation is a dicto-comp.Here, a paragraph is read at normal speed, usually two or three times;then the teacher asks students to rewrite the paragraph from the best oftheir recollection.In one of several variations of the dicto.-Comp technique, the teacher, after reading the passage, distributes a handout with key words from the paragraph, in sequence, as cues for the students. In either case, the dicto-comp is genuinely classified as an intensive, ifnot a responsive, writing task. Test-takers must internalize the content of the passage, remember a few phrases and lexical items as key words, then recreate the story in their own words.

Grammatical Transformation Tasks

Language teachers have also used this technique as an assessment task, ostensibly to measure grammatical competence. Numerous versions of the task are possible:

•                     Change the tenses in a paragraph.

•                     Change full forms of verbs to reduced forms (contractions).

•                     Change statements to yes/no or wh-questions.

•                     Change questions into statements.

•                     Combine two sentences into one using a relative pronoun.

•                     Change direct speech to indirect speech.

•                     Change from active to passive voice.

Picture-Cued Tasks

The main advantage in this teclmique is in detaching the almost ubiquitous reading and  writing connection and offering instead a nonverbal means to stimulate written responses.

1.      Short sentences.

A drawing of some simple action is shown; the test·taker writes a brief sentence.

2.      Picture description.

Test-takers are asked to describe the picture using four of the following prepositions: on, over, under, next to, around. As long as the prepositions are used appropriately, the criterion is considered to be met.

3.      Picture sequence description.

The pictures must be simple and unambiguous because an open-ended task at the selective level would give test-takers too many options. If writing the correct grammatical form of a verb is the only criterion then some test items might include the simple form of the verb.

Vocabulary Assessment Tasks

The major techniques used to assess vocabulary are (a) defing and (b) using a word in a sentence. The latter is the more authentic,but even that task is constrained by a contrived situation in which the test-taker, usually in a matter of seconds, has to come up with an appropriate sentence, which mayor may not indicate that the test-taker "knows" the word.The procedures are creatively linked by means of the target word, its collocations, and its morphological variants. At the responsive and extensive levels, where learners are called upon to create coherent paragraphs, performance obviously becomes more authentic, and lexical choice is one of several possible components of the evaluation of extensive writing.

Ordering Tasks

One task at the sentence level may appeal to those who are fond ofword games and puzzles: ordering (or reordering) a scrambled set of words into a correct sentence. Here is the way the item format appears.

While this somewhat inauthentic task generates writing performance and may be said to tap into grammatical word-ordering rules, it presents a challenge to testtakers whose learning styles do not dispose them to logical-mathematical problem solving.

Short-Answer and Sentence Completion Tasks

Such items range from very simple and predictable to somewhat more elaborate responses. Look at the range of possibilities.

A: Whos that?

B: ……Gina.

ISSUES IN ASSESSING RESPONSIVE AND EXTENSIVE WRITING

Responsive writing creates the opportunity: for test-takers to offer an array of 'possible creative responses within a pedagogic or assessment framework: test-takers are "responding" to a prompt or assignment.Freed from the strict control of intensive writing, learners can exercise a number of options in choosing vocabulary, grammar, and discourse, but with some constraints and conditions.Criteria now· begin to include the discourse and rhetorical conventions of paragraph structure and of connecting two or three such paragraphs in texts oflimited length.The learner is responsible for accomplishing a purpose in writing, for developing a sequence ofconnected ideas, and for empathizing with an audience.

The genres of text that are typically addressed here are

•                     short reports (with structured formats and conventions);

•                     responses to the reading of an article or story;

•                     summaries of articles or stories;

•                     brief narratives or descriptions; and

•                     interpretations of graphs, tables, and charts.

In extensive writing, however, the writer has been given even more freedom to choose: topics, length, style, and perhaps even conventions offormatting are less constrained than in the typical responsive writing exercise. At this stage, all the rules of effective writing come into play, and the second language writer is expected to meet all the standards applied to native language writers.

Both responsive and extensive writing tasks are the subject of some classic, widely debated assessment issues that take on a distinctly different flavor from those at the lower-end production of writing.

1.      Authenticity.

Authenticity is a trait that is given special attention: if testtakers are being asked to perform a task, its face and contentvalidity need to be assured in order to bring out the best in the writer.A good deal of writing performance in academic contexts is constrained by the pedagogical necessities of establishing the basic building blocks ofwriting;we have looked at assessment techniques that address those foundations.But once those fundamentals are in place, the would-be writer is ready to fly out ofthe protective nest ofthe writing classroom and assume his or her ownvoice. Offering that freedom to learners requires the setting ofauthentic real-world contexts in which to write. The teacher becomes less of an instructor and more of a coach or facilitator.Assessment therefore is typically formative, not summative, and positive washback is more important than practicality and reliability.

2.      Scoring.

With so many options available to a learner, each evaluation by a test administrator needs to be fmely attuned not just to how the writer strings words together (the form) but also to what the writer is saymg (the/unction of the text).The quality of writing (its impact and effectiveness) becomes as important, If not more important,than all the nuts and bolts that hold it together.

3.                  Time. Yet another assessment issue surrounds the unique nature of writing: it is the only skill in which the language producer is not necessarily constrained by time, which implies the freedom to process multiple drafts before the text becomes a fmished product.

4.                   

DESIGNING ASSESSMENT TASKSt RESPONSIVE AND EXTENSIVE WRITING

·         Paraphrasing

The initial step in teaching paraphrasing is to ensure that learners understand the importance of paraphrasing: to say something in one's own words, to avoid plagiarizing, to offer some variety in expression. With those possible motivations and purposes in mind, the test designer needs to elicit a paraphrase of a sentence or paragraph, usually not more.

Scoring of the test-taker's response is a judgment call in which the criterion of conveying the same or similar message is primary, with secondary evaluations of discourse, grammar, and vocabulary. Other components of analytic or holistic scales might be considered as criteria for an evaluation. Paraphrasing is more often a part of informal and formative assessment than of formal, summative assessment, and therefore student responses should be viewed as opportunities for teachers ana students to gain positive washback on die art of paraphrasing.

Guided Question and Answer

Essentially serve as an outline of the emergent written text. In the writing of a narrative that the teacher has already covered in a class discussion, the following kinds of questions might be posed to stimulate a sequence of sentences.ex:

1.      Where did this story take place? [setting]

2.      Who were the people in the story? [characters]

3.      What happened first? and then? and then? [sequence of events]

4.      Why did do [reasons, causes]

5.      What did think about ________ [opinion]

6.      What happened at the end? [climax]

7.      What is the moral of this story? [evaluation]

Guided writing texts, which may be as long as two or three paragraphs, may be scored on either an analytic or a holistic scale (discussed below). Guided writing prompts like these are less likely to appear on a formal test and more likely to serve as a way to prompt initial drafts of writing. Thisft.rst draft can then undergo the editing and revising stages discussed in the next section of thIs chapter.

A variation on using guided questions is to prompt the test-taker to write from an outline. The outline may be self-created from earlier reading and/or discussion, or, 'Yhich is less desirable, be provided by the teacher or test administrator.

Paragraph Construction Tasks

The participation of reading performance is inevitable in writing effective paragraphs. To a great extent, writing is the art of emulating what one reads.You read an effective paragraph; you analyze the ingredients of its success; you emulate it. Assessment of paragraph development takes on a number of different forms:

1.    Topic sentence writing. There is no cardinal rule that says every paragraph must have a topic sentence,but the stating of a topic through the lead sentence (or a subsequent one) has remained as a tried-and·true technique for teaching the concept of a paragraph. Assessment there consists of

•                     specifying the writing of a topic sentence,

•                     scoring points for its presence or absence, and

•                     scoring and/or commenting on its effectiveness in stating the topiC.

2.    Topic development within a paragraph. Because paragraphs are intended to provide a reader with "clusters" of meaningful, connected thoughts or ideas, another stage of assessment is development of an idea within a paragraph. Four criteria are commonly applied to assess the quality of a paragraph:

•                     the clarity of expression of ideas

•                     the-logic-ofthe sequence and connections

•                     the cohesiveness or unity of the paragraph

•                     the overall effectiveness or impact of the paragraph as a whole

3.     Development of main and supportihg ideas across paragraphs. As writers string two or more paragraphs together in a longer text (and as we move up the continuum from responsive to extensive writing), the writer attempts to articulate a thesis or main idea with clearly stated supporting ideas.These elements can be considered in evaluating multi elements esay.

•                     addressing the topic,main idea,or principal purpose

•                     organizing and developing supporting ideas

•                     using appropriate details to undergird supporting ideas

•                     showing facility and fluency in the use of language

•                     demonstrating syntactic variety

•                      

Strategic Options

1.      Attending to genre.

The genres of writing that were listed at the beginning of this chapter provide some sense of the many varieties of text that may be produced by a second language learner in a writing curriculum. Another way oflooking at the strategic options open to a writer is the extent to which both the constraints and the opportunities ofthe genre are exploited. Assessment of any writing necessitates attention to the conventions of the genre in question.

Reports (Lab Reports, Project Summaries, Article/Book Reports, etc.)

•                     conform to a conventional format (for this case, field)

•                     convey the purpose, goal, or main idea • organize details logically and sequentially

•                     state conclusions or fmdings

•                     use appropriate vocabulary and jargon for the specific case

Summaries of Readings/Lectures/Videos

•                     effectively capture the main and supporting ideas of the original

•                     maintain objectivity in reporting

•                     use writer's own words for the most

•                     omit irrelevant or marginal details

•                     conform to an expected length

Responses to Readings/Lectures/Videos

•                     accurately reflect the message or meaning of the original

•                     appropriately select supporting ideas to respond to

•                     express the writer's own opinion

•                     defend or support that opinion effectively

•                     conform to an expected length

Narration, Description, Persuasion!Argument, and Exposition

•                     follow expected conventions for each type of writing

•                     convey purpose, goal, or nlain idea

•                     use effective writing strategies

•                     demonstrate syntactic variety and rhetorical fluency

Interpreting Statistical, Graphic, or Tabular Data

•                     provides an effective global, overall description of the data • organizes the details in clear, logical language

•                     accurately conveys details

•                     appropriately articulates relationships among elements of the data .

•                     conveys specialized or complex data comprehensibly to a lay reader

•                     interprets beyond the d4ta when appropriate

Library Research Paper

•                     states purpose or goal of the research

•                     includes appropriate citations and references in correct format

•                     accurately represents others' research findings

•                     injects writer's own-interpretation, when appropriate, and justifies it

•                     includes suggestions for further research

•                     sums up fmdings in a conclusion

TEST OF WRITTEN ENGLISH (TWE)

One of a number of internationally available standardized tests of writing ability is the Test of Written English). Established in 1986, it has gained a reputation as a well-respected measure of written English, and a number of research articles support its validity.In 1998, a computer-delivered version of the TWE was incorporated into the standard computer-based TOEFL and simply labeled as the"writing" section of the TOEFL. The TWE is still offered as a separate test especially where only the paperbased TOEFL is available. Correlations between the TWE and TOEFL were consistently high.

The TWE is in the category of a timed impromptu test in that test..takers are under a 30-minute time limit and are not able to prepare ahead of time for the topic that will appear. Topics are prepared by a panel of experts following specifications for topics that represent commonly used discourse and thought patterns at the university level.

Test preparation manuals such as Deborah Phillips's Longman Introductory Course for the TOEFL Test (2001) advise TWE test-takers to follow six steps to maximize success on the test:

1.      Carefully identify the topic.

2.      Plan your supporting ideas.

3.      In the introductory paragraph, restate the topic and state the organizational plan of the essay.

4.      Write effective supporting paragraphs (show transitions, include a topic sentence, specify details). 

5.      Restate your pOSition and summarize in the concluding paragraph.

6.      Edit sentence structure and rhetorical expression.

SCORING METHODS FOR RESPONSIVE AND EXTENSIVE WRITING

1.      Holistic Scoring

2.      Primary Trear Scoring

3.      Analytic Scoring

BEYOND SCORING: RESPONDING TO EXTENSIVE WRITING

Formal testing carries with it the burden of designing a practical and reliable instrument that assesses its intended criterion accurately. To accomplish that mission, designers of writing t~sts are charged with the task of providing as "objective" a scoring procedure as possible, and one that in many cases can be easily interpreted by agents beyond the learner. HolistiC, primary trait, and analytic scoring all satisfy those ends.Yet beyond mathematically calculated scores lies a rich domain of assessment in which a developing writer is coached from stage to stage in a process of building a storehouse of writing skills. Here in the classroom, in the tutored relationship of teacher and student, and in the community ,of peer learners, most of the hard work of assessing writing is carried out. Such assessment is informal, formative, and replete with washback.

Most writing specialists agree that the best way to teach writing is a hands-on approach that stimulates student output and then generates a series ofse1f-assessments, peer editing and reviSion, and teacher response and conferencing (Raimes, 1991, 1998; Reid, 1993; Seow, 2002). It is not an approach that relies on a massive dose of lecturing about good writing, nor on memorizing a bunch ofrules about rhetorical organization, nor on sending students home with an assignment to turn in a paper the next day:. People become good writers by writing and seeking the facilitative input of others to refme their skills.

Assessment takes on a crucial role in such an approach. Learning how to become a good writer places the student in an almost constant stage of assessment. To give the student the maximum benefit of assessment, it is important to consider (a) earlier stages (fronl freewriting to the first draft or two) and (b) later stages (revising and finalizing) of producing a written text. A further factor in assessing writing is the involvement of self, peers, and teacher at appropriate steps in the process. (For further guidelines on the process of teaching writing, see TBP, Chapter 19.)

Assessing Initial Stages of the Process of Composing

Following are some guidelines for assessing the initial stages (the frrst draft or two) of a written composition. These guidelines are generic for self, peer,. and teacher , responding. Each assessor will need to modify the list according to the level of the

learner, the context, and the purpose in responding.

Assessment of initial stages in composing

1.      Focus your efforts primarily on meaning, main idea, and organization.

2.      Comment on the introductory paragraph.

3.      Make general comments about the clarity of the main idea and logic or appropriateness of the organ ization.

4.      As a rule of thumb, ignore minor (local) grammatical and lexical errors.

5.      Indicate what appear to be major (global) errors (e.g., by underlining the text in question), but allow the writer to make corrections.

6.      Do not rewrite··ques-titmable, ungrammatical,or·awkward sentences; rather, probe with a question about meaning.

7.      Comment on features that appear to be irrelevant to the topic.

The teacher-assessor's role is; as a guide, a facilitator, and an ally; therefore, assessment at this stage of writing heeds to be as positive as possible to encourage the writer. An early focus on overall structur'e and meaning will enable writers to

. clarify their purpose and plan and will set a framework for the writers' later refmement of the lexical and grammatical issues.

Assessing Later Stages of the Process of Composing

Once the writer has determined and clarified his or her purpose and plan, and has completed at least one or perhaps two drafts, the focus shifts toward "fme tuning" the expression with a view toward a final reVision. Editing and responding assume an appropriately different character now, with these guidelines: Comment on the specific clarity and strength of all main ideas and supporti ng ideas, and on argument and logic.

1.      Call attention to minor ("Iocal") grammatical and mechanical (spelling, punctuation) errors, but direct the writer to self-correct.

2.      Comment on any further word choices and expressions that may not be awkward but are not as clear or direct as they could be.

3.      Point out any problems with cohesive devices within and across paragraphs.

4.      If appropriate, comment on documentation, citation of sources, evidence, and other support.

5.      Comment on the adequacy and strength of the conclusion.

Two significant overarching guidelines for designing an effective assessment procedure:

1.      It is virtually impossible to isolate anyone of the four skills without the involvement of at least one other mode of performance. Don't underestimate the power of the integration ofskills in assessments designed to target a single skill area.

2.                  The variety ofassessment techniques and  item types and tasks is virtually infmite in that there is always some possibility for creating a unique variation. Explore those alternatives, but with some caution lest your overzealous urge to be innovative distract you from a central focus.