Assessment of learning 1 report gladys t. ambuyat

Company

LOGO

Sources of Difficulty in the Use of Essay TestSources of Difficulty in the Use of Essay Test

PREPARED AND PRESENTED BY:

GLADYS T. AMBUYATBSED III-ENGLISHOLIVAREZ COLLEGE

Source: Measuring and Evaluating Learning Outcomes: A textbook in Assessment of Learning 1 & 2CARLITO D. GARCIA, Ed. D.

CHAPTER 6

CHAPTER 6

FOUR SOURCES OF DIFFICULTY IN THE USE OF ESSAY TEST:

QUESTION CONSTRUCTION1

2

INSTRUMENT RELIABILITY3

INSTRUMENT VALIDITY4

READER RELIABILITY

QUESTION CONSTRUCTION

The preparation of the essay item is the most important step in the development process. Language usage and word choice are particularly important during the construction process.. The language dimension is very critical not only because it controls the comprehension level of the item for the examinee but also specifies the parameters of the task. A test constructor , you need to narrowly specify, define, and clarify what is that you want form the examinees.

Ex. “Comment on the significance of Darwin’s Origin of Species.”

QUESTION CONSTRUCTION

The question is quite broad considering that there are several ways of responding to it. While the intention of the teacher who wrote this item was to provide opportunity for the students to display their mastery of the material, students could write for an hour and still not discover what their teacher really wants them t do relatively to the aforementioned topic.

An improved version of the same question follows:

“Do you agree with Darwin’s concept of natural selection resulting in the survival of the fittest and the elimination of the unfit? Why or who not?”

READER RELIABILITY

A number of studies had been conducted then and now on the reader reliability of grading free-response test items. Results of these researchers failed to demonstrate consistently satisfactory agreement among essay raters (Payne, 2003). Some of the specific contributory factors in the lack of reader reliability include the following:

> quality of composition and penmanship> item readability > racial of ethnic prejudice on essay scoring> subjectivity of human judgment

INSTRUMENT RELIABILITY

Even if an acceptable level of scoring is attained, there is no guarantee that measurement of desired behaviors will be consistent. There remains the issue of the sampling of objectives or behaviors represented by the test. One way to increase the reliability of an essay is to increase the number of questions and restrict the length of the answers. The more specific and narrowly defined the questions, the less likely are to be ambiguous to the examinee. This procedure should result in more uniform understanding and performance of assigning tasks, and hence in the increased reliability of the instrument and scoring. It also helps ensure better coverage of the domain of objectives.

INSTRUMENT VALIDITY

The number of test questions influences both the validity and reliability of essay questions. As commonly constructed, an essay test contains a small number of items ; thus, the sampling of desired behaviors represented in the table of specifications will be limited, and the test suffers and decreased or lowered content validity. There is another sense in which the validity of an essay may be questioned. Theoretically, the essay test allows the examinees to construct a creative, organized, unique and integrated communication. Nonetheless, these examinees spend most of their time very frequently in simply recalling and organizing information, rather than integrating it. The behavior elicited by the test, then, is not that hoped for by the teacher or dictated by the table of specifications. Again, one way of handling the problem is by increasing the number of items on the test.

GUIDELINES FOR CONSTRUCTING, EVALUATING AND USING ESSAY TEST

1

limit the problem that the question poses so that it will have clear or definite meaning to most students.

2

use simple words which will covey clear meaning to the students.

3

prepare enough questions to sample the material o the subject area broadly, within a reasonable time limit

GUIDELINES FOR CONSTRUCTING, EVALUATING AND USING ESSAY TEST

4

use the essay questions for purposes it best serves, like organization, handling complicated ideas and writing.

5

prepare questions which require considerable thought, but which can be answered in relative few words.

6

determine in advance how much weight will be accorded each of the various elements expected in a complete answer.

GUIDELINES FOR CONSTRUCTING, EVALUATING AND USING ESSAY TEST

7

without knowledge of students’ names, score each question for all students.

8

require all students to answer all questions on the test.

9

write questions about materials immediately relevant to the subject.

GUIDELINES FOR CONSTRUCTING, EVALUATING AND USING ESSAY TEST

10

study past questions to determine how students performed

11

make gross judgments of the relative excellence of answers as a first step in grading.

12

words question as simple as possible in order to make the task clear.

GUIDELINES FOR CONSTRUCTING, EVALUATING AND USING ESSAY TEST

13

do not judge papers on the basis of external factors unless they have been clearly stipulated

14

do not make a generalized estimate on an entire paper’s worth.

15

do not construct a test consisting of only one question.

Company

LOGO Source: Measuring and Evaluating Learning Outcomes: A textbook in Assessment of Learning 1 & 2CARLITO D. GARCIA, Ed. D.