Test Construction and Assessment

Self-Assessment Question:

Quick Take:

In order to effectively assess student learning, teachers need to know how to write good multiple choice and essay questions, of course. But good assessment involves using a perspective and two important tasks before one gets to the writing of actual test questions.

View of Assessment: One helpful perspective on assessment is to view it as a mechanism for determining "student readiness to do something in the future." You may be trying to determine whether students are ready for: the next part of a course, the next course in a multi-course sequence, a licensing exam, graduate school, a particular kind of work, or for performing a certain activity in their personal lives (e.g., interpreting a novel they read on their own). Using this view will help students better understand both the reason(s) for a particular form of assessment and the value of the learning itself.

Developing a List of Appropriate Assessment Mechanisms: If one accepts the view of assessment as "measuring student readiness to do something," then there is a 2-step procedure for selecting the right mechanisms for assessing student learning in a course. First, the teacher needs to make a list of the major learning goals (say, 3-5) for a course, i.e., what is it she/he wants students to be able to do in the future? This might include, for example, (a) remembering key names, dates, formulates, etc., (b) using ideas, principles, and/or relationships from the course to think about (and perhaps solve) certain kinds of problems, (c) get excited about [the subject], and (d) learning how to keep on learning about the subject after the course is over.

Second, for each kind of learning, the teacher should ask, for each learning goal: "What would students have to do to convince me that they had met that learning goal?" Different kinds of learning goals will have different kinds of appropriate assessment mechanisms. The most appropriate mechanism might be: doing a project, solving problems, writing an essay, designing -____, showing a certain kind of performance under actual or simulated conditions, etc.

Going through this two-step procedure, imaginatively and thoughtfully, will help insure that you get the information that validly tells you whether students are learning what you want them to learn.

Planning a Conventional Test: Although the list of possible assessment mechanisms is long, conventional paper/pencil tests will frequently be the mechanism of choice, especially for "knowing" and "thinking" kinds of learning goals. But, before actually writing individual test questions, a teacher should plan or create an outline for the test.

To plan a test, say over the last four weeks of a course, the teacher should create a two-dimensional table, that looks something like the sample illustrated below: (Note: this example is only partially filled out.)

Types of Learning

	Recall Knowledge	Analysis	Problem Solving	Evaluating	Percentage Weight
Topic 1	12 %				40 %
Topic 2			6 %		30 %
Topic 3					30 %
Percentage Weight	30 %	30 %	12 %	20 %	100 %

Decide (a) how much of the test should be focused on each topic [the right hand column], and then (b) decide how much of the test you want to use to assess different kinds of learning [the bottom row]. The interaction of these two percentages in each cell indicate what percentage of the total test will be on, for example, recall knowledge of Topic 1 [12%] and problem solving related to Topic 2 [6%]. (Note: Some cells can be blank.) This procedure will insure that you have a test that reflects a proper weighting of different topics covered and proper attention to more than memorization.

Writing Specific Test Questions: After you have a good plan for the test, you are now ready to write specific test questions, and are in a position to decide whether a particular kind of learning on a particular topic best calls for a multiple-choice question or an essay question, and how much weight should be given to a particular question or set of questions.

The references below contain many specific suggestions for writing good test questions. But two key suggestions are provided here.

Multiple Choice (MC) Questions: Some MC questions need to focus on "recall" knowledge, but MC questions can be written that require a student to think as well; good tests will have several of these. The general principle for doing this is (a) to provide students with information or material in the stem, and then (b) write a question that calls for students to analyze, interpret, or make choices about that material. Make your answer-choices as straightforward, unambiguous, and plausible as you can.

Essay Questions: The most common problem with essay questions is that they are too wide open and do not indicate what the teacher is looking for, really. A good essay question will (a) indicate what the scope of the answer should be, and (b) indicate what criteria will be used to assess the quality of the answer.

References:

1. Assessment of Student Achievement by Norman E. Gronlund. Boston: Allyn & Bacon, 1997. The standard reference book on designing achievement tests. Has excellent advice on writing individual test questions as well as on planning a test. Describes "Interpretive Questions" as a device for using a multiple-choice format to test "thinking" questions.

2. Effective Grading: A Tool for Learning and Assessment by Barbara E. Walvoord and Virginia Johnson Anderson. San Francisco: Jossey-Bass, 1998. "Grading," as used here, refers to: identifying the most valuable kinds of learning in a course, constructing exams and assignments that test that learning, setting standards and criteria, guiding student learning, and implementing changes in teaching based on information from grading procedures. Excellent ideas in Chapter 5 on "Establishing criteria and standards for grading."

3. Educative Assessment: Designing Assessments to Inform and Improve Student Performance by Grant Wiggins. San Francisco: Jossey-Bass, 1998. Offers a vision of a better kind of assessment: "Educative Assessment", i.e., assessment that is anchored in authentic performance tasks and that provides students and teacher with feedback on how to improve their performance. Presents powerful ideas on "Backward Design" [starting with the question of what you want students to be able to do at the end of the course, and then designing learning activities that will get them there], and "Authentic Performance Tasks" - performance tasks that are real.