To determine how well student learning outcomes are being achieved, many programs choose to include exams in their assessment plan.
Using an exam for program assessment is different than using an exam for grading student work. For the purpose of assessment, items are aligned with the components of the outcome they assess, and then student responses to each item are aggregated to determine the achievement of each component of the outcome.
If you are considering using exams as part of your program’s assessment plan, the following steps can serve as a guide.
1) Determine where in the curriculum the outcome is addressed and at what level you want to assess it.
A curriculum or outcome map is helpful in accomplishing this task.
To determine if improvement in student learning has occurred across the program, look for 1) courses where the outcome is introduced, and 2) for courses later in the program that emphasize the outcome.
To determine if students have achieved the outcome by the time they are about to complete the program, look at senior-level courses.
2) Design an exam for the course(s) you have selected and the outcome(s) you want to focus on or add items to an existing exam.
For help with creating items, see Planning Your Exam, Tips for Writing an Effective Exam, and Types of Exam Items.
To keep track of how your exam items align with your outcome components – and to make sure that every component of the outcome you want to assess is adequately covered – create an item-to-outcome alignment table. List the components you would like to assess in one column, and then in another column to the right, list all of the items that align with each component. For example:
| Item-to-Outcome Alignment | |
|---|---|
| Outcome component | Test items that align |
| Component A | 1, 2, 4 |
| Component B | 3, 5 |
3) Administer and score the exam.
Instead of just marking answers as right or wrong when you score exams, use a spreadsheet, table, or list to record all answers given by students. This will later help you to better understand student achievement of the outcome.
4) Calculate how many students got each item correct, then refer to your Item-to-Outcome Alignment table (see Step 2 above) to link these scores to components of your outcomes.
Later, conduct an item analysis, that is, take account of all of the items that were commonly missed and reflect on whether the problem might be the wording of an item or students’ understanding.
If it looks like students don’t know the material, prepare to discuss with colleagues what kind of changes (e.g., to pedagogy or the curriculum) might improve student achievement (see Step 6).
If it seems like a problem with the item, make plans to edit the item before you administer the test again.
5) Present the data in a way that is user-friendly for your program’s faculty.
For each outcome, share information about how students did on the items that align. There are multiple user-friendly ways to present this information. For example, you can show the number or percentage of correct responses for each item, or the average score on each item. These may be shown in a table, a graph, or a paragraph about what you learned. Choose the approach that makes the most sense for you and your discipline.
6) Work with colleagues to determine what the results mean, and look for ways to improve areas of concern.
To determine whether or not students are meeting program expectations for achievement, establish a criterion or standard of success. For example, you might say that you expect 75% of your students to correctly answer each item. In part, these criteria will depend on the nature and difficulty of particular items.
If students meet the criteria for all aspects of the learning outcomes being assessed, then plan to reexamine them at a later time to determine if this is still the case.
If students are not meeting the criteria for certain aspects of your outcomes, then decide on changes to improve student learning in your program (e.g., to the curriculum, pedagogy, assignments). You will also want to reassess the outcomes at a later time to determine if these changes had a positive impact on student learning.