How do I apply assessment theory to assessment practice
Assessment is often focused on determining knowledge. However, there is more to being a competent health professional or scientist than having sound theoretical knowledge. Students and trainees must also demonstrate that they can apply their knowledge, ultimately being able to perform appropriately without supervision in the workplace. It is ability at this final stage that is most difficult to measure. Assessments that primarily test knowledge (e.g. examinations) are of limited value in predicting what a graduate will do as an independent practitioner. Students will also have to work with patients in a variety of settings. It may, therefore, be appropriate to teach students about appropriate attitudes and to assess students in this area.
Students may, therefore, be assessed about: what they know (cognitive domain); skills they have learned (psychomotor domain) and attitudes that they have been taught (affective domain). It is important to assess students at the appropriate level. For example, if the learning outcomes for a course specify that students will be able to recall basic facts then the assessment must test their ability to recall basic facts. As a second example, if the learning outcomes for a course specify that students will be able to expertly perform a clinical skill, then the assessment must test expert ability.
Knowledge of the different levels of learning in the three domains (cognitive, psychomotor and affective) can help to ensure that assessment(s) test students at the appropriate level i.e. assessment fits with levels of learning specified in the learning outcomes.
The cognitive domain has to do with knowledge and understanding. Students can evidence different levels of learning/achievement in this domain. Remembering (factual recall) is considered to the lowest level of learning. The ability to create new knowledge is considered to be the highest level of learning in this domain.
Much of the assessment of clinically based students or trainees occurs in the clinical setting – workplace based assessment – with judgements based on observation of performance in an effort to determine what a student can do. Dave’s taxonomy for the psychomotor domain can be useful in determining the level of learning to be evidenced for psychomotor skills.
Miller’s Triangle is another useful concept to determine the type of assessments required in the assessment of competence. The triangle represents the different levels of learning that might be demonstrated by students. Notice, however, that the triangle does not specify a range of abilities within “knowing how”. This is where the levels of learning in the psychomotor domain are useful.
Finally, we might engage students in instruction to change their attitudes. Engaging in this sort of instruction will raise a lot of questions about the sorts of attitudes that we want our students to have and about ethical considerations when teaching to change an attitude. We are not going to deal with those issues here. We are just suggesting that if the aim of instruction is attitudinal change then there needs to be a way to assess the change.
Assessment methods include:
- Short-answer questions;
- Multiple choice questions;
- Multiple-choice (single best answer and extended matching);
- Self-assessment peer assessment;
- Learning portfolios;
- Case studies and projects;
- Structured oral vivas;
- Structured objective practical assessments e.g. OSCE;
- Direct observation of professional, technical or clinical practice;
- Case/record review.
Different assessment methods are appropriate to test different attributes. Multiple Choice Questions can test broadly across the curriculum, and are useful for assessing factual recall, and, potentially, problem solving. Although difficult to write, they have the advantage of high reliability and ease of marking. However, they can cue students to the correct response, and for that reason, other forms of assessment such as Short Answer Questions, or essays, may be preferred. These written assessments are assessing at the “Knows” or “Knows How” level of Miller’s Triangle.
Direct observation of practice should form the basis of judgements of workplace performance. A range of structured workplace assessment tools have been developed in an attempt to make this form of assessment reliable. Widely used methods include the mini clinical evaluation exercise (mini-CEX), direct observation of procedural skills (DOPS), objective stuctured assessment of technical skills (OSATS), and multi-source feedback.
Before choosing methods, ask yourself the following questions:
- What is the purpose of the assessment?
- What form of assessment will align with the intended learning outcomes?
- How will feedback on the assessment be provided to students?