Main Body
By: Matthew Wilson, Shilpa Sahay and Cheryl Calhoun, edited by Steve Covello
About evaluation
Creating a program of instruction, as scientific as it intends to be, is an exercise in prediction. IDs are skilled at gathering information, making design decisions, and assembling a program of instruction into a coherent form, but there are no guarantees that learners will experience the instruction as expected, nor is there a guarantee that all learners will achieve the Learning Outcomes. Many confounding factors may impede the success of a given program such as unexpected technical problems, poor teaching engagement, or flaws in the design.
To account for these uncertainties, the ADDIE model includes a built-in self-evaluation phase. Whereas, an instructional program intends to assess learners, the Evaluation phase assesses the program itself:
- Were learners adequately prepared prior to the instructional program?
- Have learners achieved the Learning Outcomes, in whole or in part?
- Was the instruction effective for the requirements of the instructional program?
- Are learners able to transfer their learning into the desired contextual setting?
- Do the lesson plans, instructional materials, media, assessments, etc. meet the learning needs?
- Were the methods of implementation effective?
- What was the return on investment (ROI) for the cost of instruction?
The answers to these questions help determine whether the program of instruction needs to be improved (formative evaluation), and whether it has met its goals (a summative evaluation).
In the professional ID field, evaluation is a highly complex discipline with a wide range of approaches to consider for each project. This chapter will describe some of the foundation principles of evaluation which are relevant to students of Communications.
Your goal in this stage is to produce the following sections:
Evaluation goal and audience: A statement that describes the goals and purpose of the evaluation based on stakeholders’ needs and interests; a description of the audiences for whom the evaluation would be of interest.
Formative evaluation plan: A description of how the program of instruction will be reviewed, tested, and revised prior to official launch.
Summative evaluation plan: A matrix of questions and analysis plans designed to produce data from which you can determine whether the program of instruction was successful.
Determining goals and audiences
All evaluation plans must have a clearly defined goal, just as a program of instruction must have a goal. The ID works with stakeholders to establish the goals for an evaluation so that the methods of inquiry are appropriately aligned to them.
For example, stakeholders may be interested in the time it takes to complete a program of instruction given the cost of trainees’ absence from production to attend training. In such a case, return on investment (ROI) would factor into evaluating a training program.
Goals: In determining the goal of a