Training evaluation tools are used to verify that training meets its objectives. This post helps you manage training evaluation for any training initiative. It showcases instruments used to evaluate training – the tools of training evaluation..
Training evaluation tools – Background.
Don Kirkpatrick’s four-level training evaluation framework is a time-tested cornerstone for training evaluation. This framework includes the following four levels:
Reaction. An assessment of trainee impressions, and level of satisfaction with the training. This provides an opportunity for trainees to provide feedback to trainers. Items checked include subject matter relevance, trainer capability, appropriateness of the training venue, and convenience for trainees to participate in the program.
Learning. Verification that trainees absorb and understand the trained subject matter.
Behavior. Determination that the training inspires desired on-the-job behaviors.
Results. Appraisal of training’s impact on the organization. This may be reflected in profitability, reduction in defects, impact on injury rates, etc.
Training evaluation tools – When you can quantify results, you can measure them.
Training can be evaluated using a variety of tools. They are designed to add structure, and a means to generate quantitative results, to what may seem to be abstract training aspects.
evaluation tool – the evaluation form
Evaluating trainee reaction typically uses an evaluation form (sometimes pejoratively referred to as a ‘smile sheet’). Responses to questions are typically cast as a range, say “on a scale of 1 to 10, how satisfied were you with the trainer’s communication capabilities?”. Higher scores generally indicate a higher level of satisfaction with the training.
evaluation tool – the learning exam
A learning exam, can be administered as a pencil-and-paper form, recorded online, or as the basis for a trainee interview. The exam provides a way to develop a quantitative score of how well learning was accomplished.
evaluation tool – the behavior observation form
Behavior can be directly assessed by observing on-the-job behaviors or indirectly through gathering evidence that the desired behaviors are being practiced. Are bolts tightened to the desired foot pounds of torque? You can watch trainees actually tighten the bolts or you can occasionally check the bolts to see if they were installed using the proper torque. Results gathered either way can be recorded on a behavior observation form.
evaluation tool – snapshots of pre-training and post-training conditions
Results are checked by comparing snapshots of pre-training conditions with snapshots of post-training conditions. Have injury rates declined since the training was delivered? Are customer complaints about quality occurring at a lower frequency now that the training is completed. Has profitability been impacted by the training program?
Training evaluation tools – So much work to include evaluation with every training program.
When you adopt a policy that every training program will be evaluated, you establish a new litmus test for training. A question will be raised each time a training program is developed – “Is this training worth the effort it will take to evaluate its success?” When the answer is “No”, it may follow that the training is not important enough to implement. While evaluation may add effort to training delivery, its resource requirements may inspire reduction in the number of training programs actually delivered.
When you deliver training to large communities, you may choose to assess training impact on a subset of the trained population (a sample). This can make evaluation manageable. Results must be qualified with the point that they reflect the sample, not the entire trainee population.
The Kirkpatrick framework proposes that training is evaluated at all four levels. It proposes that success at the “reaction” level creates good conditions for success at “learning” level. Success at “learning” level creates good conditions for changes in “behavior”. That successful “behavior” changes are a basis for achieving desired “results”. It’s important to see the connections among these evaluation levels when you design the training.
Stephen Covey admonishes, “Begin with the end in mind.”.
Start with “Results”. What goals are set for the training? Do you want to reduce injuries? Improve customer satisfaction? Improve product quality? What before and after snapshots will you take to build evidence evidence that training has met its goals?
After results are identified, you can ask “What on-the-job behaviors must be practiced to achieve the results?”.
Once the important behaviors are identified you and your team can determine what learning must take place to inspire the required behaviors.
When the learning requirements are defined, you can complete your training plan by identifying trainees for whom the training will be relevant, setting up an appropriate training venue, and assembling course materials and support resources.
Training evaluation tools – What can be done to streamline evaluation for a time-pinched training team?
All four training evaluation levels must be integrated into training program design. At Twin Lakes Consulting, we propose that evaluation efforts ought to be pared down to match available resources – people, time, and money.
We believe training’s primary goal is to modify behavior. (If on-the-job behavior is not expected to change, why deliver the training?) As such, when you wish to streamline training evaluation, take steps to verify that essential behaviors are being practiced. When a sound connection between behaviors and results is made up-front, the desired behaviors ought to deliver the desired results.
What evaluation tool may reasonably be considered most
important for training evaluation? (drum roll please)
the Behavior Observation Form
Regard the term “form” as a metaphor. While sometimes a pencil-and-paper form, it may just as likely be an online form.
When a training team designs a behavior observation form, the related training program is improved. The observed behaviors, once articulated on a data collection form, create a recipe for effective, impactful, training. “How will the wrench be used? ” “How will attendance forms be completed?” “How should the surface be polished?”
As suggested earlier, behavior observation may be performed through direct observation or perhaps indirectly through indicators gathered through other channels. “Are forms filled out properly in accordance with how employees are trained to fill them out?” You can watch employees fill out the forms or you can check the forms to see if they are consistently filled out properly. During training, you can require a performance demonstration. Have trainees demonstrate that they know how to perform tasks in accordance with how they have been trained.
You may choose to evaluate behaviors for a subset of the trained population (a sample). Make sure your samples draw from appropriate demographics – gender? shift? time with the company? Always qualify your results by stating the sample to which the results apply.
Training evaluation tools – Build a body of objective evidence that your training produces desired behaviors.
Record observations on the behavior observation form as behaviors are formally observed – either during training or on the job. Use the form to score the results. This is your tangible, objective, evidence that important behaviors have been observed. Keep these forms as a permanent record.
It may not be possible to directly observe trained behaviors. For example, consider training for disaster response. You will not be able to observe the behaviors until the disaster occurs. Use learning exams that focus on understanding of required behaviors or use variants of behavior observation forms that can be completed during trainee interviews about essential behaviors.
Satisfactory results from evaluating sample groups may reasonably indicate training success, however, unsatisfactory results ought to trigger a prompt study of a larger, if not the complete, trained population. This is especially important when the trained subject matter relates to safety or organization prosperity.