The power of evaluation

Few would argue against the need to evaluate the value and cost-effectiveness of training. But how best to go about it? What questions are appropriate and what methodology would best provide the answers?

Kirkpatrick⁽¹⁾ first defined four levels of training evaluation back in 1954. He proposed that evaluation should start with defining mission critical outcomes (level 4 results) and the leading indicators which demonstrate that an organisation is on track to achieve those results. The leading indicators help to define a set of critical behaviours (level 3) employed by training participants to bring about the desired results. In turn comes learning (level 2) which supports the enactment of the critical behaviours and finally participant reaction to a training intervention (level 1) - the questions relating to content relevance and customer satisfaction that are captured, almost as an afterthought, on so many evaluation forms.

Transferring learning into the workplace is where the real value of training programmes are effected. Reinforcing knowledge and skills learnt during training, with support and accountability in the workplace, results in 85% of the learning being applied on-the-job, whereas relying on training interventions alone results in a success rate of just 15% ⁽²⁾. But bridging the gap between learning and workplace behaviour is challenging, which may explain why so few evaluations focus resources on this step.

The degree to which a programme is evaluated should match its importance or cost to the organisation, gathering only the data which is useful and meaningful to stakeholders. An evaluation using blended data, either formative or summative, harvested from multiple sources, including existing business metrics, can be implemented or adjusted at any stage and remain realistic and cost-effective. Observations of participant behaviour, individual or group goal setting and action planning, communities of practice and stories of success can be powerful tools to track progress towards results and to communicate this to stakeholders.

These are our key take-home messages for an effective evaluation:

• Start your evaluation design with your level 4 results clearly in mind and use these to define your leading indicators
• Focus your resources on level 3 behaviours - strong partnerships will be needed to transfer level 2 learning into the workplace
• Use a blended approach - for example, an evaluation form can be designed to capture both level 2 learning and an action plan for level 3 on-the-job application
• Use actionable intelligence and only collect data that is useful or relevant to your stakeholders
• Level 4 results are business critical so tap into metrics that are already available, maximising the use of resources

If you'd like to talk more about evaluation and how it can be applied in your organisation, contact us.

⁽¹⁾ James D and Wendy Kirkpatrick. Kirkpatrick's four levels of training evaluation. ATD Press. 2016

⁽²⁾ Brinkerhoff, R.O. 2006. Telling training's story: Evaluation made simple, credible and effective. SanFrancisco, C.A. Berrett-Koehler