Recently, I saw this question in Quora.
What is stopping companies from measuring learning today?
And here is what I think.
Most companies do measure Learning.
But among those almost of them measure the "Efficiency" - ie., how many people learnt, cost, no of hrs of learning, etc.
They also measure the Learner Reaction or Satisfaction which basically Level 1 of the Kirkpatrick's 4 Levels of Evaluating Training Evaluations (Refer: Evaluating Training Programs: Kirkpatrick's 4 Levels)
There are also quite a few organizations, that measure the "Effectiveness" of Learning ie., Knowledge Gain, Retention, Learning Transfer on the Job, etc.
But there are only very organization, that measure "Learning Outcomes & Business Impact". While measuring the Learning Outcomes and reporting on the Business Impact of Learning is difficult...it is still doable. If I have to state it in very simplistic terms...
a) Agree on expected Outcomes with the Business Stakeholder
b) Identify the Observable Behaviors that will create this Outcome and ensure Stakeholders are in agreement on that
c) Determine the Metrics & Measures to track that Behavior - and make the Stakeholder to be observer of that Behavior Change
d) Report on the Behavior Change and correlate it with the Business Performance/Outcomes
e) Compare the Actual Outcomes with the Expected Outcomes
Check out this interaction in Quora at this link http://t.co/9dL7CPrUwD
What is stopping companies from measuring learning today?
And here is what I think.
Most companies do measure Learning.
But among those almost of them measure the "Efficiency" - ie., how many people learnt, cost, no of hrs of learning, etc.
They also measure the Learner Reaction or Satisfaction which basically Level 1 of the Kirkpatrick's 4 Levels of Evaluating Training Evaluations (Refer: Evaluating Training Programs: Kirkpatrick's 4 Levels)
There are also quite a few organizations, that measure the "Effectiveness" of Learning ie., Knowledge Gain, Retention, Learning Transfer on the Job, etc.
But there are only very organization, that measure "Learning Outcomes & Business Impact". While measuring the Learning Outcomes and reporting on the Business Impact of Learning is difficult...it is still doable. If I have to state it in very simplistic terms...
a) Agree on expected Outcomes with the Business Stakeholder
b) Identify the Observable Behaviors that will create this Outcome and ensure Stakeholders are in agreement on that
c) Determine the Metrics & Measures to track that Behavior - and make the Stakeholder to be observer of that Behavior Change
d) Report on the Behavior Change and correlate it with the Business Performance/Outcomes
e) Compare the Actual Outcomes with the Expected Outcomes
Check out this interaction in Quora at this link http://t.co/9dL7CPrUwD
Srini,
ReplyDeleteThank you so much for writing such an insightful post and highlighting this key issue in learning today. I think the steps you've outlined here (from A to E) provide a great template for organizations to use when looking to measure level 4 of Kirkpatrick's model.
You mentioned in your post that most companies aren't currently measuring level 4 because it's difficult. Based on this, I'm curious about two things:
1. Why do you think it is so difficult?
2. Do you think the appropriate technology can help change this?
Thanks again for this great post.
- Jon
Hi Jon,
DeleteThanks for your comments.
Regarding your question "Why do you think it is so difficult"....it is because, it requires all the steps A to E that I have mentioned in my post , has to be done. And it demands certain skills (like Performance Consulting, Data Analysis, etc) among the L&D professionals and certain technologies. Again as I said, thought it is difficult, it is still doable.
Talking about technologies, it leads me into your 2nd question "Do you think the appropriate technology can help change this?".
And my thought is...yes technology can help for eg.,
-technologies that helps the learners & their managers to set goals (regarding application of Learning on the Job) & track the progress on those goals
-electronic performance support system implementations
-technologies that enable short bursts of micro learning modules to be delivered to the learners
-Gamified learning (which would require technical implementations)
etc.