Have you learned anything new today? The odds are that if you are reading this, you have internet access, so you must have come across at least one piece of new information, a video, or a short post by one of your friends.
How much have you learned since you had your morning coffee? That’s a tough question if you are not a student before an exam and know precisely how many pages you have gone through and how much there still is to cover. And even in that very particularly unique circumstance, it would be very difficult to quantify.
And that’s one of the problems corporate L&D faces – accounting for how much employees have learned and, more importantly, how well.
There is an acute need for standards in this field and that’s where the idea to design some Talent Development Reporting principles (TDRp) came from.
Talent development reporting principles
L&D principles were borrowed from accounting, which has four types of measures—revenue, expense, assets, and liabilities—and three standard statements—income or profit & loss, balance sheet, and cash flow. There is also a set of standards called Generally Accepted Accounting Principles (GAAP in the United States and International Financial Reporting Standards elsewhere int the world).
L&D didn’t use to have anything remotely resembling that, so several learning professionals and industry leaders decided to tackle the task. Talent Development Reporting principles were meant to become a simple and easy-to-use framework consisting of three types of standard L&D measures and three L&D standard reports.
The three types or categories of standard measure are effectiveness, efficiency, and outcomes while the three types of reports—Operations, Program, and Summary.
Effectiveness is sought out in all things business
Obviously, this includes learning. In this field, effectiveness has been measured by Kirkpatrick’s four levels and lately by Jack Philips’ fifth one (having to do with the ROI of learning).
- Level 1 measures the participant’s or manager’s satisfaction with or reaction to the learning—this is a first measure of quality.
- Level 2 measures the amount learned or how much of the knowledge or skills have been transferred into action.
- Level 3 measures the real application of that knowledge or change in behavior. If there is none of these, it’s obvious the course has missed its mark (whether it was because of lack of engagement or uninteresting content)
- Level 4 measures impact or results. Since all learning interventions are designed starting from the goals that need to be achieved it’s a very good measure of accomplishment.
- Last, ROI (Level 5) provides a final confirmation of quality (or return on our investment), assuming that all the other four levels have been passed with flying colors.
It's important to measure outcomes
Everything I have already covered is both important and relevant. However, there was one aspect of learning that seemed to be unaccounted for. Many organizations still struggle when it comes to the discussion and measurement of outcomes — the impact or results that learning has on the organization’s objectives or needs.
Of course, there is no way of making a perfect measurement – numbers can speak only for so much of it; the rest of the assessment may very well be subjective. Still, it is crucial for gaining a big picture of where talent development stands and how long it still has to go before achieving its desired goals.
Level 4 of the Kirkpatrick model indeed does some of this job, but there is a stringent need for more. Outcome measures guarantee that L&D is truly aligned with company goals and aware of the type of positive impact it can have on them.
xAPI is a gamechanger
The one thing that organizational learning has continuously sought to change (and measure the degree to which this was a reality) was behavior. Since this does not alter overnight or if it does, it’s hard to see if the effect will be long term, behavioral change has always been the most elusive of all things measured.
With xAPI, a lot of the information that was unavailable thus far becomes a priceless resource. It also allows L&D specialists to look at effectiveness measures, such as learner feedback at various points within the experience (Level 1), their learning throughout the modules (Level 2), their thoughts on applicability and what they believe they will require to be successful (Level 3), and even some measure of foreseen impact (Level 4).
Implementing TDRp is a process that requires quite a lot of resources and the buy-in and support of C-suite executives and stakeholders. It is mostly an iterative process, as the development of software solutions – requirements and solutions arise and evolve through collaboration and feedback between the developers and the end-users.