Simplifying Measures and Reports
Where GAAP has four categories of measures, TDRP has three: efficiency, effectiveness, and outcome. Efficiency measures are about quantity, and examples include number of participants, number of courses, and cost. Effectiveness measures are about quality and include four of the familiar five levels from Kirkpatrick and Phillips (level 1: participant reaction, level 2: learning, level 3: application, and level 5: ROI). In TDRP, level 4 (results or impact) is elevated to a category of its own called outcome because this is what CEOs most want to see and are unlikely to get from their L&D department.
The 100+ L&D measures are divided into these three categories or buckets, which simplifies discussion as well as guidance. TDRP recommends a balance of efficiency and effectiveness measures at both the program (a single course or multiple courses designed with similar learning objectives) and department levels because both quantity (for example, number of courses) and quality (for example, levels 1 and 3) are important.
For programs, TDRP recommends starting with the number of participants, completion rate, completion date, and cost for efficiency measures. For effectiveness measures, TDRP recommends starting with levels 1, 2, and 3. For important programs, level 5: ROI is also recommended. TDRP recommends a level 4 outcome measure for all important programs, especially those where specific, measurable results can be identified (for example, sales or safety training).
At the department level, TDRP also recommends a balanced set of efficiency and effectiveness measures. These will include aggregated, efficiency program measures (for example, total number of participants and percentage of courses completed on time) as well as measures that only make sense at the aggregate level (for example, mix of courses by type and percentage of courses utilized). Recommended aggregated effectiveness program measures include average level 1, average level 2, and average level 3 across all courses, as well as a listing of ROIs for any courses taken to that level. TDRP also recommends listing outcome measures for key courses.
Sharing Data with Users
With measures selected, the next question is how to share the data with users. Breaking new ground, TDRP identifies five types of reports and links report selection back to the reason for measuring.
If users want basic questions answered (for example, how many participants have taken ten courses by month), TDRP recommends a scorecard (excel spreadsheet format) or dashboard (report with visual elements like a pie chart or bar graph) to inform them.
If users have looked at historical results and wish to monitor the data to ensure that performance is at least as good as it has been in the past (the threshold), TDRP recommends scorecards or dashboards with thresholds.
If the reason for measurement is to evaluate a program’s effectiveness, TDRP recommends the results be shared in a program evaluation report, which is designed to share the purpose, context, plans, results, and lessons learned for the program. For this purpose, the program evaluation report is much better than a scorecard or dashboard. If a statistical analysis has been performed to test a hypothesis (for example, more training leads to higher employee engagement and retention), then a custom analysis report format is recommended, which is better suited to hypothesis testing.
If the reason for measuring is to manage a program to a successful conclusion and deliver the promised results, then a special format called a management report is recommended. This report is designed to answer two key questions every month about the program’s progress: are we on plan year to date and, given what we now know, are we expecting to end the year on plan? Answers to these two questions are critical to running learning like a business, and the reporting format is taken from management reports used by sales and manufacturing.
The Need for TDRP
TDRP offers GAAP-like guidance to classify and recommend measures and reports, which vastly simplifies L&D measurement and reporting. TDRP also recommends standard names and definitions for measures to make communication easier and to increase the accuracy of benchmarking. To learn more about TDRP, join me in my webinar, Measurement Simplified: The Key L&D Program and Department Measures, where I’ll simplify the TDRP framework and show its value to you and your organization.
