A Chance Encounter with An Industry Giant: Measuring Learning Effectiveness

I had a confused look on my face and he must have sensed it. I had recently made a transition in my career from operations management to learning and development. I had a passion for learning, but I really didn’t know the first thing about it. My company at the time decided to send me to a conference to help me get a sense of the new world I was entering. So there I was, looking lost, at the annual American Society for Training and Development (ASTD)* conference. 

“Good morning!” His greeting shook me from my thoughts. I looked up and saw an older gentleman smiling while half-sitting on a display table full of books. I mumbled a hello and looked at some of the titles on the table and there seemed to be a theme:

  • Transferring Learning to Behavior: Using the Four Levels to Improve Performance
  • Evaluating Training Programs: The Four Levels
  • Implementing the Four Levels: A Practical Guide for Effective Evaluation of Training Programs
  • Another Look at Evaluating Training Programs
  • Training on Trial: How Workplace Learning Must Reinvent Itself to Remain Relevant

I looked at the author of each title – Kirkpatrick, Kirkpatrick, Kirkpatrick, Kirkpatrick, Kirkpatrick – this Kirkpatrick guy must really love evaluating training. The gentleman asked, “What brings you into the book store this morning?” I told him I was new to the field and that I was trying to learn the fundamentals so I could go back and update our learning programs. I told him I wanted our learning to have more of an impact on the business. He smiled. 

“Well, my name is Don Kirkpatrick – and these are my books!” He waved his arm over the very large array of titles on the display table. I picked one up and flipped through it, looking interested. I came to a page that had a visual of something called the Four Levels of Learning Evaluation

“So – what are the Four Levels?” I asked. Again he smiled. “Let’s start at the beginning,” he said. “You wanted to make your learning programs have an impact on the business, right” I nodded. “Well, that’s a great place to start. Let me tell you all about the four levels…”

I sat with Don Kirkpatrick for about twenty minutes and learned about the importance of measuring the effectiveness of learning. I did not realize what an impact that moment would have on my career as a learning and development professional. At the time, I did not know he created the four levels back in 1954 as part of his Ph. D. dissertation. It was like listening to George Lucas read a copy of Star Wars. Powerful. 

When he finished, I politely thanked him for his time. I noted his name and grabbed one of the books. I’ve never forgotten his generosity. A man who was a giant in the industry took the time to share his perspective with a newbie. I was saddened by his passing earlier this year, but I imagine he shared his time with many over the years and I know his work lives on. 

There are some that criticize the model, stating that it is still too event based and that it does not take into account informal styles of learning. Others say it focuses too much on training (an event) rather than learning (a process). And some, like Phillips and Kaufman, make modifications to the levels and add a 5th tier that looks to societal contributions or Return on Investment (ROI). 

For me, the 4 Levels are the foundation that started the conversation. We no longer have the freedom or the resources to conduct training for training’s sake. In the legal world, I see an opportunity for us to get more focused on Level 4 – Results. What problem are we trying to solve with our learning programs? What business outcomes are we keeping in mind when we write documentation? What is the targeted result we are looking for with custom developed eLearning? These are the questions that we need to be asking ourselves. 

And I know many of you are making this shift. I’m hearing of some fantastic changes happening within legal IT Training and these changes are leading to results. I’d be interested in hearing your thoughts. Do you like the model? Do you use it in your firm? Feel free to comment and keep the conversation going.

* The ASTD has since been renamed the Association for Talent Development (ATD). They took the word Training right out of the title. Hmm…

Traveling Coaches is here to help! We have programs that can help you elevate your learning programs, improve your communications and help your lawyers and staff embrace change.