Beyond Smile Sheets: Measuring Learning Effectiveness

Imagine the scenario. Your firm has recently hired a new Executive Director/COO and she sets up a meeting with each function to discuss one question: What is your value to the firm? 

If you were in this position, how would you prepare? What data would you bring? What information would be critical for articulating the learning function’s value to the firm? OK. Now, take a breath as I am sure this is a fictional scenario, right? No one questions the value of learning at your firm…

This “What’s Your Value” scenario was an exercise from a training session I attended many years ago and it has stuck with me. It is very difficult for many learning professionals to answer that question. You may track some data on volume (courses attended, eLearnings viewed, documentation views), gather information about satisfaction, perhaps review calls to the Help/Service Desk, and maybe even assess skills of certain populations. However, often it’s not much deeper than that. And you hope that the leadership gets it and doesn’t put you on the spot. It’s not that you don’t want to measure results. I often have conversations with Trainers and Training Managers who have the best intentions and want to evaluate their effectiveness, but a few things get in the way.

No one ever asks – why spend the time tracking?

This is a valid point. You have the data mentioned above on volume, satisfaction and calls. You may even track more, but no one ever asks for it. The monthly and quarterly metrics reviews come and go and you may have the opportunity to present one slide, if you are even invited.  In my view, it is critical to be proactive and demonstrate value before anyone asks. Anytime an organization has discussions about budgets and resource allocation, value comes up in the conversation. It’s important to keep an eye on metrics because you may need resources from time to time in order to deliver value. Would you ever buy a stock and not look at the price or EPS over time? Would you make a bet on a game and forget to check the score? Measurement helps articulate the yield on training investment dollars. I saw a nice quote in an ASTD (now ATD) publication. It states that learning professionals “must demonstrate their value in terms that translate across and up the hierarchal [sic] structures so that learning is seen as a strategic partner in reaching the organization’s goals and not just another cost center” (Managing the Learning Function, 2007). You need to tell them about your value before they ask.

I don’t know how – where do I start?

In my experience, many of the law firm training personnel have similar backgrounds. You may have come from the support side of IT, you may have been a former secretary or document processor, or you may have worked in training or instructional design your entire career. All of these jobs are time consuming and require selfless service to others. You invest your time and effort into helping others grow. That is what attracts many to the profession. However, you may not always take the time to build your skills. Measurement and evaluation of learning is a topic that has been around for a long time and it makes sense to invest some time learning about this skill. Don Kirkpatrick is the first place to start. He created the “four level” model for training course evaluation as the subject of his Ph.D. dissertation in 1954. His company gives some great free webinars, speaks regularly at learning conferences, and has a certification program that is available. Sadly, Don passed away earlier this year. I had the opportunity to have a short breakfast with him once at a conference. He was a very smart man who did a lot for the profession. Jack Phillips is another person to research. He has another model built on the framework of the Kirkpatrick model, but Phillips includes calculations for Return on Investment (ROI). The Net Promoter Score (NPS) is another option for evaluation that is gaining some traction within learning and development. This is built on the “Ultimate Question” which is basically a likelihood-to-recommend question (Why Net Promoter).  I suggest doing a little research on these topics as a way to get started.

My customers don’t know – what do I do now?

All they do is ask us to make training! We need a class on X or we need a reference guide on Y. You may feel you do not have the chance to find out about goals or performance problems the customers are trying to solve. They simply don’t know the answers to your needs assessment questions. This is a common issue in many organizations. It requires a bit of education and a lot of courage to fully understand the business needs before heading off to develop the solution. Albert Einstein said, “If I were given one hour to save the world, I would spend 59 minutes defining the problem and one minute solving it.” How much time do you spend researching and understanding the problem before heading out to solve it? If you define the problem in business terms, it makes development and measurement easier. And I guarantee that sometimes the problem does not require training as the solution. These are hard conversations, but they are critical to connecting the dots between your efforts and value to the firm. You may want to increase your understanding of your customers' goals and challenges. This will give you insight into the technology competencies required to succeed. You may spot the training opportunities before your customers do. You will feel the shift at some point when you proactively work with your customer. You will move from training order taker to learning/business consultant. 


If you ever end up in the office with the new Executive Director/COO, it’s a good idea to know how to articulate your value to the firm. This is an area that makes the difference between a training team that is considered a cost center and a learning function that is a strategic asset to the firm. 

Traveling Coaches is here to help! We have programs that can help you elevate your learning programs, improve your communications and help your lawyers and staff embrace change. 

Feel free to comment and keep the conversation going.