Month: April 2024

Training Return-on-Investment

At Traveling Coaches, we recognize the importance of maximizing the value of training investments. Recently, one of our Learning Strategist had the honor of participating as a panelist in a Training Return-on-Investment webinar, where industry experts convened to explore strategies for effectively measuring and enhancing Training ROI. In this recap, we highlight key insights from the session.

Understanding Key Performance Indicators:
Key Performance Indicators (KPIs) play a significant role in evaluating training programs. It’s a topic that many firms are discussing, but it’s also one that can make individuals uncomfortable if they’re not sure what they should be measuring, how they should measure, and why they should measure. KPIs are measurable values that signal whether success is likely. They can range depending on the firm’s business drivers but typically include metrics such as:

User Adoption: Are they using it? Are they using it properly or efficiently? Are they using it to the fullest capabilities?

Staff Ratios: Measures the ratio of a specific type of employee to fee earners. This data can be helpful to look at on an overall staff, support staff, or trainer-to-fee earner basis. Learning programs are often implemented to positively impact overall ratios.

Customer Satisfaction: Measures the general perception of the effectiveness of learning services. This can be done at an overall, program/service-specific, or event level. Many firms use an annual Satisfaction Survey to measure this metric.

Time to Competency: Measures the average time it takes until the expected competence level is reached by employees. The measurement can be done through assessments, interviews, and surveys.

Assessment/eLearning Pass Rate: Measures the scores on assessments or eLearning programs. This data can be helpful in identifying additional learning needs.

Alignment with Business Goals:

A central theme of the webinar was the imperative of aligning training programs with overarching business objectives. KPIs need to be customized to your business situation and should be developed to help you achieve your goals. It’s important to begin with the end goals in mind when designing training initiatives to identify what success looks like. By ensuring alignment with organizational strategies, firms can accurately measure the ROI of their training investments and drive tangible results. When we are intentional and plan what gets measured, we’re able to measure what matters to clarify success and to facilitate effective stakeholder communication and engagement.

Overcoming Challenges

Common challenges faced by law firms in measuring and demonstrating the ROI of IT training were discussed. Some firms struggle with knowing what and how to measure, while others fear failure. Echoing the sentiment of Peter Drucker: “What gets measured, gets managed.” By focusing on meaningful metrics and acknowledging the value of qualitative outcomes, firms can navigate these challenges and unlock the full potential of their training programs.

By understanding KPIs, aligning training with business goals, and overcoming measurement challenges, organizations can elevate the effectiveness of their training initiatives and drive sustainable growth.

Traveling Coaches is ready to help you empower your people and elevate your firm. Reach out to us today to maximize your Training ROI.

How To Talk to Lawyers about the Effective and Responsible use of AI

Before we get into the specifics of this article, let’s level-set on a few things about Generative Artificial
Intelligence (GenAI):

  • Is it Evil or is it Good? NEITHER. It’s probably best to stop ascribing morality to a technology. It’s in the use of things that it gets such a moniker.
  • Is it intelligent? NO. It is really machine learning based on what it has to learn from (there are some interesting tangential thoughts here – think GIGO …or Garbage In, Garbage Out).
  • Is it AMAZING? Oh yeah, most definitely YES! The technology is extraordinary and has the potential to do things in seconds that would take a really talented person hours.
  • Will it take over the world and be the end of all life as we know it? NO. While it clearly provides the conspiracy theorists with fodder for Skynet references (come on … we’ve all seen Terminator), this technology will be disruptive but not destructive (IMHO).

There are a lot of directions you can take when discussing GenAI for lawyers and the legal community. You can take the complete risk-averse mindset of “Not only NO, but heck NO!” Or you can take the daredevil mindset of “these are innovations that will change everything.” Of particular interest to lawyers, however, is a balanced view of both. There are interesting stories around its misuse, some cool stories around efficiency gains, and a whole lot of speculation about its impact on job roles and how it might change the look of the modern-day law firm. But how do we talk about the GenAI juggernaut with lawyers? Here are a few things to consider:

Know your audience

In fact, I would take this one step further. Don’t assume you know your audience. Ask them. A mistake many have made over the years is to take a new technology and try to “make it fit” into a lawyer’s way of working. Instead, we should start with the business issue(s) facing the lawyer and how this can help solve the business need(s). The best way to know the business issues facing lawyers …is to ask lawyers.

Something else to consider … what matters to you may not matter to a lawyer. Always approach the introduction of technology changes with lawyers based on the real (not perceived) business needs of the lawyer.

Know the Ethical Guiding Principles to which All Lawyers Are Bound

Ethical rules are not just platitudes or suggestions or “it would be nice if” types of things for lawyers. They are rules that lawyers are bound by and that they’ve sworn to. In fact, there are sanctions for not staying within the ethical guidelines established for lawyers. This is no small thing. Lawyers are held to a higher standard …and they should be. This is also not a bad thing. In fact, it’s a good thing. Ethical standards and rules help tremendously when faced with a decision tree moment. For example, they provide guidance and direction for when to and when not to use certain technologies (which is OH SO relevant to the GenAI discussion).

The American Bar Association (ABA)’s model rules are guides for this discussion. For reference, the ABA sets model rules that are, in most cases, adopted by each state for lawyers with their State Bar in that state. Using the ABA’s model rules provides us with a broad, yet widely adopted place to start. Two notable rules come to mind.

Duty of Competence (ABA Model Rule 1.1): Let's cover some salient points to this rule:
It requires you to maintain requisite knowledge and skill. This is the reason CLE is required. It also makes clear that things are constantly changing, and a lawyer has a responsibility to stay current …including technologically. It requires that you keep abreast of the changes in the law and its practice. The laws and rules around the use of GenAI are changing by the minute. In fact, some judges are adding rules for their court specifically around the use of GenAI. It requires you to understand the risks and benefits associated with relevant technology. See any connections here to the use of GenAI? You are required to be aware not only of the good things (the benefits), but also the not-so-good things (the risks).

Duty of Confidentiality (ABA Model Rule 1.6): This one should be clear, but sometimes gets lost in the practical application. Comment 16 to this rule states that “Reasonable precautions” must be taken to safeguard and preserve confidential information. Understanding the technology tools and how they handle, store, and secure access to confidential information is part of that reasonable precaution mandate. 

What about Data Privacy? It has been said that public GenAI solutions are the exact opposite of Data Privacy. Adding confidential information to a public GenAI solution makes that confidential information very public. Data Privacy laws are changing rapidly world-wide. All ethical use of GenAI solutions must align with a lawyer’s (and law firm personnel’s) responsibility to maintain data privacy and protection. It’s the law.

Take a Balanced Approach

Lawyers want a breakdown of the facts so they can make informed decisions. Lawyers also want practical and actionable steps to use a technology like GenAI. Put GenAI topics into ethical and effective use case scenarios. Find the uses of GenAI that add efficiencies and solve business problems without violating their ethical requirements. Align the discussions of the effective use of GenAI with the policies and procedures established by the law firm and the firm’s core values.

About the Author

Kenny Leckie

Senior Technology & Change Management Consultant

In his role as Senior Technology and Change Management Consultant, Kenny provides thought leadership and consulting to the legal community in areas of information security/cybersecurity awareness, change management, user adoption, adult learning, employee engagement, professional development, and business strategy. He also works with clients to develop and deploy customized programs with an emphasis on user adoption and increased return on investment. Kenny is a Prosci
Certified Change Practitioner, a Certified Technical Trainer and has earned the trust of firms across the US, Canada, The UK, Europe and Australia.

Kenny has more than thirty years of combined experience as a law firm Chief Information Officer, Manager of Support & Training, and now consultant providing him a unique point of view and understanding of the challenges of introducing change in law firms. He combines his years of experience with a strategic approach to help clients implement programs that allows focus on the business while minimizing risk to confidential, protected, and sensitive information. Kenny is an author and speaker and a winner of ILTA’s 2018 Innovative Consultant of the Year.