How do you define effective KPIs?
Some thoughts on how to define high-quality Key Performance Indicators
Happy New Year!
Welcome back to the Data Rampage newsletter. I hope you had a very nice break, whether you were celebrating Christmas or not. I spent a week in Valencia, Spain, with my family, and I can assure you that we had a very very nice time!
I'm excited to present this fourth edition of the newsletter, and interested to see where it goes from here over the next year. Many thanks to everyone who has already subscribed and shared the content; feel free to add a comment or to reach out directly with any questions or suggestions.
So ... let's get into it!
In this edition I am going to discuss how to design key performance indicators, or KPIs. If you're not familiar with the concept of KPIs (and here I'm assuming that most people reading this newsletter actually are familiar with the term), the basic idea is that KPIs are those quantitative measurements that most closely represent the 'true' state of business performance, hence why they are called 'key'.
I'm well aware that there is a huge amount of content online about creating KPIs, but much of it (from what I can see with an admittedly cursory Google search) has been written by marketing types, so I'd like to provide a more analytics-based perspective on the process.
I'm going to start by discussing what KPIs are and are not, what are the characteristics of a good KPI, how to develop them collaboratively, and what I see as the role of Data professionals in this process. As usual, I'm going to draw on my own experiences in writing this, and share what's worked for me, and where I feel I fell short in the past ... and what I learned from those mistakes!
Designing a good KPI process
These are the three key elements of a good KPI process:
Establish a clear business strategy
Agree a definition of success
Measure progress towards your goals
KPIs are not your business strategy, they are a representation of your progress towards achieving your business strategy. Your business strategy is the goal of your business, and every business has an existential goal.
Here are some examples of goals that different types of businesses might have:
The goal of a small barbershop might be to provide a decent living standard for the owner as well as a living wage to the employees
The goal of a software startup might be to grow very quickly and either achieve an IPO or be purchased by a larger firm
The goal of a privately-held midsize family firm might be to weather business trends and achieve multigenerational wealth and financial stability
The goal of a major multinational corporation might be to maximize shareholder value via steady growth and high profitability
Different goals mean different definitions of success. Which means, naturally, different KPIs. The hypothetical barbershop owner will likely be paying much more attention to profit margins than the startup founder burning VC cash in an attempt to reach escape velocity.
What makes a good KPI?
So a KPI should describe an essential business outcome; not something tangential, but actually crucial, which is why it's important to have some discipline and limit the number of metrics that you define as KPIs.
You might have the best of intentions, but if you have dozens of KPIs, then effectively you have none. I read a very interesting book years back called The Paradox of Choice by Barry Schwartz (here is a link to a summary article in PDF form), which made the point that people struggle to process increasing choices past a certain point, and that unlimited choices actually reduce happiness. Similarly in business, it's long been a truism that focus is key, which is why the most effective companies tend to focus on their core competences. In the data world, however, it can be easy to avoid these lessons, because there is a tendency to think that more information is automatically better, which is how you end up with 'KPI' dashboards with dozens of metrics (and here I've certainly been guilty of this as well, let me assure you!).
How to avoid this?
I've come to see KPIs as a pyramid, running from the top to the bottom of the organization, where the smaller KPIs are building blocks that control progress on the higher-level KPIs. To ensure focus, you should only choose a few KPIs at each level of the pyramid.
For the sake of discussion, let's use a simple three-level pyramid, running team -> department -> company.
Company-level KPIs: These indicate the company's progress towards its big strategic goals
Department-level KPIs: These indicate the department's contribution to the company-wide KPIs
Team-level KPIs: These indicate the team's contribution to the department's KPIs
Ideally, the KPIs at each level should seamlessly contribute to the KPIs above.
Let's look at a more explicit example: growth (aka sales and marketing).
Here's a diagram showing a simplified company structure and a chain from the company-level down through the growth department into the individual teams. Each level should have its own set of KPIs that help to drive the KPIs at the level above. So in this example you would likely have customer growth as a KPI for the growth department, then each of the individual teams would have their own KPIs that would be expected to contribute to customer growth; for example organic search performance for the SEO team, and cost of customer acquisition (CAC) for the performance marketing team.
What are the characteristics of good KPIs that I’ve seen?
Clarity: Everyone should understand the metric and what it means; if it still needs to be explained months after being introduced, it's a bad KPI
Easy to remember: Easier when you don’t have to keep track of dozens!
Buy-In: There should be widespread (ideally unanimous!) agreement that this KPI represents an important definition of success
Changeable: If the metric can't really be altered by the direct efforts of the team/department/company, then it's probably not worth focusing on
Business impact: The KPI is something with real business impact; avoid vanity numbers that sound nice (x million pageviews! y million users!) but don't push you towards your business goals
Accessible: The KPI should be easy to automate or, if automation isn't possible, easy to calculate and/or copy-paste from somewhere. If you have to spend hours per week generating a number via lots of manual work, that's time you can better spend elsewhere
How can you work with your colleagues to define KPIs?
The KPI definition process shouldn't be rushed; it's worth taking time to think deeply about it. If you're going to let these numbers guide you, and affect the way you manage the business, make sure you are clear about what you are doing!
I think the KPI development process should be an intensely collaborative process, ideally involving all the teams, from the c-suite downwards. Data leadership should work closely with other members of the senior leadership team to identify actionable, tangible KPIs that reflect the company's strategy, and it's also important to create goals for each KPI. These goals should be tough but achievable ... I'm sure I'm not alone in having worked in companies where the KPI goals seemed completely arbitrary!
One thing I’ve learned is that the Data Team has a key role to play in the discussion phase; it’s important to have those conversations about why a metric is actually important, and even to use statistical reasoning to try to test whether or not it actually has the impact that its proponents claim. I think that sometimes in the past I’ve been too eager to figure out the automation part that I haven’t stopped to really question my stakeholder as to why they think a particular metrics counts as a KPI.
The Data Team, therefore, should take the lead in facilitating the conversation around KPIs, and making sure that the process generates KPIs that match the characteristics I identified above. One important thing to mention is that if any of the agreed top-level KPIs cannot be easily/accurately automated with existing data, it should be a company-level priority to make that possible; in some cases this might mean making fixes/improvements to production data tables, in others the responsibility might be solely on the data team to make it happen (in which case they will need to pause at least some other projects to make it happen).
Here are some other ways in which I see the Data Team contributing to defining and propagating KPIs:
Accessibility: The KPIs should be easily accessible to everyone in the company; especially the company-wide ones. Is the data locked up in a Tableau dashboard that not everyone has access to? Find a way to share that data directly to somewhere that everyone has access to
Usability: As much as possible, the Data Team should make it possible for people to not just see their KPIs, but also to work with them and understand the underlying drivers that affect performance
Reliability: The Data Team must accurately calculate the KPIs; the data must be trustworthy!
Explain: Don't just list changes or progress towards goals, but also explain to the best of your ability why KPIs are changing, and what affects them
Repetition: Don't assume that everyone gets it, make sure that you're constantly sharing reiterating the goals
Give everyone a sense of ownership!
Thanks for reading! I hope this post is useful and has some helpful advice for you about how to define your KPIs.
Something else to read
It’s well worth reading this post-mortem of the analytics firm Whywhywhy by the founders; very insightful on some of the challenges around building new products in this space.
One last (music) thing
As I mentioned before, I’ve been a dj for 25 years, so I’ve decided to end each newsletter with one of my mixes from my (extremely extensive!) back catalogue.
Self-indulgent? Sure.
For this one, I’ve chosen a mix I did last year; a tribute to my all-time favorite house producer, Armand van Helden. I’m not a mega house head, but I’ve always loved his stuff, so this mix was a real labor of love. The mix itself is a 30 track long mix that covers some 30 of his tracks and remixes from his wildly creatively fertile period in the mid-90's. It's organized chronologically, from 1994(ish) to 1997(ish), and it has remixes, aliases, collaborations, and everything from well-known anthems to more obscure stuff (and this is by no means comprehensive, the man's back catalogue is deeeeep!). Enjoy!
Thanks for reading!
If you missed the last post, it’s here:
