Misaligned metrics and crap maps
Misaligned metrics and crap maps
Organisations often suffer from alignment issues and poor models. Are we part of the problem? Can we find the cure?
Picture yourself running a cross-country trail. It’s close to nightfall and you want to finish asap. Unfortunately, your heart is working towards something else: it's optimising for a low average heart rate. You move at a trudge. Your muscles have the capacity to burn through far more oxygen than they’re given as the blood pumping around your body is capped due to a different, contradictory goal. Now, imagine your map of the trail is elegant and convincing and created by someone with only a theoretical notion of the route. You’re in for a rough night.
As product managers, we know how important it is to establish vision statements and clear, measurable outcomes. But, unless you’re working on a very small or early stage product, it’s likely that elements of the work will be handled by different teams. They are organs within a larger organism and, like a heart that’s misaligned to the body as a whole, friction will develop if they do not work in sync towards a common cause. And, if this body is not fed useful information about the world around it, this dysfunction will be compounded further still.
How can we help improve organisations with inefficient team topologies and poor models of the world around them? In this blog post, I'll explore how we can spot such issues and mitigate them through data, focus and aligned incentives.
Case Study
Let’s use the example of a fictitious health insurance company, ‘LoveLife’. It’s membership is acquired through multiple channels, including D2C (Direct to Customer, in which individuals purchase the product for themselves via the company’s website or call centre), brokers (where individuals purchase the product via an intermediary, who takes a cut of revenue) and enterprise (where employees receive cover as part of their package). Profit varies by channel, but broadly speaking, LoveLife needs to retain customers for 2+ years for a positive ROI.
The company has organised itself around Steps: Acquisition, Onboarding & Membership, and Retention, which leaders believe all users must go through. It’s their model of the customer journey, and product managers and many other functional specialists are tasked with delivering high performance within their Step. For example,
Acquisition have chosen to focus on sales volume and cost per acquisition (CAC)
Onboarding & Membership focuses on engagement with product perks while reducing the contact centre volumes
Retention focuses on end of contract renewals.
The expectation is that Step-level excellence will equal organisational brilliance:
Smart people have designed this approach. They want to be ‘customer-first’ while keeping org structures simple. Better still, each Step has a sponsor exec tasked with driving it forward. What could possibly go wrong?
Well, two things:
Misalignment of incentives and metrics
The model isn’t based on reality.
Let’s dig deeper.
Acquisition and misalignment
The Acquisition team focuses on selling LoveLife to new customers. Funnels are built which define a win as First Payment Made. Optimisation work lasers in on conversion rate and volumes, and bonuses are linked to deals closed. Yet, the Sales team are incentivised to acquire customers who are not a good fit for the product. The organisation’s profit relies on member retention but the Step team’s strategy might boost sales-related metrics while negatively impacting lagging metrics, such as lifetime value (LTV).
Again, the individuals here may be doing great work within their box. But working in silos is dangerous, even if this silo is cross-functional and data-driven. In complex organisations, it is vital that targets and incentives align, and ladder up to something greater. Consider a key tenet of lean manufacturing: you should improve your part of the workflow but the measure is optimising for the production line as a whole.
LoveLife should create a collective focus on a North Star metric: a leading indicator of retention, such as ‘Percentage of customers onboarded and engaged X months after joining’. In turn, bonuses for the Acquisition step might only mature when a customer has passed this threshold. Paid search and other marketing activities would optimise for this metric, not straight-forward sales conversion, and so on.
By aligning around metrics for the whole organisation, not individual organs, the body as a whole is better served.
Onboarding and theoretical models
At LoveLife, the onboarding experience is focused on turning new customers into engaged members. The Step team knows that engaging with membership benefits is the ‘aha moment’ and a strong indicator of long-term retention. In this sense, they are strongly aligned to the notional North Star metric in a way that their Acquisition peers are not.
Yet their elegant concept of ‘onboarding’ as a universal post-sales experience may not bear contact with reality. Why?
Let’s look back at what the Acquisition team structured around: channels. The sales team did not see the customers won through D2C, brokers and employers as a single mass. For example, D2C members actively choose LoveLife and a strong interest in member benefits might therefore be expected. Members who get the product as a perk of their employment package may be less engaged or educated on the product's benefits. Broker members may lie somewhere in between.
How many onboarding experiences utilise channel data to segment audiences and create journeys that fits users? LoveLife’s onboarding and engagement performance might benefit by leveraging the understanding of Acquisition, creating experiences that account for the different knowledge and needs of new customers.
It’s a trap!
Sounds simple, right? Yet this model might also be wrong. The only way to optimise effectively is to analyse quantitative and qualitative data and to run experiments to test hypotheses. My hypothetical channel-centric approach might not hold water. Drilling into the motivations and actions of users is the best way to create compelling customer experiences: just remember to share this understanding widely, so the whole organisation benefits from what you learn.
A Step beyond
What then, might an improved org structure look like at LoveLife?
We need not junk the overall Steps concept, but a couple of big changes could be in order:
Step metics (and performance incentives) ladder to the North Star
Customer segmentation is considered at every step.
On an organisational level, we can also see how team topologies and internal coordination might be structured around channel ‘swimlanes’.
With customers more clearly in view and incentives aligned, it’s not hard to see how the Retention team could optimise their approach, compounding the Retention Rate improvements delivered upstream. LoveLife investments will have a better ROI: let’s just hope they spend some of their profits on further refining their models through data and experimentation.
Seeing the bigger picture
As product managers, we are often tasked with building products and journeys to meet a specific need. But we need to have the depth of thought and emotional maturity to see where our work might drive dysfunction. Step back and consider the principles of good Product. Advocate for the application of North Star metrics - or other measures that encourage holistic excellence. Even more importantly, place data at the centre of your decision making and use it to challenge theoretical models and assumptions. Reference your new and evolving understanding of reality wherever possible. Rail against misaligned metrics and crap maps, fight for the whole body, not the individual organs.
Good way to get alignment incrementally!