Rapid discovery and data visualisation

Learn how to combine smart surveys with high-impact charts to move everyone's understanding forward, fast

It’s the start of the pandemic and I’m on Zoom with a new stakeholder. My portfolio has just expanded to cover a B2B product and I’m trying to get my head around its purpose, user-base and the market. It’s a positive call, but within ten minutes I know that: 

1. I’m in a feature factory

2. The product sucks: users ‘hate it’ and are ‘threatening to stop working with us’. This hasn’t fully resonated internally, but senior leaders hate the design and want to introduce a dark mode asap

3. There’s an understanding that retention and sales are valuable, but outcomes and success metrics haven’t been properly defined. No instrumentation is in place

4. Individual clients are having bespoke feature requests fulfilled. My stakeholder’s current priority is integration with a mid-sized client’s back office. The feature can’t be scaled to help others.

There’s a lot to do. I need to rapidly orientate myself in this area, determine true priorities and bring people with me.

Orientation

Time and resources are scarce in this environment. There isn’t a willingness to wait on substantial Discovery work but I need to start moving towards a customer-centric approach immediately. Any delay will see customer complaints increase, eroding whatever tolerance exists for ‘unconventional’ approaches. It’s crucial that I quickly understand what’s important, create a framework for measuring performance and build my story.

I decide to focus on four things:

1. Familiarity with the product

   Even if it’s unclear what value its features provide, I need to understand what’s there, how it works - and the technology stack on which it’s built. This will give me a baseline understanding for my next steps.

2. Business line mechanics

   I’m relatively new to this area, so I need to understand some of the fundamentals. I might not immediately be an expert, but some knowledge will allow me to draft performance metrics and understand stakeholder perspectives.

3. Meet some customers

   I embark on a small series of customer interviews, helping me appreciate the gaps between our product’s features and experience, and it users’ wants and needs.

4. Evaluate the competition

   Unexpectedly, an interviewee gives me sight of the equivalent products from two major competitors. This furthers my understanding of what success looks like and establishes where the current bar is set.

After a few days, I have a solid understanding of the challenges we face. These cluster around a handful of themes derived from the customer interviews, like Stability, Support, Navigation, Logins and Ease of Use. But I haven’t yet established a strong way to prioritise competing opportunities or tell this story to my stakeholders. I need a way to refine my thinking, broaden our collective understanding of what’s wrong with our product - and rally energy around what we need to focus on now.

Prioritisation

I need to find an approach that does three things:

1. Helps me prioritise my high-level themes

2. Creates a data-set and narrative to overcome existing biases

3. Gets me there fast.

I don’t have a great memory for detail, but love concepts and read material on Product, Lean and Analytics avidly. Thinking through my problem, I remember an approach outlined in The Lean Product Playbook, Dan Olsen’s brilliant guide to moving things forward quickly. Throughout the book, Olsen recommends a series of lightweight approaches to move the needle at pace - without compromising the essentials of good practice. This includes a method for rapidly appraising and prioritising features by Importance and Satisfaction. Respondents are asked to score features along these two parameters from 1-7. By looking for gaps between the two data points, opportunities and priorities are revealed.

Following Olsen’s approach, I create a simple customer survey centred on the themes uncovered through the earlier interviews - plus some additional areas perceived as important internally. It takes about 30 minutes to construct. I get hold of a distribution list and… fire!

One week later: and the results are in. For all the internal talk of scrappy design and the desire to acquiesce to noisy clients, we have two outstanding problem areas to tackle first: Stability and Login. People just want to get into the damn thing without having identity issues - and they don’t want their work to be lost mid-session when they’re in there.

Telling the story

Finding my visual hook

I now have a clear set of priorities to go after. But the job is only half done: internal communications are vital and I need my stakeholders to believe in the data. These ambitious leaders have come to believe they always know what’s right for their customers. They are not accustomed to making data-driven decisions, but they are smart and pragmatic. 

I need a novel but straight-forward way to sell the data. A bar-chart is too bland, while formats like pizza-charts are too intricate. So I go with something in-between: a Radar chart. More often used in competency analysis or console games like FIFA, it feels just right for this data and audience. Within ten minutes, I’ve put it together, looking something like this:

The meeting

Just a couple of weeks after taking the product on, I’m back on Zoom - this time with a wider group. I present my initial diagnosis of the product’s malaise, give a broad sense of where we stand compared to competitors, make recommendations around what we should look to measure - and give a clear, data-supported view on where we should focus our immediate attention. The structure of the Importance / Satisfaction survey helps raise the level of debate and the radar chart helps everyone grasp where our biggest problems lie. The requests for UI tinkering go out the window - as do client X’s feature request. All eyes turn to stabilising the product and making login in easier and more reliable.

With priorities now set, all we need to do now is turn the product around.

Key takeaways

Pairing Dan Olsen’s survey structure with the radar chart worked well for me here. But this project underlined some far broader principles that I’ve remembered ever since:

• Quick discovery is always better than no discovery

• Talking to customers is often less time consuming - and more fun - than debating assumptions in meetings

• Data will trump opinion in most environments -  it will raise the level of debate too

• Data visualisation can be used as a narrative tool. Choose a compelling (but not gimmicky) format that’s right for your audience and story. This doesn’t need to be technically complex. Playing around with tools like Sheets and Excel can help you stay current

• Product Managers must move beyond stakeholder assumptions. We didn’t suddenly leap from feature factory to empowered team here: far from it, in fact. But we must assert our agency and take steps towards domain expertise, no matter how challenging the environment. This is how we add value to our users and businesses.

  ---