Tell Compelling Product Stories by Visualising Your Analytics
Perception is Reality. Create a shared perception of your product with analytics visualisation to create a shared reality
“Simplicity – the art of maximising the amount of work not done – is essential.”
- Agile principle number 10.
It is often said that the mark of a good Product Manager is to be data-led, and that good product decisions are rational and grounded in data. But engaging with large organisations also means engaging with stakeholders who aren’t nearly as close to that data as your product team, and can often misinterpret it. They may even find themselves spooked when confronted with a piece of data which, out of context, might point to a misfiring product or feature.
This can easily wreak havoc if you do not approach the problem with due care and attention. Your stakeholders’ perception often drives the reality within your or your client’s organisation. One conversation on the subject can quickly spread like wildfire, until the bosses get involved to sort out the mess which you created, by not correctly setting the tone in the first place. This can all be avoided if you provide a proper analysis of the data yourself, in context, and find a visually engaging way to present it. In doing so, you get them on your side, and focused on the areas that matter.
You’re closer to the data than your stakeholders, and some of your team are even closer still.
Tracking analytics is hard, but it is only the first step. The analysis is more complex, and there is still a lot of room for misconstrual. If you leave it up to whatever analytics platform you or your clients choose, you are beholden to its quirks and idiosyncrasies, of which there are many, regardless of which one you choose. The way things are tracked and set up often leads to misleading data points, which together lead to the wrong conclusions.
If for whatever reason you don’t know, there is an engineer in your team who will, and you should help them understand that their support will circumvent problems further down the line. Let them help you analyse the data which is already being tracked. At this stage, you should make a conscious effort to empathise with your stakeholders, anticipate the sorts of questions they might ask, and try to either answer them directly, or reframe the question to something more pertinent.
Some of these questions might seem obvious to you, but they won’t be for 80% of your audience. For instance, on a recent working call for a financial services client, the question was asked why there was such a significant drop-off rate from the start of a new document-upload feature compared to the end, based on a funnel that had been set up in Adobe Analytics. Based on that funnel, it appeared to only have a ~40% success rate, which is incredibly low for a feature of this type. Objective data, subjective interpretation.
In context, in terms of what the customer is seeing, thinking, and doing at each stage in the process, we could account for the vast majority of that drop-off. At the start of the flow, customers were discovering the feature, learning about how it works, and perusing the onboarding content with zero commitment at that point in time. Taking a later stage of the process as the trigger for commitment, the success rate grew to almost 97%. But it took proper data analysis to come to that conclusion.
Visualisation makes your analysis so much more powerful.
Analysis is hard, and requires no small measure of both empathy and rigour, but it is still only the second step. You have to take what you see in the data, and make it highly visible within your organisation. An answer to a stakeholder question, provided in writing and buried in an email chain, isn’t visible enough to set the tone. That is where visualisation comes in.
Take the data off the analytics platform, and create your funnel, which maps onto existing artefacts, such as a journey map, using the actual screens, in the flow your users go through.* Where drop-offs are accounted for with legitimate errors, draw lines out of the funnel with screens depicting those error states.
*This assumes you already have such artefacts, and I highly recommend you create them if you haven’t already, but the reasons in support of them are beyond the scope of this article.
This can be done in Miro (see above), directly into slides, or any platform of your choosing, in collaboration with your product or experience designer. Let your analysis guide the significant stages within the funnel, and characterise each stage in terms of what the user is doing at this point in the flow. In our case, that was Awareness, Curiosity, Commitment, Confirmation and Success. Then, add the numbers you can glean from your analytics for each screen, with any additional or contextual information there may be.
This has two main benefits. First, it takes what seems evident to you and makes it obvious to your audience. Second, it frames the discussion on the areas that matter, which in our case was Commitment onwards. Third, because it overlays neatly onto the actual screens in their proper context, with entry points and drop-off reasons at the right stages, you bring your analysis to life. You’ve created something everyone can engage with, bringing them along with you.
Shared perception is shared reality.
In doing so, your perception becomes their perception, and your shared perception, in turn, gives rise to a shared reality. You can then spend the rest of your time with them having a more productive conversation about what comes next, which areas need genuine tweaking and optimisation, and how high a priority that work is. In our case, this was what happened after the flow (with the exception of some light-touch UI and copy changes) when the document is in process.
None of this comes out of the box, whatever analytics platform you choose. It is hard, but making things simple often is. You reap the benefits further down the line, in the form of maximising the work not done, thereby enabling you to focus on the next high priority outcomes. All the while helping your team shine in the eyes of your organisation, by demonstrating the quality work they have produced.