I’ve randomly had to speak about churn recently, and instantly had my body freeze. In 12 years of product analytics, I’ve seen a few versions of it. I was lucky to be introduced kindly and politely to it: in the case of the gaming industry, where engagement is usually high, churn meant “how many players do we lose at each level”.
Most importantly, there was an action for the direction of churn: if it goes down in a few days, the level is made easier. There was a general strategy to follow for level creations, and tools to act on it.
Fast forward a few years, I’ve been in different companies, where churn had specific definitions, that no one agreed with, and ownership was split across 3 teams. Different industry, different levers to impact it, Marketing thinks they can improve it, but doesn’t want to put the budget, product does some changes but are blamed for the outcome, and so on and so forth.
Metrics can be perfectly defined but still be useless. You can track it harder, but the result will be the same:
you built a precise measurement of nothing.
This brings me back at the original idea of “data driven” and the many different shapes I’ve seen. Most often it seems to be defined by creating dashboards, and adding metrics, after which we want to add more metrics.
The promise of a decade of self served analytics came and went. The dashboards were there for the analysts to have what to pull up to prove their point. No one ever looked at them ever again once delivered. (And yes, we know this, we have tracking of dashboard usages).
The dysfunction of organisations in this sense comes from both splitting teams too much and incredibly specific, while still have to cross functionally discuss with other verticals. Working in silos is frowned upon. The analysts end up not having anymore the grip to the decision makers, to understand exactly why are we building what we are building.
We don’t get to ask the direct ones in charge, what would they do if the metrics moved up or down. And what sort of actions would they do. Attempting to be proactive in the void rarely has a valuable result, and most often it leaves the receiver of the self-serving dashboard with a blank stare and a few “thank you”s. We try to predict problems & actions but end up doing both poorly.
Newly started Print on Demand business
To contrast the previous examples, let’s go to the opposite side of the business spectrum, which are small 1 person businesses that run on creativity and grit.
Resources online are pretty clear on how to get started: you get the business set up with Meta-Ads, you leave the “algorithm running”, to let it “learn” who are the right users. Shout out to who’s been writing about this and teaching us that it’s actually a bit convoluted and the algorithm is doing a much simpler job. But I digress.
As any new product owner, it was exciting to keep track of even the few hundred pageviews that it was getting, but no sale yet. The business was 3 days old. It was too early to get sad or excited of any outcome.
Meta Ads & PoD businesses are rarely in the Product Analytics realm, as such, it was completely new and exciting territory for me to explore.
While there weren’t “final” conversions, there were intermediate steps to look in the funnel:
Once people scroll through the page, are they clicking to view an item?
Simple. Clear. The kind of thing that seems obvious when you just start. But most people don’t get that sort of help when they just start, just generic internet advice.
What particularly contrasted to me were items low on the page, but high in click through rate. I suggested to change the ad campaigns to reflect that item instead of the original one.
The first sale came 5h later.
This is not to brag about some incredible instinct or an insane idea never before tried. If anything, my professional experience would scream from the depths of my soul not to take decisions based on such low data.
But in this case:
you act on the data you have, proportionate to the business you have.
The learning here isn’t that large companies are dysfunctional and small ones are efficient. Small ones have the luxury of starting from scratch and can apply new technology from the get go. And if it were that simple, we’d be all running one-person companies all across the globe and asking each other to buy our products.
The main takeaway is that it matters more to have someone dedicated to act and move, and to have the right tools in place for acting. To have clear ownership and actionables shared. When you start smaller, that ownership is clear: it’s you. When you grow, that needs to be made explicit.
I’ve already helped a few companies along the way, to get their metrics sorted, from the simplest setup to OKRs backed up by input metrics tracked monthly. I do have a preference, but only due to really being allergic to inaction.
If you’re a CPO or a founder who recognised a little bit of themselves in here, who felt the last few months like you’ve been running on vanity metrics but you know it’s time to get serious about how you look at all the data, I am a real person and also a click away.
If you’re still on the fence, you don’t have to do it alone, let’s figure out first how we’d solve it together.