●Jan Layola
Blog/03/Product · Analytics

From guesswork to evidence.

Product analytics is a way to answer questions, not a way to collect data. This is how I take a team from opinions to evidence: starting from decisions, keeping the vocabulary small, closing the loop, and designing for privacy from day one.

Illustration: scattered opinions converging into a single rising line, with a decision marked on it

Most product teams have strong opinions about their users and very few ways to check them. Product analytics closes that gap, but only if you treat it as a way to answer questions rather than a way to hoard data.

I've introduced analytics to a team before, and the tool turned out to be the easy part. This post covers the rest: what to measure, how to name it, how to make sure someone actually looks, and how to do all of it without treating your users' privacy as an afterthought.

01 — Where it startedOpinions are cheap

At YieldComputer I introduced product analytics and instrumentation to the team. Before that, we could see what we shipped but not how people used it, so prioritisation leaned on judgement, anecdotes and whoever made the strongest case in the room. That isn't a criticism of anyone. Without evidence, judgement is all you have.

The change I cared about most wasn't a dashboard. It was the team moving from pure development to product-oriented thinking: asking who a feature is for, what it should change, and how we'll know. Analytics gave that shift something to stand on. "I think users want this" became "here's what users actually do", and prioritisation got a basis the whole team could see.

What I took from it
Installing a tracker takes an afternoon. Getting a team to agree on what it wants to know, and to act on the answer, is the actual work.

02 — Decisions firstStart from the decision

The most common way to start is also the worst: install a tracker, switch on everything it can capture, and hope insight falls out. It doesn't. You end up with a large pile of events that nobody trusts and nobody reads.

I work backwards instead:

  1. Decision
  2. Question
  3. Signal
  4. Event
  5. Review

First, name a decision the team will actually have to make. Then the question behind it, the signal that would answer that question, and only then the event that produces the signal. The last step is agreeing when you'll look at it again. A few illustrative examples:

DecisionQuestion behind itSignal that answers it
Invest more in onboarding?Where do new accounts stall before their first success?Drop-off between the steps of the first-run flow
Keep, fix or remove a feature?Do people who find it come back to it?Repeat use by accounts that tried it once
Is the new flow better than the old one?Do more people finish it, with fewer errors?Completion and error events, split by variant

My rule is simple: if I can't name the decision, I don't add the event. That alone keeps the tracking plan small enough to trust.

03 — A small vocabularyName things once

Events are an API between the product and everyone who reads the data, so I treat them like one: a small vocabulary, a consistent grammar, and changes that get reviewed like code.

tracking plan · illustrative
# object_action · past tense · snake_case
report_exported            { format: "csv", surface: "dashboard" }
onboarding_step_completed  { step: "connect_site" }
filter_applied             { filter: "date_range", surface: "table" }

# properties describe the context, never the person:
# no emails, names, free text or anything that identifies someone
  • Object, then action, in the past tense. Events record something that happened. export_clicked tells you about a button; report_exported tells you about an outcome.
  • Properties add context, not identity. Which surface, which format, which step. They never say who the person is.
  • One tracking plan, in the repo. It changes through pull requests, so a rename is a reviewed change rather than a silent break in every chart that depends on it.
  • Instrumentation is part of "done". If a feature exists to change behaviour, the pull request that ships it also ships the events that show whether it did.

04 — FocusInstrument the path that matters

You don't need to measure the whole product on day one. You need to measure the path that decides whether someone gets value: from arriving, to the first moment it works for them, to coming back.

Activation is usually where the biggest and cheapest wins hide, because every new user walks through it. Everything that goes wrong there goes wrong for everyone.

So I start with one path, instrumented end to end and checked against reality before anyone draws conclusions from it. One funnel you trust beats ten you argue about. Then I widen, one question at a time.

05 — RitualsA metric nobody looks at is noise

The failure after launch is quieter than the one before it: dashboards that exist and nobody opens. A number only matters if it's attached to a moment when someone will act on it, so I tie metrics to the rituals the team already has:

  • Planning. Every item meant to change behaviour says which signal it expects to move, and in which direction.
  • Retros and release reviews. We look at what moved, and just as importantly at what didn't.
  • An owner per metric. Someone who notices when it breaks, drifts or stops making sense.

Numbers need conversations

Quantitative data tells you what is happening and where. It rarely tells you why. For that I pair it with qualitative evidence: watching real sessions (with inputs masked), reading support conversations, and talking to the people who use the product.

“The numbers tell me where to look. The conversations tell me what I'm looking at.”

06 — Privacy by designTrack behaviour, not people

Privacy and security are long-standing interests of mine. In 2024 I took Cisco's Cybersecurity Foundations, which covers privacy and data confidentiality, and my own accounts live on Proton. So when I introduce analytics, privacy isn't a legal checkbox at the end. It's part of the design:

  1. Consent first.No tracking before someone has said yes, and saying no is as easy as saying yes.
  2. Collect the minimum.Every property has to earn its place against a question. If it can't, it goes.
  3. No personal data in events.No emails, names or free text in properties, and identifiers that are pseudonymous rather than personal.
  4. Mask by default.Session replays and heatmaps hide inputs and sensitive text unless someone deliberately decides otherwise.
  5. Tooling that fits the jurisdiction.In Europe that means EU hosting, clear processing terms and sensible retention. PiwikPro, which we used at YieldComputer, is built for exactly that world.
  6. Keep data only as long as the question needs it.Raw events aren't an archive. Retention should follow purpose.

Done this way, privacy and good analytics pull in the same direction. Fewer, better events are easier to trust, and easier to defend.

07 — How engineering changesSmall, observable steps

Once a team can see behaviour, the way it builds changes too. Features ship in smaller slices, because each slice can be checked. Risky changes go behind feature flags, so you can roll them out to a few accounts, watch, and then widen or roll back. When a question is genuinely open, you run an experiment instead of an argument.

The most valuable change is the least visible one: deciding what not to build. Evidence makes it much easier to push back, gently, when a feature is solving the wrong problem, because the conversation is about what users do rather than whose opinion wins.

What I do now
I stay close to design and product, prototype early, and ship in small, observable steps. Each step answers a question before the next one gets built.

08 — The honest partWhat can go wrong

  • Vanity metrics. Page views and total sign-ups go up and to the right and tell you almost nothing. Prefer signals tied to a decision.
  • Instrumentation debt. Events rot like code: buttons get renamed, flows get removed, naming conventions get half-migrated. Without a tracking plan in the repo and a review step, your charts quietly start lying.
  • Over-reading small numbers. In B2B products especially, a handful of accounts can swing a chart. Treat small samples as a prompt for a conversation, not as a verdict.
  • Correlation dressed up as causation. People who adopt a feature are often simply your most engaged users. Experiments or careful comparisons beat eyeballing a trend.
  • Dashboards as theatre. A wall of charts can make a team feel data-driven without changing a single decision.

09 — TodayEvidence as a product

At InSpace, the Experience Team builds NOVA, where evidence is the product itself. Clients open it to see how their organic traffic, their rankings and their citations in AI answers are moving. That makes these rules doubly relevant. Every number we put on a client's screen should answer a question they actually have, mean the same thing everywhere it appears, and lead somewhere they can act.

For my own product work, these days I reach for PostHog. Having events, funnels, feature flags and session replays in one place makes the whole loop much easier to close.

If you're starting tomorrow
Write down the next few decisions your team has to make and the question behind each one. Instrument only what answers them.
Then keep it honest
Put the tracking plan in the repo, give every metric an owner and a ritual, and keep personal data out from day one.
Written by
Senior Engineer, Experience Team Lead at InSpace · Breda, NL

I build NOVA's client platform and the design system behind it, and write about product engineering, frontend craft and shipping software with coding agents.

What would you measure first?

Questions, pushback or your own take? I read every email.