What is this book about?

If you’re involved with a startup, analytics help you find your way to the right product and market before the money runs out. But with a flood of information available, where do you start? This book shows you what to measure, how to analyze it, and how to report it, whether you’re evaluating your business model, testing new features, enticing investors, or reporting progress to advisers.

Foreward

  • Innovation, new sources of growth, and the glory of product/market fit all make for riveting drama. But all of this work rests on a foundation made of far more boring stuff: accounting, math, and metrics.
  • Vanity metrics - the numbers that make you feel good but seriously mislead.
  • Accounting is at the heart of our modern management techniques

Preface

Lean Analytics helps you identify the riskiest parts of your business plan, then finds ways to reduce those risks in a quick, iterative cycle of learning.
Don’t sell what you can make; make what you can sell.
  • Measuring something makes you accountable. You’re forced to confront inconvenient truths.
  • Lean Analytics is the dashboard for every stage of your business, from validating whether a problem is real, to identifying your customers, to deciding what to build, to positioning yourself favorably with a potential acquirer. It puts data front and center, making it harder for you to ignore.
  • Customer development is focused on collecting continuous feedback that will have a material impact on the direction of a product and business.
  • The waterfall model is a breakdown of project activities into linear sequential phases, where each phase depends on the deliverables of the previous one and corresponds to a specialisation of tasks
A startup is an organization formed to search for a scalable and repeatable business model
This cycle isn’t just a way of improving your product. It’s also a good reality check. Lean Analytics is a way of quantifying your innovation, getting you closer and closer to a continuous reality check.
This cycle isn’t just a way of improving your product. It’s also a good reality check. Lean Analytics is a way of quantifying your innovation, getting you closer and closer to a continuous reality check.

Chapter 1 - Stop lying to yourself

Analytics is the necessary counterweight to lying.
Instincts are experiments. Data is proof.
  • If we follow the Lean model, it becomes increasingly hard to lie, especially to yourself (because data won't let us)
  • Data-driven learning is the cornerstone of success in startups. It’s how you learn what’s working and iterate toward the right product and market before the money runs out.
  • There’s a reason the Lean Startup movement has taken off now. You can iterate quickly - you can build something, measure its effect, and learn from it to build something better the next time.
  • Minimum Viable Product is the smallest thing you can build that will create the value you’ve promised to your market.
  • Now that it’s cheap, even free, to launch a startup, the really scarce resource is attention. A concierge approach in which you run things behind the scenes for the first few customers lets you check whether the need is real; it also helps you understand which things people really use and refine your process.
  • Sometimes, growth comes from an aspect of your business you don’t expect. When you think you’ve found a worthwhile idea, decide how to test it quickly, with minimal investment. Define what success looks like beforehand, and know what you’re going to do if your hunch is right.
  • Analytical thinking is about asking the right questions, and focusing on the one key metric that will produce the change you’re after.

Chapter 2 - Finding the right metric

Metrics matter because they relate to your business model—where money comes from, how much things cost, how many customers you have, and the effectiveness of your customer acquisition strategies.
Signs of a good Metric:
  • A good metric is comparative.
  • A good metric is understandable.
  • A good metric is a ratio or a rate - Ratios are easier to act on and are inherently comparative.
  • A good metric changes the way you behave.
  • Experimental metrics help you to optimize the product, pricing, or market.
  • Metrics often come in pairs. Example - Conversion rate (the percentage of people who buy something) is tied to time-to-purchase (how long it takes someone to buy something)
  • Quantitative data is nice and scientific - you can aggregate it, extrapolate it, and put it into a spreadsheet. Qualitative data is messy, subjective, and imprecise. It’s the stuff of interviews and debates.
  • Collecting good qualitative data takes preparation. Unprepared interviews yield misleading results.
  • Whenever you look at a metric, ask yourself, “What will I do differently based on this information?” If you can’t answer that question, you probably shouldn’t worry about the metric too much.
  • Total active users is a vanity metric. It will gradually increase over time, unless you do something horribly wrong. A better metric would be percent of users who are active.
Vanity Metrics to Watch Out For:
  1. Number of page views.
  1. Number of visits. Is this one person who visits a hundred times, or are a hundred people visiting once?
  1. Number of unique visitors.
  1. Number of followers/friends/likes.
  1. Number of downloads. While it sometimes affects your ranking in app stores, downloads alone don’t lead to real value. Measure activations, account creations, or something else.
  • A leading metric tries to predict the future. On the other hand, a lagging metric, such as churn gives you an indication that there’s a problem.
  • In the early days of your startup, you won’t have enough data to know how a current metric relates to one down the road, so measure lagging metrics at first. Lagging metrics are still useful and can provide a solid baseline of performance. For leading indicators to work, you need to be able to do cohort analysis and compare groups of customers over periods of time.
  • Finding a correlation between two metrics is a good thing. Correlations can help you predict what will happen. But finding the cause of something means you can change it.
  • Make early assumptions and set targets for what you think success looks like. Lower the bar if necessary. Use qualitative data to understand what value you’re creating and adjust only if the new line in the sand reflects how customers are using your product
 
badge