Principles

Why ABMeter works the way it does, one design decision per principle.

An A/B test can go wrong in quiet ways: a user moves to the other group halfway through, two tests change the same button, a user sees the new checkout button on the old checkout page. The results still look fine, and they are wrong. Each principle is one decision that keeps a mistake like this out of your results, so you can run a sound test without having run one before.

When a step carries a risk, ABMeter explains it, and only a person can take it. Your AI assistant cannot. Start with the introduction, then read them in order.

  1. Introduction

    A/B tests are the unit tests of product decisions

    A unit test tells you the code does what you meant. An A/B test tells you the feature does what the customer needed. With AI writing the code, the second question is the one left for you.

  2. Principle 4

    A person decides: why a tool description is not a lock

    An AI assistant can design an experiment and read its results, but it cannot ship the winner. MCP only gives the assistant hints, so ABMeter keeps that rule on the server and uses the browser to bring the decision to a person.

  3. Principle 8

    Plant a lift you know: how ABMeter tests its SDKs end to end

    Unit tests show that each SDK function works. They cannot show that a user's group, exposure and purchase reach the result intact. So every ABMeter SDK runs inside a small app that plays users with a lift known in advance, and ABMeter must report that lift back.