The first two articles were about what your code reads: parameters, not flags, and variants that come as a bundle. This one is about how an experiment gets set up in the first place. ABMeter has two doors, the REST API and the MCP server, and one small window, the web UI.
One experiment, two ways
Take the green checkout button from the first article. The parameter checkout-button-color exists, and so does a variant green-button that sets it to green. Now create an experiment that gives half of the traffic to the variant.
With the API, it is one request:
curl -X POST https://abmeter.ai/api/v1/experiments \
-H "Authorization: Bearer $ABMETER_API_TOKEN" \
-H "Content-Type: application/json" \
-d '{
"slug": "checkout-button-green",
"hypothesis": "A green button gets more checkouts than gray",
"allocation": 100,
"audiences": {
"control": 50,
"green": {"percentage": 50, "variant": "green-button"}
}
}'With an AI assistant connected over MCP, it is one sentence:
Create an experiment that tests the green-button variant against control, 50/50, on all traffic.
The assistant calls the create_experiment tool with the same fields: a slug, a hypothesis, the allocation and the two audiences. ABMeter checks them with the same rules and stores the same experiment. Either way, nothing reaches your users yet. A new experiment waits in its first state until somebody starts it.
There is no third way. The web UI has no form that creates an experiment, and no form for parameters or variants either. That is not a missing feature.
Why the UI is small
The reason is my own experience. In the experimentation products I tried, setting up an experiment, defining the metrics and checking that events actually arrive was a project in itself: screen after screen, each with its own vocabulary, before the first exposure came in. I did not want to build another UI like that, and I did not want to learn one. It is far easier to describe the hypothesis you want to check to an AI assistant and let it create the parameters, the variants and the metrics, and check that the events arrive.
The rules live in one place, and both doors go through it. A variant that sets different parameters than its siblings, or a feature flag that would override a running test, is refused the same way through the API and through MCP.
So what is the UI for? Two things.
- Decisions a person should make. Starting an experiment, stopping it, picking the winner, and turning a feature flag on or off. These change what real users see, so they happen where you are signed in and press a button.
- Looking at results. Exposure counts over time, and effects with their confidence intervals. A chart is easier to read than JSON.
Everything else, the configuration you would type into forms, comes through the API or through MCP.
MCP in two paragraphs
MCP, the Model Context Protocol, is a standard way for an AI assistant to use an outside service. The app you chat in is the client. ABMeter runs a server at https://mcp.abmeter.ai. The server offers tools: named operations with typed inputs and a short description, like create_experiment or evaluate_experiment_effects. When you ask for something, the assistant picks a tool, fills in the inputs, and reads the answer. ABMeter’s tools create, update and search your configuration and events, read exposures and effects, and bring your browser to the page where you decide.
To connect, you add the server URL to your client. The client opens your browser, you sign in with GitHub or Google, and you approve the connection. Each connection shows up on the MCP Clients page in ABMeter, where you can revoke it. The Setup MCP Connection page has the steps.
What the assistant is good at, and what it cannot do
An assistant is good at the parts of an experiment that are mostly reading and writing:
- Design. It turns “I think a shorter checkout converts better” into parameters, a variant, a hypothesis, and a metric to judge it by. It can search what already exists, so it reuses a parameter instead of creating a second one.
- Monitoring. It reads exposure counts and tells you whether traffic arrives and whether the groups are the sizes you set.
- Explaining results. It reads the effects, with their confidence intervals and p-values, and tells you in plain words what they say and what they do not.
It cannot start or stop an experiment, pick a winner, turn a flag on or off, or delete anything. There is no MCP tool for any of these. When you ask for one, the assistant can open the right page in your browser, if you have linked it, and you press the button there. A person decides explains why these steps stay with a person.
The REST API can start and stop experiments. That is on purpose too. A script you wrote and reviewed, running with your API key, is a decision you already made. A sentence in a chat is not.
Try it
Connect your assistant with the steps on the Setup MCP Connection page, and ask it to call setup_demo. That loads a demo space with experiments and data. Then ask it which experiment in the demo looks like a winner, and why. When it answers, open the same experiment in the web UI and check its reading against the chart.