# ABMeter

> ABMeter is an AI-first platform for feature flagging and A/B experimentation. It provides client SDKs for server-side, browser, and mobile apps (each documented in its own section below), a REST API, and an MCP server for managing experiments through AI assistants.

## Overview

ABMeter is a platform for running experiments and feature flags. It is designed to keep integration with your code minimal — SDKs provide access to parameters managed by the platform. Your code uses parameters as configuration without being dependent on what experiments or feature flags are running.

### AI-First Design

ABMeter is designed to be used through your AI coding assistant (Claude Code, Cursor, Windsurf, etc.) via MCP (Model Context Protocol). Connect the ABMeter MCP server to your assistant, and it can design experiments, launch them, analyze results, and integrate your code with the platform — all through natural conversation. You don't need to know what a "p-value" is; just ask how confident the experiment results are, or how long it still needs to run.

### How ABMeter Differs from Other Platforms

ABMeter gives you control over features found on platforms like Optimizely, Statsig, and EPPO without a long learning curve or requiring statistics expertise. Key differences:

- Events, metrics, exposures, and parameters are all stored on ABMeter's side
- Minimal Web UI — interact primarily through your AI assistant via MCP for a more flexible experience
- User assignments are evaluated locally without network API calls (milliseconds)
- No warehouse-native (WHN) support
- No multi-user support — single subscription gives full access, shareable with your team

### Features

- Experiments with statistical analysis
- Feature flags with audience targeting
- Audiences (user lists, predicates, random allocation)
- Metrics built from event data
- AI integration via MCP
- REST API access to all resources
- Client-side assignment evaluation (no API call per parameter resolution)
- Spaces for organizing non-overlapping experiments sharing parameters
- Ruby SDK (`abmeter` gem)
- Python SDK (`abmeter` PyPI package)
- Go SDK (`github.com/abmeter/abmeter-go` module)
- Browser JavaScript SDK (`abmeter` npm package)
- React Native SDK (`@abmeter/react-native` npm package)

## Domain Model

### User

A user of the customer's application. Identified by `user_id` (String) — the sole randomization unit. Optionally carries an `email` (String), used only for predicate-audience targeting (e.g. email-domain match); never used for identity, bucketing, or event submission.

### Space

Represents 100% of users; each space gets an independent allocation pool. A container for feature flags and experiments sharing parameters.

- Purpose: Allocation isolation — experiments within a space don't overlap traffic; experiments across spaces run in parallel
- Common patterns: by product area (checkout, search), by team, by platform (web, mobile)
- Note: Spaces provide allocation separation, not causal independence. Experiments in different spaces can interfere if their effects share causal pathways.

### Parameter

A typed configuration value with a slug, type, and default value. Parameter slugs are unique across all spaces. Parameters can be overridden in variants.

Supported types: `String`, `Boolean`, `Integer`, `Float`.

### Variant

A named group of parameter overrides. When a user is assigned to a variant, they receive all parameter values from that variant as a coherent unit (variant coherence).

### Experiment

A tool for validating hypotheses. Experiments expose multiple variants to random audiences, measure impact on metrics, and have allocation (in percent) within a space. Experiments track user exposures to variants for correct analysis.

**Experiment States:**

- **INIT**: Design phase. Not started, not included in assignment config, does not count toward space allocation. Can modify freely.
- **RUNNING**: Actively collecting data and serving variants. Included in assignment config. Audiences must total exactly 100%. Counts toward space allocation.
- **STOPPED**: Decision phase. Review results and choose: apply winner, archive without changes, or reset to fix and restart. Does not count toward allocation.
- **ARCHIVED**: Terminal state. Historical record. Cannot be modified or restarted.

State transitions: INIT → RUNNING → STOPPED → ARCHIVED. Reset from RUNNING or STOPPED back to INIT is also possible.

### Feature Flag

Incremental rollouts to non-random audiences for testing. Exposes a single variant to a non-random audience. Tracks user exposures.

### Audience

A subset of users, defined in one of three ways:

1. **User List** (named set of user IDs) — non-random audience
2. **Predicate** (condition on user properties, e.g., email domain match) — non-random audience
3. **Percentage** (random allocation) — random audience

### Event

A record of a user's action. Structure:
- `slug`: determines the event type
- `user_id`: String
- `occurred_at`: Timestamp
- Custom fields as defined by the event type

### Event Type

Defines the schema for events — which custom fields are expected and their types. When submitting a batch of events, each event is validated against its event type. Valid events are recorded; invalid events (missing required fields, unknown event type) are rejected with error details. A batch containing both valid and invalid events will record the valid ones and return failures for the invalid ones.

### Measure

An aggregation function applied to event data. All measures must reference an event type:

- `count(<event-slug>)` — number of events
- `count_distinct(<event-slug>)` — number of distinct users
- `count(<event-slug>.<field>)` — count where field is not null
- `count_null(<event-slug>.<field>)` — count where field is null
- `count_distinct(<event-slug>.<field>)` — unique values (String, Integer)
- `min`, `max`, `avg`, `sum(<event-slug>.<field>)` — for numeric fields

### Metric

A customer-scoped business KPI defined by a slug, name, description, and formula. The formula is an arithmetic expression over measures using `+`, `-`, `*`, `/`, `(`, `)`.

Examples:
- `count_distinct(trial-button-clicked.user_id) / count_distinct(session-started.user_id)` — conversion rate
- `sum(purchase-completed.purchase_value) / count_distinct(session-started.user_id)` — revenue per user

Metrics are reusable across experiments. Experiment effects are evaluated on metrics, not raw measures.

### Effect Card

Statistical effect of one audience vs control for a metric. Per-audience statistics include:
- `n`: sample size (number of users)
- `mean`: average metric value
- `variance`: spread of user values
- `std_error`: precision of mean estimate

For non-control audiences compared against control:
- `absolute_lift` and `relative_lift`: measured difference vs control
- `z_score`: test statistic
- `p_value`: statistical significance
- `confidence_interval_95`: 95% confidence interval bounds

The backend returns raw numbers only — no verdicts, winners, or recommendations. The LLM interprets results in business context.

### Assignment Algorithm

Enables deterministic parameter resolution:

```ruby
# email is optional — supply it only for audiences that target email patterns
user = ABMeter::User.new(user_id: current_user.id.to_s)
button_color = ABMeter.resolve_parameter(user: user, parameter_slug: 'button-color')
```

The algorithm evaluates in order:
1. Feature flags (checked by audience priority — user lists, predicates)
2. Experiments (hash-based random assignment using user ID mixed with experiment salt)

Assignment can be evaluated server-side (via API) or client-side (via assignment config downloaded by SDKs). SDKs keep the config in sync by periodically fetching it (once per minute, using ETags). All SDKs must produce identical assignment results.

### Variant Coherence

When a user is assigned to a variant, they receive ALL parameter values from that variant as a coherent unit. This ensures parameters work together as designed, user experience is consistent, and experiment integrity is maintained.

## Authentication

ABMeter supports three authentication methods:

### API Token

Two kinds of API keys, both shown exactly once at creation (only a hash is stored — copy the key when it is minted):

- **Secret keys** (`api-...`) — for developers, CI/CD, automation, and server-side SDKs. Full API access; accepted on `/api/*` endpoints. Keep confidential. Also accepted by the MCP server, but only as a fallback (e.g. via an `mcp-remote` proxy) for MCP clients that cannot do the OAuth flow — OAuth is the primary MCP authentication.
- **Publishable keys** (`pk_...`) — safe to embed in browser JavaScript. Limited to `POST /api/v1/user-assignments`, `POST /api/v1/exposures`, and `POST /api/v1/events`; any other endpoint returns 403. Cannot be used with the MCP server. Because browser clocks are untrusted, client-reported `resolved_at` / `occurred_at` on these submissions are replaced with server ingest time; secret-key submissions keep them verbatim.

Pass either as a Bearer token:
```
Authorization: Bearer <api-token>
```

Keys are revoked (not deleted) in the Lab UI. For zero-downtime rotation, create the replacement key first, deploy it, then revoke the old one — revocation takes effect immediately.

### OAuth2

For MCP clients and third-party applications. Uses authorization code flow with PKCE. Returns time-limited access tokens (2 hours) with refresh capability. Single `platform` scope grants full access.

Accepted on `/api/*` endpoints and the MCP server.

## REST API

Base path: `/api/v1`
Authentication: Bearer token in `Authorization` header (API token or OAuth2 access token)
Rate limiting: 300 requests per 10 minutes per API key

### Assignment Config

- `GET /api/v1/assignment-config` — Get assignment configuration (used by server-side SDKs, supports ETag caching)
- `POST /api/v1/user-assignments` — Get pre-evaluated assignments for a single user (used by browser SDKs; rules and salts stay server-side; supports ETag caching). Body: `{"user_id": "...", "email": "..."}`. Returns per-parameter resolved values plus exposure metadata for experiment resolutions.

### Spaces

- `GET /api/v1/spaces` — List all spaces
- `POST /api/v1/spaces` — Create a new space
- `GET /api/v1/spaces/{space_slug}` — Get space details
- `PATCH /api/v1/spaces/{space_slug}` — Update space
- `DELETE /api/v1/spaces/{space_slug}` — Delete space

### Parameters

- `GET /api/v1/parameters` — List parameters
- `POST /api/v1/parameters` — Create a new parameter
- `GET /api/v1/parameters/{parameter_slug}` — Get parameter details
- `PATCH /api/v1/parameters/{parameter_slug}` — Update parameter
- `DELETE /api/v1/parameters/{parameter_slug}` — Delete parameter

### Variants

- `GET /api/v1/variants` — List all variants
- `POST /api/v1/variants` — Create a new variant
- `GET /api/v1/variants/{variant_slug}` — Get a specific variant
- `PUT /api/v1/variants/{variant_slug}` — Update a variant
- `DELETE /api/v1/variants/{variant_slug}` — Delete a variant

### Experiments

- `GET /api/v1/experiments` — List all experiments
- `POST /api/v1/experiments` — Create a new experiment
- `GET /api/v1/experiments/{experiment_slug}` — Get a specific experiment
- `PATCH /api/v1/experiments/{experiment_slug}` — Update an experiment
- `DELETE /api/v1/experiments/{experiment_slug}` — Delete an experiment
- `POST /api/v1/experiments/{experiment_slug}/transition` — Transition experiment state
- `GET /api/v1/experiments/{experiment_slug}/effects` — Evaluate experiment effects
- `GET /api/v1/experiments/{experiment_slug}/exposure-stats` — Get exposure statistics

### Feature Flags

- `GET /api/v1/feature-flags` — List all feature flags
- `POST /api/v1/feature-flags` — Create feature flag
- `GET /api/v1/feature-flags/{feature_flag_slug}` — Get feature flag details
- `PATCH /api/v1/feature-flags/{feature_flag_slug}` — Update feature flag
- `DELETE /api/v1/feature-flags/{feature_flag_slug}` — Delete feature flag
- `POST /api/v1/feature-flags/{feature_flag_slug}/transition` — Transition feature flag state
- `GET /api/v1/feature-flags/{feature_flag_slug}/exposure-stats` — Get exposure statistics

### Audiences

- `GET /api/v1/audiences` — List all global audiences
- `POST /api/v1/audiences` — Create a new global audience
- `GET /api/v1/audiences/{audience_slug}` — Get a specific global audience
- `PATCH /api/v1/audiences/{audience_slug}` — Update a global audience
- `DELETE /api/v1/audiences/{audience_slug}` — Delete a global audience

### Event Types

- `GET /api/v1/event-types` — List event types
- `POST /api/v1/event-types` — Create event type
- `GET /api/v1/event-types/{event_type_slug}` — Get event type details
- `PUT /api/v1/event-types/{event_type_slug}` — Update event type
- `DELETE /api/v1/event-types/{event_type_slug}` — Delete event type

### Events

- `GET /api/v1/events` — Search events
- `POST /api/v1/events` — Submit batch of events

### Metrics

- `GET /api/v1/metrics` — List metrics
- `POST /api/v1/metrics` — Create metric
- `GET /api/v1/metrics/{metric_slug}` — Get metric details
- `PUT /api/v1/metrics/{metric_slug}` — Update metric
- `DELETE /api/v1/metrics/{metric_slug}` — Delete metric

### Exposures

- `GET /api/v1/exposures` — Get parameter exposures
- `POST /api/v1/exposures` — Submit parameter exposures

## Ruby SDK

Gem name: `abmeter` (on [RubyGems](https://rubygems.org/gems/abmeter))
Source: [github.com/abmeter/abmeter-ruby](https://github.com/abmeter/abmeter-ruby)
Ruby >= 3.2.0 required.

### Configuration

```ruby
ABMeter.configure do |config|
  config.api_key = ENV['ABMETER_API_KEY']
  # Optional settings:
  # config.base_url = 'https://abmeter.ai'  # default
  # config.flush_interval = 60   # seconds between async event/exposure flushes
  # config.fetch_interval = 60   # seconds between assignment config fetches
  # config.logger = Rails.logger
  # config.log_level = :info
  # config.error_callback = ->(error) { Sentry.capture_exception(error) }
end
```

### Resolve Parameters

```ruby
# email is optional — supply it only for audiences that target email patterns
user = ABMeter::User.new(user_id: current_user.id.to_s)

button_color = ABMeter.resolve_parameter(
  user: user,
  parameter_slug: 'button-color'
)
```

Parameter resolution is evaluated client-side using the locally cached assignment config — no API call per resolution. The SDK periodically syncs the config (every 60 seconds by default) using ETag-based caching.

### Track Events

```ruby
ABMeter.track_event('purchase', user.user_id, {
  plan_slug: plan.slug,
  price: price
})
```

Events are queued and submitted asynchronously in batches for performance.

### Debug Exposure

```ruby
# Returns full exposure details without submitting to the API
exposure = ABMeter.get_exposure(user: user, parameter_slug: 'button-color')
```

### Static Configuration

For testing or environments without API access:

```ruby
ABMeter.configure do |config|
  config.static_config = '{"spaces":[],"parameters":[],...}'
end
```

## Python SDK

Package name: `abmeter` (on [PyPI](https://pypi.org/project/abmeter/))
Python >= 3.11 required (tested on 3.11, 3.12, 3.13).

### Configuration

```python
import os
import abmeter

abmeter.configure(
    api_key=os.environ["ABMETER_API_KEY"],
    # Optional settings:
    # base_url="https://abmeter.ai",   # default
    # flush_interval=60,               # seconds between async event/exposure flushes
    # fetch_interval=60,               # seconds between assignment config fetches
    # logger=my_logger,
    # error_callback=lambda err: sentry_sdk.capture_exception(err),
)
```

### Resolve Parameters

```python
# email is optional — supply it only for audiences that target email patterns
user = abmeter.User(user_id=str(current_user.id))

button_color = abmeter.resolve_parameter(user, "button-color")
```

Parameter resolution is evaluated client-side using the locally cached assignment config — no API call per resolution. The SDK periodically syncs the config (every 60 seconds by default) using ETag-based caching.

### Track Events

```python
abmeter.track_event("purchase", user.user_id, {
    "plan_slug": plan.slug,
    "price": price,
})
```

Events are queued and submitted asynchronously in batches for performance.

### Debug Exposure

```python
# Returns full exposure details without submitting to the API
exposure = abmeter.get_exposure(user, "button-color")
```

### Shutdown

```python
# Flush queued exposures/events; blocks up to `timeout` seconds.
abmeter.reset(timeout=5.0)
```

Call `abmeter.reset()` at process shutdown to flush queued work; pass `force=True` to discard queued work immediately with no network I/O (used mainly in tests).

### Static Configuration

For testing or environments without API access:

```python
abmeter.configure(static_config='{"spaces":[],"parameters":[],...}')
```

## Go SDK

Module path: `github.com/abmeter/abmeter-go` (docs on [pkg.go.dev](https://pkg.go.dev/github.com/abmeter/abmeter-go))
Go >= 1.24 required. Zero dependencies outside the Go standard library.

### Configuration

```go
import (
    "context"
    "os"

    abmeter "github.com/abmeter/abmeter-go"
)

err := abmeter.Configure(abmeter.Config{
    APIKey: os.Getenv("ABMETER_API_KEY"),
    // Optional settings (zero values take the defaults):
    // BaseURL:       "https://abmeter.ai",
    // FlushInterval: 60 * time.Second, // background submit cadence
    // FetchInterval: 60 * time.Second, // assignment-config re-check cadence
    // Logger:        myLogger,
})
```

### Resolve Parameters

```go
// Email is optional — supply it only for audiences that target email patterns
user := abmeter.User{UserID: currentUserID}

buttonColor, err := abmeter.ResolveParameter(user, "button-color")
if err != nil {
    buttonColor = "green" // your own fallback; the SDK never panics
}
```

Parameter resolution is evaluated client-side using the locally cached assignment config — no API call per resolution. The SDK re-checks the config (every 60 seconds by default) with `If-None-Match`, so an unchanged config costs a `304`. A failed refresh keeps serving the last-known config. Errors are returned as values (Go idiom), never panics.

### Track Events

```go
abmeter.TrackEvent("purchase", user.UserID, map[string]any{
    "plan_slug": plan.Slug,
    "price":     price,
})
```

Events are queued and submitted asynchronously in batches for performance.

### Debug Exposure

```go
// Returns full exposure details without submitting to the API
exposure, err := abmeter.GetExposure(user, "button-color")
```

### Shutdown

```go
// Flush queued exposures/events; bounded by the context (default 5s).
err := abmeter.Reset(context.Background())

// Or discard queued work immediately with no network I/O (mainly tests):
abmeter.ResetNow()
```

### Static Configuration

For testing or environments without API access:

```go
abmeter.Configure(abmeter.Config{StaticConfig: `{"spaces":[],"parameters":[],...}`})
```

## Browser JavaScript SDK

Package name: `abmeter` (on [npm](https://www.npmjs.com/package/abmeter))
Targets evergreen browsers (ES2020). TypeScript types ship with the package.

Unlike the server-side SDKs, the browser never receives assignment rules, salts, or audience definitions — shipping those to a client would expose every running experiment and let any key holder forge assignments. Instead the SDK requests **pre-evaluated assignments** for the current visitor (`POST /api/v1/user-assignments`) and caches the value map. Resolving a parameter is a synchronous lookup with no network call; an exposure is recorded only when a value is actually read.

### Installation

```bash
npm install abmeter
```

Or from a CDN, which exposes `window.abmeter`:

```html
<script src="https://cdn.jsdelivr.net/npm/abmeter@0.3/dist/abmeter.iife.min.js"></script>
```

`@0.3` is a range that picks up patch releases automatically. Pin the exact version (`abmeter@0.3.0`) before production; avoid the version-less form, which follows breaking releases too.

### Publishable keys

The browser SDK requires a **publishable key** (`pk_...`), safe to embed in page source and restricted server-side to the three endpoints the SDK uses. `configure` rejects secret keys (`api-...`).

### Configuration

```javascript
import * as abmeter from 'abmeter';

abmeter.configure({
  apiKey: 'pk_your_publishable_key',
  user: { userId: 'user-123' },  // omit for an anonymous generated track id
});
```

`configure` returns immediately; identity, `localStorage` hydration, and the first assignments fetch (with ETag caching) settle in the background. `await abmeter.ready()` is the gate before reading parameters.

### Resolve and track

```javascript
const color = abmeter.resolveParameter('checkout-button-color');  // undefined if uncontrolled
abmeter.trackEvent('purchase', { price: 49.99 });
await abmeter.flush();
```

### Identity

Anonymous visitors get a generated track id (random UUID in a cookie and `localStorage`). Safari caps JavaScript-set cookies at roughly seven days; set the cookie server-side for long-running experiments.

### Submission

Exposures and events are queued and submitted in background batches, draining on `visibilitychange` and `pagehide` via `keepalive` requests with a `sendBeacon` fallback. All read and track functions are error-safe and never throw.

## React Native SDK

Package name: `@abmeter/react-native` (on [npm](https://www.npmjs.com/package/@abmeter/react-native))
A thin platform adapter over the `abmeter` JS core: same pre-evaluated-assignments model, same API, with AsyncStorage instead of `localStorage`, `AppState` transitions instead of tab events, and no cookies. Works in React Native and Expo apps; requires a publishable key (`pk_...`) exactly like the browser SDK.

### Installation

```bash
npm install @abmeter/react-native abmeter @react-native-async-storage/async-storage
```

In an Expo app, install AsyncStorage with `npx expo install @react-native-async-storage/async-storage` so its version matches the Expo SDK. `react-native` itself is a peer dependency the app already has.

### Configuration

```javascript
import * as abmeter from '@abmeter/react-native';

abmeter.configure({
  apiKey: 'pk_your_publishable_key',
  user: { userId: 'user-123' },  // omit for an anonymous generated track id
});
await abmeter.ready();
```

The anonymous track id is persisted in AsyncStorage and settles behind `ready()`, so identity — and therefore the assigned variant — is stable across app launches. Resolve and track calls are identical to the browser SDK: `resolveParameter`, `trackEvent`, `getExposure`, `flush`, `reset`.

### Submission

Exposures and events are queued and submitted in background batches; the queue also drains when the app leaves the foreground (`AppState` → `inactive`/`background`) with fire-and-forget requests — the browser `keepalive`/`sendBeacon` tab-death tricks are harmless no-ops in RN. If the OS kills the app before a background flush completes, that tail of the queue is lost. All read and track functions are error-safe and never throw.

## MCP Server

ABMeter provides an MCP (Model Context Protocol) server for AI assistant integration.

### Connection

Add the ABMeter MCP server to your MCP client:
- Server URL: `https://mcp.abmeter.ai`

### Authentication

- OAuth 2.1 with PKCE (authorization code flow)
- Dynamic Client Registration (RFC 7591)
- Single `platform` scope for full access
- Access tokens expire after 2 hours with refresh capability

### Available MCP Prompts

- `welcome` — Introduction to ABMeter and overview of features
- `design_and_launch_experiment` — Step-by-step guide to design and launch an A/B experiment
- `analyze_running_experiment` — Determine if an experiment can answer its hypothesis
- `conclude_experiment` — Analyze results, recommend a decision, and guide through concluding
- `show_resource` — Display details of a specific ABMeter resource

### Browser Linking

ABMeter supports human-in-the-loop confirmations for critical operations. After connecting your MCP client to your browser, operations like starting or stopping experiments require explicit confirmation in the browser UI.

### MCP Tools

The MCP server exposes tools for managing all ABMeter resources: spaces, parameters, variants, experiments, feature flags, audiences, event types, events, metrics, and exposures.

## Quick Start

1. **Connect MCP server** to your MCP client using the server URL
2. **Use MCP prompts** to explore: run the `welcome` prompt for an introduction
3. **Explore the demo space** — it contains realistic experiments with simulated data for safe experimentation
4. **Install an SDK**:
   - Ruby — add `gem 'abmeter'` to your Gemfile
   - Python — `pip install abmeter`
   - Go — `go get github.com/abmeter/abmeter-go`
   - Browser — `npm install abmeter`
   - React Native — `npm install @abmeter/react-native abmeter @react-native-async-storage/async-storage`
5. **Configure the SDK** with your API key
6. **Resolve parameters** in your code: `ABMeter.resolve_parameter(user: user, parameter_slug: 'button-color')`
7. **Track events**: `ABMeter.track_event('purchase', user_id, { price: price })`

Your MCP client will guide you through SDK installation, parameter creation, experiment setup, and results analysis.

## Use Cases

### Feature Flag Rollout

Roll out a new feature to internal users first, then expand:

1. Create parameters for the new feature (e.g., `plans` configuration)
2. Create a variant with the new feature values
3. Create a feature flag targeting an internal users audience
4. Internal users see the new feature; everyone else sees defaults
5. No code changes needed — code references parameters by slug

### A/B Experiment with KPI Measurement

Test a hypothesis with statistical rigor:

1. Define event types and metrics for the KPI you want to measure
2. Create a variant with the treatment values
3. Create an experiment with random audience allocation (e.g., 60% treatment, 40% control)
4. Start the experiment — users are randomly assigned
5. Monitor metrics and effects via MCP or the API
6. When results are statistically significant, stop the experiment and apply the winner

### Variant Winner Application

After a successful experiment:

1. Stop the experiment (enters decision phase)
2. Apply the winning variant — parameter defaults are updated to the winning values
3. Archive the experiment — it becomes a historical record
4. Remove experiment-specific code (variant references) from your codebase

## Statistics

ABMeter evaluates A/B experiment effects using a z-test methodology based on the Central Limit Theorem (CLT).

### Methodology

For any metric type (conversions, revenue, averages), the z-test formula is:

```
Z = (mean_treatment - mean_control) / SE
SE = sqrt(variance_treatment/n_treatment + variance_control/n_control)
```

Metrics are calculated per user, then aggregated. The CLT guarantees the sampling distribution approaches normal as sample size grows.

### Effect Statistics

For each non-control audience compared against control:
- **Lift**: relative change `(treatment - control) / control`
- **Confidence interval**: 95% CI bounds for lift
- **P-value**: statistical significance from z-test

| Z-score | Confidence | Significance |
|---------|------------|--------------|
| 1.65    | 90%        | p < 0.10     |
| 1.96    | 95%        | p < 0.05     |
| 2.58    | 99%        | p < 0.01     |

### Attribution Windows

Control which events are linked to a user's experiment participation:
- **Short (1 day)**: immediate effects — button clicks, same-session purchases
- **Long (7-14 days)**: delayed effects — email campaigns, considered purchases

### Design Principles

- Backend returns raw numbers only — no verdicts, winners, or recommendations
- Effects are cumulative from experiment start (no time filtering to prevent p-hacking)
- Time series uses cohort-based view (X-axis = exposure date, Y-axis = metric value for that cohort)
- The LLM interprets raw data in business context