No network call on the hot path: why ABMeter evaluates in your process

An experiment service sits in front of your most important code: the checkout page, the signup form, the pricing table. If every parameter lookup is a request to that service, its latency becomes your latency, and its outage becomes your outage. ABMeter’s server SDKs are built so that a lookup is a function call, not a request.

Count the network calls

Here is a checkout handler with the Ruby SDK. It reads two parameters and records a purchase:

def checkout
  user = ABMeter::User.new(user_id: current_user.id)
  color = ABMeter.resolve_parameter(
    user: user, parameter_slug: 'checkout-button-color'
  )
  layout = ABMeter.resolve_parameter(
    user: user, parameter_slug: 'checkout-layout'
  )

  order = place_order(color: color, layout: layout)
  ABMeter.track_event(
    'purchase', current_user.id, { price: order.total }
  )
end

How many requests to ABMeter does this handler make? Usually none. The two resolve_parameter calls compute the values in your process. The two exposures they record and the purchase event go into a queue in memory, and a background thread sends them later.

How a value is computed locally

When the SDK starts, it downloads the assignment config: what it needs to compute any parameter’s value for any user, for every space in your account.

With the config in memory, assigning a user is arithmetic. ABMeter keeps a random string, a salt, for each space and each experiment, and a hash of a salt and the user id decides whether the user is in an experiment and which group they get. The same user id always produces the same answer, in every process and in every SDK, so a user sees the same value on every request without ABMeter storing who got what.

The SDK refreshes the config on an interval: every 60 seconds by default, and you can configure it. So a request waits on ABMeter at most once per interval, when a refresh is due. Every other request reads the config that is already in memory.

Exposures and events go through a queue

An exposure says that a user saw a value an experiment controls. ABMeter needs exposures to know who was in the experiment, and events to measure what they did. Neither has to arrive while the user waits.

So resolve_parameter and track_event only add an item to a queue, and the SDK sends the queue in the background once per flush interval. The flush interval is also 60 seconds by default and also configurable. Recording an exposure or an event never depends on ABMeter answering, so it never fails a request for your user.

The browser is different

A browser cannot hold the assignment config. The config holds the salts and the rules for every experiment you run, and anything you send to a browser is public. With the salts, anyone could compute which group any user id lands in, and the config would also show every test you run.

So the browser SDK asks ABMeter instead. On configure it asks for the values assigned to the current user, and gets back only those. After that, resolveParameter is a synchronous lookup, and exposures and events go through a queue just like on the server.

The browser also cannot hold your secret API key. It uses a publishable key, which starts with pk_. With it, the browser can read its own user’s values and report what happened, and nothing else.

Drain the queue before you exit

A process that exits takes its queue with it. Drain it on purpose in three cases:

• Short scripts and jobs that finish before their first flush.
• Tests that check what reached ABMeter.
• Deploys and restarts.

In Ruby:

ABMeter.reset(timeout: 5.0)

reset sends what is queued, waits up to the timeout, and returns true if everything went out or false if time ran out. The Python, Node and Go SDKs have a reset call with a timeout too. The browser SDK drains on its own when the page is hidden or closed.

Try it

Install the Ruby SDK with gem install abmeter, pick a parameter, and call resolve_parameter a thousand times in a loop while a packet capture watches the traffic to ABMeter. You will see one config download and nothing else until the next flush. Call ABMeter.reset and watch the batches go out.