SDK errors stay in the SDK: why an experiment never breaks your request

An experiment changes a button color. It should never be the reason a checkout fails. So ABMeter’s SDKs follow one rule: an error inside the SDK does not escape into your code. The SDK reports it and gets out of the way.

The busiest hour

Here is a checkout handler with the Ruby SDK. It reads one parameter:

def checkout
  user = ABMeter::User.new(user_id: current_user.id)
  color = ABMeter.resolve_parameter(
    user: user, parameter_slug: 'checkout-button-color'
  ) || 'blue'

  render_checkout(color: color)
end

Now imagine the worst time for a failure: your busiest hour, and the SDK cannot reach ABMeter.

If the SDK raised, the error would travel up through resolve_parameter, through checkout, and out as a 500. Every customer who tried to pay in that hour would see an error page, and the cause would be a button color.

ABMeter’s SDK catches the error instead. resolve_parameter returns no value, the SDK reports the error, and || 'blue' serves the color you had before the experiment. The customer pays. You read the report later.

Your fallback is the value you shipped before

The fallback lives in your code, not in ABMeter. When the SDK cannot answer, it cannot tell you the parameter’s default either, so the line that reads a parameter also says what to serve without it. Make that the value you would ship without ABMeter: the color the button had before anyone tested it.

That one fallback covers every failure the same way. Your code does not need to know what went wrong inside the SDK to keep serving the page.

Configure fails loudly

There is one exception. configure raises when the setup is wrong, for example without an API key. A wrong setup is a mistake you want to see at boot, once, before any user arrives. A failure at request time is something you want to survive.

Go returns the error

The Go SDK follows Go. ResolveParameter returns the value and an error, like most Go functions, and your code checks the error and serves blue.

Go code does not throw. A Go developer expects a function that can fail to say so in its signature, and to decide at the call site. Hiding the error would surprise them the same way a raising call would surprise a Ruby developer. The goal is the same in both: an SDK failure becomes your failure only if you decide it should. In Go, you make that decision in plain sight.

Errors reach your logs and your tracker

A caught error is not a hidden one. The SDK writes it to your log, and you can send it to the error tracker you already use with an error callback:

ABMeter.configure do |config|
  config.api_key = ENV.fetch('ABMETER_API_KEY')
  config.error_callback = ->(error) { ErrorTracker.notify(error) }
end

The Python, Node.js and browser SDKs have the same option. So an SDK error shows up where your other errors do, and you decide how loud it should be.

After you add a parameter to your code, check two things: your error tracker has no errors from the SDK, and the experiment’s exposures grow in the ABMeter UI soon after you start it.

Try it

Install the Ruby SDK with gem install abmeter. Configure it with a base_url that points to a port where nothing listens, and call resolve_parameter in a console. You get no value back and one error in the log, and the console keeps running.