> ## Documentation Index
> Fetch the complete documentation index at: https://docs.trevosdk.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Reading results

> Conversion rate, lift, guardrails, and why you can check whenever you like.

An experiment produces one number per arm — the conversion rate — and one judgement: whether
the difference between them is real.

## Conversion rate

```
conversion rate = conversions ÷ exposures
```

Per arm. Raw counts are not the comparison, because arms rarely receive identical traffic and
splits are not always even.

|             | Control   | Variant   |
| ----------- | --------- | --------- |
| Exposures   | 500       | 500       |
| Conversions | 90        | 130       |
| **Rate**    | **18.0%** | **26.0%** |

## Lift

Two ways to express the same movement:

* **Absolute** — subtract the rates: `26.0% − 18.0%` = **+8.0 points**
* **Relative** — the proportional change: `8.0 ÷ 18.0` = **+44%**

Both are correct. Relative is the larger, more quotable number; absolute tells you how many
additional humans actually converted. When someone reports a big win, check which one they
mean — a lift from 0.2% to 0.3% is "+50%" and also almost nothing.

## Significance, and why you can peek

A gap between two rates might be a real effect or ordinary randomness. Flip 500 coins twice
and you won't get 250 both times.

Classical A/B testing handles this by making you fix a sample size in advance and look only
once at the end. Checking early inflates your false-positive rate badly — a dashboard you
refresh daily will show you "winners" that evaporate.

Trevo uses **mSPRT**, a sequential test. Its results are *always valid*: you can look as often
as you like, stop as soon as it declares a winner, and the error rate still holds. That fits
how people actually use these tools, rather than requiring a discipline nobody sustains.

Practically: check whenever you want, and act when Trevo says the result is conclusive rather
than when the numbers first look good.

## Guardrails

The decision metric decides the winner. A **guardrail** catches the case where winning that
metric broke something downstream.

Say a more aggressive CTA lifts `signup_started` by 40%. Real win — unless
`signup_completed` fell, in which case you pulled in people who bounce a screen later and
your funnel is worse overall.

Pick a guardrail one step *after* your decision metric, or one that represents quality rather
than volume. A result that moves the decision metric and damages the guardrail is not a win,
regardless of how good the headline looks.

## Inconclusive results are results

Many experiments end with no detectable difference, and that's normal — it means your change
didn't matter as much as you hoped, which is worth knowing cheaply.

Two different endings look similar and aren't:

* **No effect detected, enough traffic** — the change genuinely doesn't move this metric. Ship
  whichever version you prefer for other reasons, and move on.
* **Not enough traffic yet** — the test cannot yet distinguish a real effect from noise. Keep
  running, or accept that an effect this small needs more traffic than you have.

Small effects need a lot of users. If your conversion rate is 2% and you're hoping to find a
10% relative improvement, that's tens of thousands of exposures per arm. Testing bolder
changes is often the faster path than waiting on more traffic.

## After a winner

When a result is conclusive, the losing code is dead weight. Trevo opens a cleanup PR that
promotes the winner into permanent code and removes the variant along with the `getVariant()`
call.

Merging that promptly matters more than it sounds: unresolved experiments accumulate into
conditional logic nobody remembers the reason for, which is the flag debt this whole workflow
exists to avoid.