Team
Growth teams
The experiment you want to run is behind a release you do not control
The funnel says the fifth step is where the money leaves. Fixing that step means engineering time you have to bid for, against a roadmap that was set last quarter, so the test that would settle it waits and the drop stays where it is.
The numbers
20%
typical win rate across all experiments, falling to 10% for experiments tied directly to revenue
Optimizely, experimentation benchmarks
41.6%
of tests end inconclusive, neither a significant winner nor a significant loser
DRIP, A/B testing statistics, 2026
60%
rise in customer acquisition cost over five years, which is what makes the funnel the cheaper place to work
ProfitWell, published by Paddle
What ARCY sees
ARCY reports the step and the reason together, so a hypothesis starts from what the user was trying to do rather than from where the line fell.
- The step people stop on, per segment, per environment and per source
- What they asked the agent at that step, in their own words
- Struggle signals: the repeated attempt, the long pause, the field filled and cleared
- Which segments never reach the step at all, which is a different experiment
What ARCY does
A flow is authored and published in a day, without a deploy, so the intervention you want to test is not queued behind a release. It is delivered through the chat surface, never as an overlay painted on your product.
- Publish a flow at the step, in both languages, and change it the same afternoon
- Answer the question the step raises from the sources you approved
- Target the step by what it does rather than by a fragile selector, so a redesign does not silently end the test
What it measures
Every measured flow reports the difference between the users it helped and the users who never used it, over the same window. That is measurement most in-product tooling leaves to you. It is a comparison between two groups of people rather than a controlled test, and ARCY says so beside the number instead of letting you find out later.
- ARCY compares the users it helped against the users it never touched
- Completion in both groups, per flow
- A result that is inconclusive says so, which on these odds is most of them
+36.3% activation lift on this flow
With ARCY
42.8%
1,204 users who ran this flow
Without ARCY
31.4%
2,910 users who never used ARCY
ARCY compares the users it helped against the users who reached the same step and never used it. Both are counted as finished the same way.
Related
Run the test without waiting for the release
Book a walkthrough and we will look at the step you would test first, using your own funnel.