Which Calculator Should I Use?
A quick guide to choosing the right statistical tool for your experiment.
What Are You Trying to Do?
“I want to reduce my sample size using baseline data”
You have pre-experiment measurements (last week's revenue, baseline lab values, historical engagement) that correlate with your outcome. CUPED uses this to reduce variance by 25-50%, meaning you need fewer subjects.
“I want to plan interim analyses with early stopping rules”
You're planning a trial where you want to look at the data partway through and potentially stop early for efficacy or futility. GSD provides statistically rigorous stopping boundaries that control Type I error.
“I want to design a trial using Bayesian methods”
You want to incorporate historical data, use informative priors, or design a trial with Bayesian decision rules. The Bayesian Toolkit is a 5-step workflow that takes you from prior specification through sample size determination.
The 5-step workflow:
“I want to evaluate whether to continue at an interim”
You're at an interim analysis and want to know the probability that the trial will succeed if you continue. Bayesian PPoS gives you a direct probability to inform Go/No-Go decisions.
Note: If you're designing a new trial (not just evaluating an interim), start with the Bayesian Toolkit Overview to understand the full workflow.
Quick Comparison
| I want to... | Use This | Key Output |
|---|---|---|
| Reduce sample size using baseline data | CUPED | Adjusted sample size, variance reduction % |
| Plan interim analyses with early stopping | GSD | Stopping boundaries, expected sample size |
| Design a trial with informative priors | Bayesian Toolkit | Prior specification, sample size, operating characteristics |
| Incorporate historical control data | Bayesian Borrowing | Effective prior, sample size reduction |
| Size a single-arm Bayesian trial | Bayesian Sample Size | N with Type I error and power |
| Size a two-arm Bayesian RCT | Two-Arm Bayesian | N per arm, allocation ratio impact |
| Evaluate trial success probability at interim | Predictive Power | PPoS (%), Go/No-Go recommendation |
| All of the above for a complex trial | See below | Combined approach |
Combining Calculators
For sophisticated trial designs, these tools complement each other:
Start with GSD for the overall design
Plan your interim analysis schedule and stopping boundaries. This gives you your maximum sample size and expected sample size under various scenarios.
Add CUPED for variance reduction
If you have baseline covariates, calculate how much smaller your trial can be. Apply the reduction to your GSD sample size.
Use Bayesian PPoS for internal Go/No-Go
At each interim, use PPoS to inform your decision to continue. GSD boundaries handle regulatory stopping; Bayesian informs internal strategy.
For Bayesian trial designs, follow the 5-step workflow
If using Bayesian methods for primary analysis (not just interim monitoring), start with Prior Elicitation and work through the toolkit sequentially.
Common Scenarios
| Scenario | Description | Calculator(s) | Approach |
|---|---|---|---|
| Tech A/B Test | Testing a new feature on user engagement | CUPED | Use last week's engagement as baseline |
| Phase III Clinical Trial | Pivotal trial with planned interim analyses | GSD + Bayesian PPoS | GSD for boundaries, Bayesian for futility |
| Phase II Proof-of-Concept | Single-arm or small RCT with Go/No-Go gate | Bayesian Toolkit | Prior → Sample Size → PPoS for decision |
| Single-Arm with Historical Borrowing | Phase II using external control data | Bayesian Toolkit | Prior → Borrowing → Sample Size → PPoS |
| Rare Disease Trial | Small population, strong historical data | Bayesian Toolkit | MAP prior with aggressive borrowing |
| Pediatric Extrapolation | Leveraging adult trial data | Bayesian Borrowing | Power prior with appropriate discount |
| Long-Term Outcomes Study | 5-year follow-up with event-driven endpoints | GSD | O'Brien-Fleming boundaries |
Decision Tree
What's your primary goal? │ ├─ Reduce sample size with baseline data │ └─ → CUPED │ ├─ Plan frequentist interim stopping rules │ └─ → GSD │ ├─ Design a new Bayesian trial │ │ │ ├─ Do you have historical data to borrow? │ │ ├─ YES → Bayesian Toolkit (Prior → Borrowing → Sample Size) │ │ └─ NO → Bayesian Toolkit (Prior → Sample Size) │ │ │ └─ Single-arm or two-arm? │ ├─ Single-arm → Bayesian Sample Size calculator │ └─ Two-arm → Two-Arm Bayesian Design calculator │ └─ Evaluate an ongoing trial at interim └─ → Predictive Power (PPoS)
Still not sure?
Read the detailed documentation for each calculator.
Frequentist Tools
Bayesian Toolkit
Conceptual Guides