Docs/Bayesian Sample Size

Bayesian Sample Size Determination

Technical documentation for sample size calculation in single-arm Bayesian trials with operating characteristics. This module determines the minimum sample size to achieve target power while controlling Type I error under a Bayesian decision framework.

1. Overview & Problem Setup

This calculator determines the sample size for a single-arm Bayesian trial comparing a response rate to a null hypothesis threshold using posterior probability as the decision criterion.

Design Setting

Endpoint: Binary outcome (response/no response)
Arms: Single-arm (no control group)
Prior: Beta distribution (from Prior Elicitation module)
Decision: Posterior probability criterion

Hypotheses

Null Hypothesis

H_0: \theta \leq \theta_0

Response rate is at or below an unacceptable threshold

Alternative Hypothesis

H_1: \theta > \theta_0

Response rate exceeds the threshold (drug is effective)

Typical Application: Oncology Phase II

A common scenario is testing whether a new therapy achieves a response rate significantly above the historical standard. For example: $\theta_0 = 0.10$ (null, historical rate) vs. $\theta_1 = 0.20$ (alternative, target rate).

2. Bayesian Decision Rule

The trial succeeds if the posterior probability that the response rate exceeds the null threshold exceeds a decision threshold:

\text{Declare Success if } P(\theta > \theta_0 | \text{Data}) \geq \gamma

Where $\gamma$ is typically 0.95 or 0.975, analogous to one-sided frequentist significance levels of 0.05 or 0.025.

Posterior Computation

With a Beta prior $\text{Beta}(\alpha, \beta)$ and observed data $k$ successes in $n$ trials:

\theta | k, n \sim \text{Beta}(\alpha + k, \beta + n - k)

The decision probability is computed as:

P(\theta > \theta_0 | k, n) = 1 - F_{\text{Beta}}(\theta_0; \alpha + k, \beta + n - k)

Example Calculation

Prior: $\text{Beta}(6, 44)$ (mean 12%, ESS 50)
Observed: 25 responders out of 100 patients
Null threshold: $\theta_0 = 0.10$

Posterior: $\text{Beta}(6+25, 44+75) = \text{Beta}(31, 119)$
Posterior mean: $31/150 = 20.7\%$
$P(\theta > 0.10) = 1 - F_{\text{Beta}}(0.10; 31, 119) = 0.9998$
Decision: Success (99.98% > 95% threshold)

3. Operating Characteristics

Despite using a Bayesian decision rule, we evaluate the design using frequentist operating characteristics to satisfy regulatory requirements. This is computed via Monte Carlo simulation.

Type I Error Rate

Probability of declaring success when $\theta = \theta_0$

\alpha = P(\text{Success} | \theta = \theta_0)

Target: ≤ 0.05 (or 0.025 for confirmatory)

Power

Probability of declaring success when $\theta = \theta_1$

1 - \beta = P(\text{Success} | \theta = \theta_1)

Target: ≥ 0.80 (or 0.90 for pivotal)

Monte Carlo Simulation Algorithm

for simulation in 1...N_simulations:
    # Under null hypothesis (θ = θ₀)
    k_null = Binomial(n, θ₀)
    posterior_prob_null = P(θ > θ₀ | k_null, n, prior)
    if posterior_prob_null >= γ:
        type1_count += 1

    # Under alternative hypothesis (θ = θ₁)
    k_alt = Binomial(n, θ₁)
    posterior_prob_alt = P(θ > θ₀ | k_alt, n, prior)
    if posterior_prob_alt >= γ:
        power_count += 1

type1_error = type1_count / N_simulations
power = power_count / N_simulations

Prior Impact on Operating Characteristics

An informative prior can inflate Type I error if it's too optimistic, or reduce power if it's too skeptical. The calculator reports prior ESS and its contribution to the posterior to help assess prior influence.

4. Sample Size Search Algorithm

The algorithm searches for the minimum sample size that achieves target power while maintaining Type I error control using a grid search approach.

Search Strategy

1.Grid Search: Evaluate operating characteristics at $n \in \{n_{min}, n_{min}+n_{step}, n_{min}+2 \cdot n_{step}, ..., n_{max}\}$
2.Constraint Check: At each $n$ , verify Type I error ≤ target AND power ≥ target
3.Selection: Return the smallest $n$ meeting both constraints. If none found, return the best trade-off (prioritizing Type I error control).
4.Sensitivity: Evaluate at $n \pm 5, \pm 10, \pm 20$ for robustness

Search Output

The calculator returns the smallest $n$ satisfying both constraints, along with all evaluated sample sizes (for custom analysis) and sensitivity results at nearby sample sizes.

5. Power Curve Analysis

The power curve shows how the probability of declaring success varies with the true response rate. This is essential for understanding the design's operating characteristics across the parameter space.

\text{Power}(\theta) = P(\text{Success} | \theta) = P\left(P(\theta > \theta_0 | k, n) \geq \gamma\right)

Key Points on the Power Curve

Point	True Rate	Interpretation
Type I Error	$\theta = \theta_0$	False positive rate at null
Power at Design Alternative	$\theta = \theta_1$	Primary endpoint
Crossover	$\theta^*$ where Power = 50%	Minimum detectable effect

Interpreting the Power Curve

A steeper power curve indicates better discrimination between null and alternative. The curve should cross 50% somewhere between $\theta_0$ and $\theta_1$ , ideally closer to $\theta_0$ .

6. Sensitivity Analysis

The calculator provides sensitivity analysis across sample sizes near the recommendation to understand the robustness of the design.

Default Sensitivity Grid

Operating characteristics are computed at:

$n_{rec} - 20$ (under-powered option)
$n_{rec} - 10$
$n_{rec}$ (recommended)
$n_{rec} + 10$
$n_{rec} + 20$ (over-powered buffer)

Prior Sensitivity (Regulatory Requirement)

Per FDA guidance, sensitivity to prior specification should also be assessed. Consider re-running with an uninformative prior $\text{Beta}(1, 1)$ to understand the marginal benefit of the informative prior.

7. Regulatory Considerations

FDA Bayesian Guidance Requirements

The January 2026 FDA guidance mandates reporting of frequentist operating characteristics (Type I error and power) even for Bayesian designs to ensure proper error rate control.

SAP Documentation Checklist

Prior Specification: Report $\alpha, \beta$ , source, and ESS
Decision Rule: State $\theta_0$ , $\gamma$ , and success criterion
Sample Size: Report $n$ with operating characteristics
Type I Error: Demonstrate control at ≤ 0.05 (or 0.025)
Power: Report power at design alternative $\theta_1$
Sensitivity: Show robustness to prior and sample size

8. API Quick Reference

POST /api/v1/calculators/bayesian-sample-size

Key Parameters

Parameter	Type	Description
prior_alpha, prior_beta	float	Beta prior parameters (from Prior Elicitation)
null_rate	float	Null hypothesis threshold θ₀
alternative_rate	float	Design alternative θ₁
decision_threshold	float	Posterior probability threshold γ (default: 0.95)
target_power	float	Target power (default: 0.80)

Key Response Fields

recommended_n — Recommended sample size
operating_characteristics — Type I error and power at recommended n
constraints_met — Whether design meets error constraints
power_curve — Power across range of true rates

View full API documentation →