Blog

More signal, less noise: using ANCOVA analysis in RCTs

23 September 2026

Cher Li, Richard Kneller, Anwar Adem

Experimental research

When an RCT measures the same outcome before and after an intervention, analysts often reach for difference-in-differences. Our experience reanalysing two business-support trials shows why ANCOVA is usually the sharper default: it uses baseline information without forcing an inefficient one-for-one adjustment.

Randomisation is only the first analytical choice

Random assignment does the heavy work of causal identification. On average, the treatment and control groups are comparable, so their post-treatment difference identifies the effect of being offered the intervention. But randomisation does not make every valid estimator equally precise. Once baseline outcomes are available, the way we use them can materially affect how much signal we recover from the same experiment.

We encountered this directly in our field experiments on peer benchmarking and business websites (Adem et al., 2026, forthcoming in Research Policy). Across two randomised controlled trials involving approximately 7,000 UK firms in distilling and retail, businesses were randomly assigned either to receive a bespoke report comparing their website performance with industry peers or to remain in an uncontacted control group. 

Our original paper analysed the outcomes using difference-in-differences (DiD), an approach also described in our earlier IGL blog on the experiment. In a revised version, we followed the approach recommended in IGL’s quantitative analysis guide and re-estimated the results using analysis of covariance (ANCOVA). This was not a new experiment or a new outcome: it was a more efficient way to analyse the same randomised comparison.

What ANCOVA does differently

A simple DiD analysis first calculates each firm’s change between baseline and follow-up, then compares that change across the treatment and control groups. This sounds natural, but it implicitly gives the baseline outcome a coefficient of exactly one. In other words, a one-unit baseline difference is subtracted in full from the follow-up outcome, whether or not baseline performance predicts follow-up performance one-for-one.

ANCOVA instead models the follow-up outcome directly and includes the baseline value as a predictor:

Follow-up outcome = a + b(Treatment) + c(Baseline outcome) + other pre-treatment controls + error

The treatment coefficient, b, is the intention-to-treat effect. The crucial difference is that the data estimate c, the amount of baseline adjustment that is actually useful. When outcomes are noisy or only moderately correlated over time, DiD can overcorrect. ANCOVA absorbs the predictable part of follow-up variation and leaves less residual noise around the treatment estimate.

This is the core result in McKenzie’s 2012 analysis: with one baseline and one follow-up, the relative efficiency gain from ANCOVA is greatest when the outcome is weakly autocorrelated. If the baseline–follow-up correlation were 0.25, DiD would need about 60 per cent more observations to achieve the same power under the paper’s assumptions. A 2025 practical review likewise identifies the lagged outcome in ANCOVA as usually the most valuable control for improving power.

What changed in our results

The revised analysis did not simply turn null results into positive ones. It produced a more discriminating account of where benchmarking worked, where it did not and for whom. By interacting treatment assignment with baseline performance, we found a catch-up pattern not visible when using DiD: firms with slower websites at baseline responded more strongly. Among retailers, average desktop effects remained absent, but improvements emerged for initially underperforming mobile sites. ANCOVA did not manufacture these effects; it made better use of pre-treatment information and provided a clearer, more interpretable test of heterogeneity.

Practical lessons for RCT teams

• Make ANCOVA the default when a comparable baseline outcome exists. Regress the follow-up outcome on treatment assignment, the baseline outcome and any variables used for stratification.
• Keep firms with missing baseline values where appropriate. A pre-specified strategy can impute a neutral value for the baseline covariate and add a missing-baseline indicator, avoiding the automatic loss of units that DiD cannot difference.

The bottom line

RCTs are costly, and statistical power is a scarce resource. Researchers and delivery organisations should use baseline information efficiently. In our trials, ANCOVA reduced residual noise, strengthened the analysis of who responded and helped separate genuine null results from effects that were previously estimated too imprecisely.

With a credible baseline outcome, pre-specify ANCOVA and report it transparently. It cannot rescue a weak experiment, but it can sharpen a good one.