Effect size quantification for interrupted time series analysis: implementation in R and analysis for Covid-19 research
Yael Travis-Lumer, Yair Goldberg, Stephen Z. Levine
Abstract
BACKGROUND: Interrupted time series (ITS) analysis is a time series regression model that aims to evaluate the effect of an intervention on an outcome of interest. ITS analysis is a quasi-experimental study design instrumental in situations where natural experiments occur, gaining popularity, particularly due to the Covid-19 pandemic. However, challenges, including the lack of a control group, have impeded the quantification of the effect size in ITS. The current paper proposes a method and develops a user-friendly R package to quantify the effect size of an ITS regression model for continuous and count outcomes, with or without seasonal adjustment. RESULTS: The effect size presented in this work, together with its corresponding 95% confidence interval (CI) and P-value, is based on the ITS model-based fitted values and the predicted counterfactual (the exposed period had the intervention not occurred) values. A user-friendly R package to fit an ITS and estimate the effect size was developed and accompanies this paper. To illustrate, we implemented a nation population-based ITS study from January 2001 to May 2021 covering the all-cause mortality of Israel (n = 9,350 thousand) to quantify the effect size of Covid-19 exposure on mortality rates. In the period unexposed to the Covid-19 pandemic, the mortality rate decreased over time and was expected to continue decreasing had Covid-19 not occurred. In contrast, the period exposed to the Covid-19 pandemic was associated with an increased all-cause mortality rate (relative risk = 1.11, 95% CI = 1.04, 1.18, P < 0.001). CONCLUSION: For the first time, the effect size in ITS: was quantified, can be estimated by end-users with an R package we developed, and was demonstrated with data showing an increase in mortality following the Covid-19 pandemic. ITS effect size reporting can assist public health policy makers in assessing the magnitude of the entire intervention effect using a single, readily understood measure.