E Values and Incidence Density Sampling

Abstract

To the Editor: The E value1 has quickly become a relatively popular approach for assessing the robustness of exposure–outcome associations to potential unmeasured confounding. While the E value is not a fully adequate substitute for a more extensive sensitivity or bias analysis, it can be an easy-to-use alternative when either investigators or journal editors are unwilling to conduct or to report or document a more extensive sensitivity analysis.2 The E value is derived from sensitivity analysis techniques that make reference to hypothetical associations on the risk ratio scale between the unmeasured confounder and the outcome, conditional on the exposure and measured covariates (RRUD) and also between the exposure and the unmeasured confounder conditional on the measured covariates (RREU).1,3 When odds ratios derived from logistic regression models are employed, or when hazard ratios derived from proportional hazards models are used, and the outcome is rare, then the E value calculations can be employed directly with the odds ratios or hazard ratios taken as close approximations to risk ratios.1,3 When odds ratios or hazard ratios are employed but the outcome is relatively common at the end of follow-up (e.g., >15%), the standard E value calculations for risk ratios cannot be used directly since the odds ratio or hazard ratio will no longer approximate the risk ratio. In these cases, approximate conversions can be employed4,5 to derive approximate risk ratios from odds ratios or hazards ratios and from these one can obtain approximate E values.1 Software and online websites to carry out these approximations and calculations directly are available.6 Questions have arisen as to whether incidence density sampling within a case–control study alters these considerations. In a case–control study with incidence density sampling, one will obtain estimates of the hazard ratio directly, even without the assumption of a rare outcome.7 However, without the rare outcome assumption, the hazard ratio will not itself approximate a risk ratio if the outcome is not relatively rare during the cumulative follow-up being considered. For outcomes that are not rare during the total follow-up being considered, one thus cannot apply the risk ratio E value calculations directly, but rather one must still use the approximate conversions from hazard ratios to risk ratios, even in a case–control study with incidence density sampling. However, these considerations do depend critically on the total follow-up period being considered. Whether the “rare outcome” assumption is applicable or not depends on the time-frame to which one desires to extrapolate the results. If the time-frame is very short, such as 1 day or 1 month, then the incidence of the outcome for that time-horizon is likely to be low, the “rare outcome” assumption would be applicable, the hazard ratio will closely approximate the risk ratio, and one can proceed directly with the risk ratio E value calculations. However, using the same hazard ratio from the same study, if one is interested in a longer follow-up period, say 1 year, then if the cumulative incidence of the outcome is say >15% during that year, then the rare outcome assumption is not applicable, and one must use the approximate conversions from hazard ratios to risk ratios before employing the risk ratio E value formula. See the Table for a summary. The E values will thus be different in these two scenarios, depending on whether a 1-day follow-up or a 1-year follow-up is being considered. One will obtain a larger E value for the 1-day follow-up period than for the 1-year follow-up period. But this is reasonable, because the effect sizes on the risk ratio scale for the effect of the unmeasured confounder on the outcome occurring within a single day may be very different than those for the outcome occurring over the course of a much longer period of time. Table 1. - A Summary of the Need for Approximate Conversions in Case–Control Studies with Incidence Density Sampling when Calculating E values Estimand from Study Calculation Required for Input Into the E Value Formula RR RR HR, with rare outcome in cumulative follow-up being considered RR ≈ HR HR, with common outcome, e.g., >15%, in cumulative follow-up being considered RR ≈ (1 − 0.5sqrt(HR))/(1 − 0.5sqrt(1/HR)) The final formula for the approximate risk ratio is derived in reference 4.HR indicates hazard ratio; RR, risk ratio. For example, if one were examining associations between extreme caloric intake and myocardial infarction (MI) among those strongly genetically predisposed to MI and the potential unmeasured confounder were vigorous exercise, then vigorous exercise for someone who is generally lethargic can dramatically increase the probability of MI over a 24-hour period.8 Thus a large E value, if only a 24-hour follow-up is being considered, may not provide much evidence for robustness to confounding over this time-interval because the unmeasured confounder may have very large effects over such a short time frame. In contrast, if a 10-year period is being considered, unmeasured vigorous exercise may not generate risk ratios of the same magnitude (and will also generally be in the protective, rather than causative, direction during this longer time-frame). Thus, a more modest E value over a 10-year period may in fact provide more substantial evidence that an association is robust to unmeasured confounding. The time-frame under consideration does matter, both for the necessity of the approximate conversions for E values, and for the interpretation of the plausible magnitude of the parameters for the unmeasured confounder associations. When the approximate conversion is required, because the outcomes are not rare over the cumulative follow-up being considered, the application of this conversion beforehand is important, and results will be misleading if it is not applied. For example, if in a case–control study with incidence density sampling, one obtained a hazard ratio of 1.3 and the conversion were not applied, this would give rise to an (incorrect and exaggerated) E value of 1.92. However, if a 10-year follow-up is being considered and the outcome is not rare during this follow-up, then, after conversion, the approximate E value becomes 1.69, which is somewhat lower. More generally, if the outcome probabilities during the cumulative follow-up considered are between 0.2 and 0.8, the untransformed hazard ratio can be biased up to 80% for the risk ratio, whereas, after the approximate conversion, the quantity will be biased for the risk ratio by at most 16%.4 In summary, even in case–control studies with incidence density sampling, if the cumulative incidence of outcome is not low under the follow-up period being considered, one must use the approximate conversions from hazard ratios to risk ratios before calculating E values (which again software will do automatically6). Whether the cumulative incidence is relatively low or not, and thus whether the approximate conversions for the E values are necessary or not, depends on the follow-up period being considered, but this is reasonable because the strengths of unmeasured confounding associations likewise will vary by the length of follow-up time being considered.

Topics & Concepts