Confidence interval statistical significance
- CONFIDENCE INTERVAL STATISTICAL SIGNIFICANCE FULL
- CONFIDENCE INTERVAL STATISTICAL SIGNIFICANCE TRIAL
9 The confidence interval can be interpreted as providing a range of treatment effects supported by the study. Focus therefore shifted to the reporting of confidence intervals. However, neither approach acknowledges the importance of the size of any treatment effect. On the other hand, Fisher argued for an approach based on a continuum with no set threshold, also arguing for the consideration of other contextual factors. 2 One approach, advocated by Neyman–Pearson, uses an objective but arbitrary cut‐point (usually a P value of 0.05) for statistical significance. The imperfect nature of any approach to hypothesis testing is now widely recognised. 7 The second study investigated whether lopinavir–ritonavir provides any treatment benefit in patients with severe coronavirus disease 2019, reporting non‐significant findings which it interpreted as “no difference” or “no benefit”.
CONFIDENCE INTERVAL STATISTICAL SIGNIFICANCE TRIAL
The trial reported a non‐significant finding which it interpreted as “no significant difference”. 7, 8 The first study aimed to determine whether the efficacy of the N95 respirator in controlled settings could be maintained in real life, where compliance may be suboptimal. We illustrate these recommendations using two topical case studies (online Supporting Information). While we advocate for a more holistic interpretation considering contextual factors, we urge transparency in these arguments.
CONFIDENCE INTERVAL STATISTICAL SIGNIFICANCE FULL
To this end, we provide recommendations for the interpretation of the primary outcome result, where we urge interpretation of the full range of the confidence interval and its overlap with effect sizes considered to be clinically important. Here we provide a practical guide, bridging the gap between statistical philosophy and the desire to draw conclusive findings from most trials ( Box 1). 5, 6 Despite availability of publications addressing the statistical philosophy underpinning hypothesis testing, there is a dearth of practical guidelines for investigators, reviewers and editors in correct interpretation of findings from randomised controlled trials. It is this paradox that led to recent campaigns demanding appropriate interpretation of confidence intervals. Thus, while researchers are abiding by reporting guidelines and including confidence intervals, these are rarely fully interpreted in the conclusions (ie, researchers are not abiding by the philosophy underpinning the reason for the guidelines). Second, in some trials it might be correct to conclude that a treatment is effective (or harmful), despite the non‐statistically significant result yet researchers persist in unhelpful language such as “not statistically significant”. 4 First, many trials are interpreting absence of evidence as evidence of no effect, concluding an intervention is ineffective when in fact the results suggest its effectiveness is uncertain. 3 Despite this, misinterpretation stubbornly persists. 2 Reviews of contemporary trials show that researchers mostly adhere to this advice. 1 Confidence intervals are informative as they show the likely range of effect sizes supported by the findings, whereas P values dichotomise the findings based on statistical significance at an arbitrary cut‐off. Guidelines for reporting results from randomised trials have long underscored the importance of confidence intervals.