Fragility Index for Randomized Controlled Trials

An interesting question – how ‘fragile’ are the results of a randomized controlled trial (RCT)?

RCTs are gold-standard for determining the efficacy of a treatment. RCTs randomly assigned patients to an intervention arm or to a placebo (or standard of care or active treatment) arm. Because patients are randomly assigned, we expect that all patient characteristics (genetic, environmental etc.) would be balanced in the two groups. RCTs are often published with significant results – small RCTs without significant results are generally either not published or published in low-tier journals and don’t get prominence

Even when we randomly assign patients to a treatment arm or control arm, there is always a possibility that the two arm may be imbalanced by chance on measured or unmeasured variables. Further, it is also possible that just by chance one group may have statistically significant result than the other group without any true benefit (or harm) of the treatment. While we generally use a p-value of less than 0.05 (which says that there is a 1 in 20 probability that the results are observed due to chance only – not the right definition but its simple!), there remains a small chance that results will be not significant.

In comes ‘Fragility Index’, a measure of determining the robustness of results. Fragility index is the minimum number of patients who must be moved from the nonevent group to the event group to turn a significant result nonsignificant. The fragility index is an easy to calculate index that provides an intuitive way to understand the precision of trial results. Knowing the fragility index and comparing it to the number of patients lost to follow-up can help to understand the uncertainty in evidence even when a study has positive results.

Khan et al, examined cardiovascular clinical trials published between 2007-17 in major Cardiology journals. All trials were large with >500 patient enrollment. Among the 123 RCTs the median fragility index was 13.In almost 1/3rd of trials, the number of patients lost to follow-up was more than the fragility index.

In another study Gaudino et al examined the clinical trials used to support evidence in cardiology guidelines and determined the fragility index. They found that more than a quarter of RCTs supporting current guidelines on myocardial revascularization have a fragility index 3 or lower. Further over 40% of trials had a fragility index which was lower than the number of patients lost to follow-up.