Health Informatics Virtual Journal Club: “The Magic of Randomization”

For those who are working at home or can’t get into university/college for health informatics studies, I thought I would share some papers for discussion as a “virtual journal club” on the forum.

The first paper is called “The Magic of Randomization versus the Myth of Real-World Evidence” by Rory Collins, Louise Bowman, Martin Landray and Richard Peto from the University of Oxford and published in the New England Journal of Medicine as a “Sounding Board” article.

Why is this paper relevant to health informatics? Traditionally, RCTs have been conducted separately from hospital and community information systems with each study setting up their own methods for recruiting patients, managing allocation of treatment vs controls and collecting outcome data. Over time this has become increasingly expensive – the paper says that the contract research organisation industry that conducts many trials has grown from $2 billion in the early 1990s to over $40 billion in 2019. Some have more recently been advocating for using “real world evidence” from routine health information systems as an alternative to RCTs for collecting evidence about how well treatments work.

The authors position is that this type of observational evidence is good for monitoring for rare adverse events and for detecting large beneficial effects of treatments that would not otherwise have been expected (such as using insulin for controlling diabetic ketoacidosis), but observational studies are prone to bias and could produce misleading associations that a non-causal or mistakenly null. For detecting smaller but significant effects and causal associations, RCTs remain the gold standard. RCTs give balanced groups of patients with respect to known and unknown risk factors, can show whether or not the treatment caused the effect (rather than just being associated with it), and the proportional effects of treatments revealed through RCTs can be generalised to other populations so can help clinical decision making in general.

So, if observational studies using “real world evidence” are problematic, how do we conduct useful research when the costs of doing so are becoming prohibitive? The authors propose several technological and policy solutions to reducing costs and complexities of running RCTs including:

• Appropriate trial guidelines based on scientific principals
• Enhanced recruitment using Electronic Health Records (EHRs) and avoiding unduly restrictive inclusion and exclusion criteria
• Promote better adherence through interactive electronic case-report forms and centralised monitoring
• Use EHRs for reducing loss to follow-up and smartphones and digital sensors to extend the range of outcomes measured

These suggestions should be developed into comprehensive guidelines that are based on scientific principles that can reduce costs and unnecessary complexity and support the use of RCTs for non-commercial purposes such as evaluating existing treatments.