Evidence for Change – When to Adopt a Change in Practice
Neurosurgery, like all of medicine, is in constant flux. New treatments, techniques, drugs and devices are offered in every issue of our journals and at every national meeting. In most cases, there is a persuasive case that the new treatment offers better outcomes. But when should we actually change our surgical practice? Should a persuasive case series or meta-analysis convince us, or do we really need to see a randomized trial? How much evidence is enough?
Treating Chronic Subdural Hematomas: Time for Change?
As an example, let’s consider the treatment of chronic subdural hematoma using middle meningeal artery (MMA) embolization. First described about 20 years ago in single case reports, embolization of the MMA ipsilateral to a convexity chronic subdural hematoma (CSDH) conceptually offers to interrupt the fragile neovasculature that supplies intra-hematomal microhemorrhages that can lead to expansion. Early reports of success in single cases or small case series have now been followed by several series that include comparison controls treated without MMA embolization, with improved outcomes. The largest such series to date1 compared 72 CSDH treated with MMA embolization, collected prospectively, to 469 historic controls from the same institution treated between 2003 and 2015. Treatment failure rate (a composite of residual hematoma > 10mm and/or need for surgical drainage) was 1.4 percent in MMA embolization patients, compared with 27.5 percent in historic controls (P = 0.001). A recent meta-analysis found two much smaller MMA embolization studies with similar results.2
Given promising evidence for a new treatment in nonrandomized comparisons, it is tempting to suggest that a randomized trial is unnecessary or even unethical. After all, studies have shown that nonrandomized comparisons can reach a similar conclusion to randomized clinical trials (RCTs), particularly when the methods used mimic those of an RCT.3 Is a demand for more evidence gilding the lily, akin to designing an RCT investigating the use of penicillin for pneumonia or the use of parachutes when jumping out of airplanes?4
It is useful to remember that the many types of bias that can affect nonrandomized studies can add up to an impressive apparent “treatment effect” in favor of a new therapy that doesn’t withstand the more stringent test of an RCT. In 2008, a one study reported that anomalous extracranial venous outflow was significantly more common in patients with relapsing/remitting multiple sclerosis (MS) (odds ratio 43, P < 0.0001) and that these stenoses were associated with significant pressure gradients.5 In an open-label series of 65 MS patients treated with percutaneous angioplasty of such stenoses, pressure gradients were relieved, MRI lesions were reduced (12 percent vs. 50 percent, P < 0.0001), freedom from relapse doubled, and quality of life gains were recorded, all with statistical significance.6 Yet in the follow-up RCT, angioplasty was not associated with a significant reduction in MRI lesion frequency, relapse rate or improvement in a composite functional measure.7 The investigators concluded that angioplasty could not be recommended.
In considering outcome after MMA embolization for CSDH, some types of bias could clearly affect even a well-conducted, carefully designed comparison against historical controls.
- The assumption that no improvement in CSDH outcome would occur during a study period as long as 15 years may not be valid.
- The use of surgical adjuncts such as subdural drains8, corticosteroids and tranexamic acid would be impossible to standardize retrospectively.
- The rate of second surgical drainage procedures after initial surgical treatment in the retrospective study (19 percent) was about twice that seen in the subdural drain arm of an RCT of burr hole CSDH drainage8; as well as in a NSQIP registry study and among Medicare beneficiaries nationally.9
- Patients with minimally symptomatic lesions who are not admitted to hospital might be incompletely captured in a retrospective control group because of coding irregularities (“ascertainment” or “staging bias”).
- Criteria for salvage surgery in a retrospective study would be impossible to standardize and might be subtly different in a prospective, open label group treated with the new therapy; use of corticosteroids could be standardized in this group prospectively.
- Important endpoints such as neurological deterioration rates during treatment, length of hospital stay, adverse discharge disposition, quality of life and other patient-centered endpoints, and cost to the health care system remain largely unexplored in retrospective studies.
The power of a positive RCT to convince neurosurgical practitioners, healthcare payors, and – in cases with adverse outcomes – the legal system of the value of a novel treatment should not be discounted. CSDH is a common condition; with patient and physician equipoise, accruing sufficient patients to a reasonably powered multicenter RCT should be feasible. In this instance, awaiting the results of an RCT before accepting a new standard of care for CSDH seems like a reasonable choice. The neurosurgical community should support such a trial when it becomes available.
ReferencesClick to View All
The New England Master Class - Anterior Skull Base Surgery
May 20-21, 2019; Boston
6th Annual Meeting on Neurosurgery and Neurologica
May 22-23, 2019; London
INS 14th World Congress
May 25-30, 2019; Sydney
12th Annual Cervical Spine Research Society Hands-on Cadaver Course
May 30-June 1, 2019; St. Louis
Brain Tumor Biotech Summit
June 7, 2019; New York
Be the first to reply using the above form.