Jump to comment:

• Study rigor may not impact finding of neuroablation superiority over deep brain stimulation for obsessive-compulsive disorder
  Austin Y. Feng, Uma V. Mahajan, Kevin K. Kumar, Summer S. Han and Casey H. Halpern
  Published on: 29 April 2020

• Published on: 29 April 2020

  Study rigor may not impact finding of neuroablation superiority over deep brain stimulation for obsessive-compulsive disorder

  • Austin Y. Feng, Medical Student Stanford University
  • Other Contributors:
    • Uma V. Mahajan, Medical Student
    • Kevin K. Kumar, Neurosurgeon
    • Summer S. Han, Statistician
    • Casey H. Halpern, Neurosurgeon

  Introduction:

  Obsessive-compulsive disorder (OCD) is a neuropsychiatric disease characterized by distressing thoughts or urges that often require repetitive behaviors to suppress. OCD affects 2-3% of the general population and can have debilitating effects on normal functioning.[1] While most cases of OCD can be addressed through psychotherapy and/or medication, about 10% remain refractory, requiring neurosurgical intervention, such as neuroablation (ABL) or deep brain stimulation (DBS). These options possess their own respective advantages and disadvantages. ABL lacks the hardware concerns of DBS (e.g. device failure, battery replacement, etc.) and may be incisionless (e.g. stereotactic radiosurgery). Alternatively, DBS is non-lesional, and stimulation parameters can be titrated. While both ABL and DBS appear to be effective for refractory OCD, there is no clear consensus on their relative superiority/non-inferiority.

  Our group previously sought to address this question by comparing the two treatments’ relative utility. [1] Using a random-effects, inverse-variance weighted meta-analysis of 56 studies, utility was calculated from Yale-Brown Obsessive Compulsive Scale (Y-BOCS) scores and adverse event (AE) incidence. In our analysis, no significant differences were found between stereotactic radiosurgery and radiofrequency ablation, so their studies were combined and all considered under ABL. Ultimately, ABL yielded a significantly greater utility compared to...

  Show More

  Introduction:

  Obsessive-compulsive disorder (OCD) is a neuropsychiatric disease characterized by distressing thoughts or urges that often require repetitive behaviors to suppress. OCD affects 2-3% of the general population and can have debilitating effects on normal functioning.[1] While most cases of OCD can be addressed through psychotherapy and/or medication, about 10% remain refractory, requiring neurosurgical intervention, such as neuroablation (ABL) or deep brain stimulation (DBS). These options possess their own respective advantages and disadvantages. ABL lacks the hardware concerns of DBS (e.g. device failure, battery replacement, etc.) and may be incisionless (e.g. stereotactic radiosurgery). Alternatively, DBS is non-lesional, and stimulation parameters can be titrated. While both ABL and DBS appear to be effective for refractory OCD, there is no clear consensus on their relative superiority/non-inferiority.

  Our group previously sought to address this question by comparing the two treatments’ relative utility. [1] Using a random-effects, inverse-variance weighted meta-analysis of 56 studies, utility was calculated from Yale-Brown Obsessive Compulsive Scale (Y-BOCS) scores and adverse event (AE) incidence. In our analysis, no significant differences were found between stereotactic radiosurgery and radiofrequency ablation, so their studies were combined and all considered under ABL. Ultimately, ABL yielded a significantly greater utility compared to DBS (0.189±0.03 vs. 0.167±0.04, respectively; P<0.001). Due to perceived advantages of DBS over ABL, this result was surprising and inquiries were raised about the reliability of the analysis. A primary concern was whether study rigor could have impacted our finding. In this follow-up study, we performed an in-depth assessment of rigor of the ABL vs. DBS studies that previously met criteria for inclusion.

  Methods:

  Assessment of rigor requires the selection of a grading scale. However, as rigor scales are designed for a specific type of study, a single type of scale is incapable of properly assessing the variety of analyzed study types (e.g., RCT, cohort, and case series). To circumvent this issue, we decided to score only case series because they were the most common study type in both treatment groups. These studies were assessed with the Institute of Health Economics (IHE) quality appraisal checklist, which is a validated scale designed exclusively for case series.[2,3] These rigor scores were then used as covariates to a mixed-effects model of treatment against outcome (i.e. Y-BOCS score, adverse event (AE) incidence, and overall utility).

  Results:

  There were a total of 20 DBS and 17 ABL case series used in the meta-analysis; they represent 71.4% and 94.4%, respectively, of all studies in each treatment group. For overall number of patients, case series contribute 68.7% (125/182) and 97.7% (347/355) to DBS and ABL groups, respectively. In addition, they represent 40% (4/10) and 83.3% (10/12) of DBS and ABL studies that report AE incidence. DBS studies had significantly higher rigor scores than ABL studies (13.5/20 versus 11.0/20, respectively; p<0.001).

  Similar to the results of Kumar et al., ABL still imparts a significantly higher utility than DBS with rigor as covariate in the mixed-effects model of treatment (p<0.0001). AE incidence was also significantly lower in ABL case studies (p=0.0024). For Y-BOCS score, ABL case series report greater percent reduction compared to DBS studies, though it was only marginally significant (p=0.0609). When rigor added to the mixed-effects model of treatment against outcome, neither the treatment group nor the rigor scores were significant predictors of % Y-BOCS score reduction (p=0.2261 and 0.4179, respectively). Similarly, neither treatment group nor rigor was a significant predictor of AE incidence (p=0.2223 and 0.4602, respectively). When considered as the sole predictor, rigor was not found to be significant predictor of % Y-BOCS score reduction (p = 0.1115). However, higher rigor scores were predictive of higher AE incidence (p = 0.0064).

  Discussion:

  Rigor is an important consideration as it may lend insight to a study’s validity and robustness. One hypothesis of ABL’s superiority over DBS was that ABL studies were less rigorous, producing outcomes that may be less reflective of reality. As rigor was not a significant covariate for utility, it suggests that Kumar et al.’s original conclusion of ABL superiority is accurate. However, even though rigor is not a significant covariate of % Y-BOCS score reduction, it is a significant predictor for AE incidence. Higher rigor studies generally had a higher reported AE incidence.

  A notable limitation to this analysis is the exclusive use of case series. The mixed collection of study types precluded the perfect fit of a single scale to all studies. Since case series only represent a portion of all studies, our calculations slightly vary from those reported previously in Kumar et al. Nevertheless, case series represent the vast majority of studies and patients in both DBS and ABL groups. Though not a perfect facsimile, they are still highly representative. Regarding AE incidence, DBS case series constitute only 40% of overall DBS studies with AE reports. Yet, they were sufficient in establishing the significant association of high rigor and AE incidence. Given that the remaining DBS studies likely have even higher rigor (i.e. RCTs and cohort studies), an even stronger association with their inclusion is not an unreasonable inference. Furthermore, although treatment types were no longer significant predictors, this may be a consequence of working with fewer studies and overfitting data.

  While study rigor was not a significant covariate for the current utility calculation, it is nevertheless an important consideration for similar meta-analyses, especially when assessing older and newer technologies. As the field of functional neurosurgery advances, there will be a greater need for similar comparisons to evaluate and guide clinical decision-making and adoption of novel techniques. Developments in DBS technologies can all plausibly improve DBS utility for OCD, necessitating further studies. Despite its limitations as an imperfect measure, study rigor may provide valuable insight into the reliability of reported data.

  Bibliography:

  1. Kumar KK, Appelboom G, Lamsam L, et al. Comparative effectiveness of neuroablation and deep brain stimulation for treatment-resistant obsessive-compulsive disorder: A meta-analytic study. J Neurol Neurosurg Psychiatry 2019;90:469–73. doi:10.1136/jnnp-2018-319318
  2. Institute of Health Economics. Quality Appraisal Checklist for Case Series Studies. 2014;:1–2.
  3. Guo B, Moga C, Harstall C, et al. A principal component analysis is conducted for a case series quality appraisal checklist. J Clin Epidemiol 2016;69:199–207.e2. doi:10.1016/j.jclinepi.2015.07.010

  Show Less

  Conflict of Interest:
  None declared.

  • Back to top