Objective: To study the reliability of the diagnosis of blepharospasm, oromandibular dystonia, cervical dystonia, and writer's cramp among neurologists.
Methods: 12 patients with adult onset focal segmental dystonia were videotaped in a standardised way. The tape was sent to six neurologists who are involved in clinical practice without a specific interest in movement disorders (general neurologists), and to four neurologists expert in movement disorders. The observers had to recognise whether the patients were affected by dystonia and to distinguish among blepharospasm, oromandibular dystonia, cervical dystonia, and writer's cramp. Interobserver reliability was assessed by κ statistics, and the degree of agreement was classified according to the Landis classification.
Results: The 10 neurologists reached slight to moderate agreement on the diagnosis of these four disorders. When the observers were subdivided according with their professional experience in the field, a moderate to perfect agreement on the diagnosis was achieved by specialists in movement disorders, and a fair to moderate agreement by the general neurologists.
Conclusions: Neurologists may have different ability to recognise adult onset focal dystonia, depending on their experience and on the type of dystonia.
Statistics from Altmetric.com
The most common primary dystonia is focal with adult onset, presenting as blepharospasm, oromandibular dystonia, cervical dystonia, or writer's cramp.1 Adult onset dystonia may remain focal or it may spread to involve other muscles.1 The ability to identify dystonia at different body sites, either in the same or in different individuals, is essential in assessing the frequency of dystonic involvement of different body regions or in describing patterns of spread for identification of long term prognostic factors.
The diagnosis of dystonia and the distinction between the different focal types is purely clinical and can be affected by several factors, including the circumstances of the examination, the psychological status of the patient, and the attitude of the observer. It is therefore open to bias. Because of the absence of a gold standard (a diagnostic test or biomarker), the validity of the diagnosis of the different types of focal dystonia cannot be assessed. Reliability refers to the degree of reproducibility of observations and is the only way of assessing the quality of diagnosis when validity is not measurable.2
No study has dealt with the reliability of the diagnosis of dystonia at different sites, or with factors that may affect the diagnosis. We designed the present study to evaluate the degree of agreement on the diagnosis of dystonia at different sites in the body among neurologists from different centres. This is an important issue because clinical and epidemiological studies of a relatively rare disorder such as dystonia may require large patient samples and thus collaboration among different institutions.
Twelve patients with adult onset focal or segmental dystonia affecting the cranial cervical region or the upper limb were selected from among outpatients attending the movement disorder clinic of the department of neurological and psychiatric sciences of the University of Bari, Italy. Their demographic and clinical features are given in table 1. There were four diagnostic categories (blepharospasm, oromandibular dystonia, cervical dystonia, and writer's cramp), and at least four cases in each category. Diagnoses were made by the senior neurologist of the movement disorder clinic (GD). Following informed consent, video recordings of the patients were made by a staff neurologist who did not participate in the study. Each video segment lasted three to five minutes and included a short neurological examination and the following manoeuvres that may trigger prominent spasm in otherwise asymptomatic subjects: opening and closing the eyes or mouth, looking upward and downward, speaking, walking, reading, and writing.
The tape was sent to 10 neurologists working in general hospitals in the Puglia region in the south of Italy. All observers (nine male, one female, mean (SD) age 47 years) were asked to report their personal data, their professional experience in neurology, and their familiarity with movement disorders. According to their previous experience, the observers were subdivided into two groups: the first included six general neurologists without a specific interest in movement disorders; the second included four neurologists with a special interest in movement disorders. The general neurologists and the specialists in movement disorders had completed their training in neurology 14 (2) years and 13 (2) years previously, respectively (p = 0.24).
The examiners were told that all the subjects included in the videos were patients with adult onset focal/segmental dystonia and there were no healthy controls or subjects with movement disorders other than dystonia. The examiners had to decide whether the different types of dystonia—including blepharospasm, oromandibular dystonia, cervical dystonia, and writer's cramp—were present, alone or in combination, in each patient examined, regardless of the relative severity. There was no prior discussion among examiners about the criteria for diagnosing the different types of dystonia. Diagnoses were made independently, and the raters remained unaware of each other's assessment until the end of the study. Intraobserver reliability could not be tested because most examiners were not available to retest within a few weeks after the first assignment.
Interobserver reliability on the diagnoses was assessed by the κ statistic which measures the level of agreement after excluding chance expected agreement.3 A standard statistical package (STATA 7) was used to calculate κ values, their 95% confidence intervals, and p values testing the hypothesis that the observed agreement was no better than chance. We compared the κ values with the Landis classification,3 which defines six levels of reliability: < 0.0, poor; 0.0 to 0.2, slight; 0.21 to 0.40, fair; 0.41 to 0.60, moderate; 0.61 to 0.80, substantial; and 0.81 to 1.0, almost perfect.
According with the Landis classification,3 the overall chance corrected level of agreement among the 10 neurologists (table 2) was moderate for blepharospasm (κ = 0.51) and cervical dystonia (κ = 0.52), fair for writer's cramp (κ = 0.29), and slight for oromandibular dystonia (κ = 0.20). As indicated by the 95% confidence intervals (table 2), the differences between blepharospasm/cervical dystonia and oromandibular dystonia/writer's cramp were significant.
In order to investigate the relation between reliability and expertise, observers were subdivided according with their professional experience in movement disorders (table 2). Among general neurologists, agreement was moderate for blepharospasm and cervical dystonia, and fair for oromandibular dystonia and writer's cramp. As indicated by the 95% confidence intervals, the differences in agreement between blepharospasm/cervical dystonia and oromandibular dystonia/writer's cramp were significant. Among specialists in movement disorders, agreement was perfect for the diagnosis of blepharospasm, substantial for cervical dystonia, and moderate for oromandibular dystonia and writer's cramp. As indicated by the 95% confidence intervals, the differences in agreement between blepharospasm and oromandibular dystonia/writer's cramp were significant, whereas the cervical dystonia estimate overlapped the other categories.
We were aware that disagreement on the diagnosis of dystonia among examiners can arise from several sources, including the status of the patient and the circumstances of the clinical examination, tests done, treatment given, and the attitude of the observer. Our study design focused on the amount of disagreement among observers. Thus assessments were made from video recordings, there was no pretest training period, and discrepancies were not discussed at any time during the study. Although the diagnostic criteria for each type of dystonia were tested on a limited number of patients, there was sufficient power to detect significant differences.
According with the Landis classification,3 the 10 Italian neurologists reached a slight to moderate interobserver agreement on the diagnosis of blepharospasm, cervical dystonia, oromandibular dystonia, and writer's cramp. When observers were subdivided according with their professional experience in the field of movement disorders, a moderate to perfect agreement on the diagnosis of these dystonias was achieved by the specialists in movement disorders, and a fair to moderate agreement by the general neurologists.
Compared with general neurologists, specialists in movement disorders showed a better interobserver agreement on the diagnosis of all the focal dystonias examined. Both categories of observer, however, reached the highest level of agreement on blepharospasm and cervical dystonia, and the lowest level on oromandibular dystonia and writer's cramp. The worse recognition of the latter categories may be for various reasons. First, oromandibular dystonia and writer's cramp are probably seen less frequently than blepharospasm and cervical dystonia,4–9 even in neurological settings.8 Thus general neurologists may be less familiar with these disorders. Second, oromandibular dystonia and writer's cramp may result in various different movements or postures: for example, oromandibular dystonia may be characterised by jaw closing, opening, or deviation. In contrast, blepharospasm is characterised by involuntary contractions of the orbicularis oculi muscles inducing eye closure alone. Finally, there are certain neurological and non-neurological conditions potentially mimicking oromandibular dystonia and writer's cramp, such as bruxism, dental problems, and temporomandibular joint dysfunction in the case of oromandibular dystonia, or carpal tunnel and other nerve entrapment syndromes, tennis elbow, apraxia, and chronic fatigue syndrome in the case of writer's cramp.10
The list of disorders involved in the differential diagnosis of blepharospasm and cervical dystonia is shorter. It includes so called apraxia of eyelid opening and blepharitis/keratoconjunctivitis in the case of bilateral blepharospasm,11,12 and cervical spine abnormalities which usually induce fixed abnormal head postures in the case of cervical dystonia.13
Regardless of the explanation, our observations suggest that the localisation of dystonia may be a further source of variability in the assessment of patients affected by different types of focal dystonia or by dystonia at multiple body sites.
The results of interobserver agreement cannot be considered satisfactory, particularly because variability caused by the psychological status of the patient and the test circumstances was not taken into account, healthy controls or subjects with movement disorders other than dystonia were not included in our series, and the examiners knew that they were dealing with dystonia. Thus our study design did not test what happens in actual clinical practice, and this might have decreased the threshold for making the correct diagnosis. Nevertheless, the observation that neurologists may have different abilities to recognise these dystonias depending on their experience and the site of the dystonia may have important implications. For example, it raises the possibility that a large proportion of the differences in prevalence estimates of blepharospasm, cervical dystonia, oromandibular dystonia, and writer's cramp across available studies (which were based, at least in part, on existing diagnosis verified by dystonia specialists)4–9 may be the result of differences in the education of neurologists about dystonia. It is worth noting that our neurologists reached the highest level of agreement on blepharospasm and cervical dystonia, which are considered to be the most common forms of dystonia.4–9
We ignored the consistency of the neurologists' ability to diagnose dystonia because intraobserver reliability could not be tested. Lack of this information, however, probably does not affect the relevance of the data on interobserver agreement reported here, or their potential implications.
Our findings suggest that standardising the clinical diagnostic criteria for each form of dystonia among neurologists from different institutions should be a preliminary step in epidemiological and clinical studies in order to achieve a satisfactory agreement on the diagnosis of dystonia at different body sites.
Competing interests: none declared
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.