Article Text

PDF

Validating a screening questionnaire for parkinsonism in Australia

Abstract

Parkinson's disease is a common neurodegenerative disorder in elderly people. Epidemiological studies of the disease can be labour intensive. A two phase design including a screening questionnaire as the first phase has become a popular method in prevalence studies of Parkinson's disease. Such a design has many advantages including less work for assessing physicians and enhanced recruitment of people to be screened. However, its wider application may be questioned because validation has been limited to samples that are drawn from hospitals (or clinics) and may be inappropriate for a community setting. This study assesses whether validating screening questionnaire by using a hospital sample yields the same result as a community based sample. Furthermore, it seeks to establish whether the screening instrument can be simplified to involve less questions. The findings show that some of the questions used in the screening phase yield different responses when comparing a hospital group with a community group. This study also provides a simplified model of questions that may be relevant for screening in the community setting.

  • Parkinson's disease
  • screening questionnaire
  • validation
  • Australia

Statistics from Altmetric.com

Parkinson's disease is one of the two most common neurodegenerative disorders in elderly people. Most epidemiological studies suggest that its prevalence is much lower in China, Japan, and Africa but higher in the United States, Europe, and South America (Zhang and Roman1). Recent studies have used a two phase design. In phase 1, participants are screened using a symptom based questionnaire and in phase 2, those who screened positive are examined to confirm the diagnosis. This design has advantages including less work for the examining physicians and capacity to screen more people. Screening questions can range from very brief (for example,Tisonet al 2) using only two questions to very detailed (for example, Andersonet al 3) involving 13 questions. Currently, there is no consensus as to whether the differences in screening questionnaires are relevant to the validity of the studies. More importantly, most of the screening questionnaires were validated using people already diagnosed as having Parkinson's disease and being invited into the hospital setting for clinical examination. This validation method may be flawed because those with Parkinson's disease in the community may have much earlier and more subtle signs than those already diagnosed. Therefore the performance of the screening questionnaire may be overestimated when applied to a community setting.

This paper reports the developmental process of a questionnaire specially designed for use within the community setting. It has incorporated earlier design strategies with a sample of patients already diagnosed as having Parkinson's disease plus control subjects. An extra step incorporated the same questionnaire being applied to a random sample from the community to test its performance and to compare the sensitivity and specificity of the two settings. The piloted questionnaire was then modified, on the basis of statistical factor analysis, to devise an efficient and effective tool that allows screening of parkinsonism within the community. This approach has optimised the number of questions used to produce a tool that is easily adapted to different settings.

Method

The screening questionnaire was designed using questions based on symptoms suggestive of Parkinson's disease. Although its purpose was to screen for the disease, other causes of parkinsonism would also be screened positive and differentiation relied on the examination phase. The questionnaire comprised 11 questions (appendix). Fifty six participants (16 with Parkinson's disease, two with essential tremor along with 38 controls of similar age range, and sex ratios) were invited into our hospital (called the hospital group hereafter). The age range was 53–86 (mean 70.6) years; male to female ratio was 1:1. All were questioned by volunteers trained for this exercise. Specialist geriatricians or neurologists, blinded to the results of the screening test or diagnosis, were then asked to examine the patients. Subsequent to the examination, the medications of the subjects were disclosed to the examiners. The questions and examined results were then analysed. The diagnostic criteria for Parkinson's disease were based on those by de Rijk et al 4—namely, at least two of the following: resting tremor, bradykinesia, or rigidity, in the absence of other apparent causes of parkinsonism. If a patient was already on levodopa and historically had a good response, this was regarded as strong supportive evidence provided that the assessment had been performed by a specialist geriatrician or neurologist in the past. Other causes for parkinsonism were ascertained according to neurological examination and clinical history.

In the second stage, a random sample of 136 people (called the community group hereafter) was drawn from a larger community sample of 507 people. These people were living in random census districts, chosen by computer. The people were approached door to door by trained volunteers and asked to consent to the study. They were tested with the same screening questionnaire by the same group of volunteers. Specialist geriatricians or neurologists who were again blinded to the results of the questionnaires or diagnosis were asked to examine the subjects and the subjects' medications were disclosed only after the assessment. The questions and examination results were again analysed.

Statistics and results

The participation rate of the community study was 75%. The age range was 52–90 (mean of 69.8) years; the male to female ratio was 1:1.2. Fourteen of the 136 people in the community group had parkinsonism. Five had Parkinson's disease previously diagnosed by neurologists and treated with levodopa with good responses. Five were possible new patients (including one who had both Parkinson's disease and essential tremor). Four had vascular causes of parkinsonism (two were previously diagnosed and two were new). No other possible causes of parkinsonism were found. One patient, diagnosed by his general practitioner as having possible Parkinson's disease, was found to have no supportive signs by two specialists and excluded. In one patient both the specialist geriatrician and neurologist found the diagnosis inconclusive and a third opinion was sought from a neurologist who rejected Parkinson's disease and the patient was excluded.

The data were analysed initially for difference in sensitivity. The questionnaire performed equally well with 100% sensitivity in the hospital group and the community group if screening positive for parkinsonism meant answering “yes” to any one of the 11 questions. The specificity was only 45% in the hospital group and 41.7% in the community group. The test of significance of two proportions5 was applied to both groups to discern if the proportion of positive responses to individual questions in the questionnaire was similar (or different) between the groups. Those questions that had similar proportions of true positive responses for parkinsonism and false positive responses for absence of parkinsonism, between the two groups, were used to build a model for prediction of parkinsonism.

Each question was analysed for true positive or false positive responses based on the subjects' clinical status. Proportions of subjects who were truly positive for parkinsonism in the hospital group were compared with the community group question by question. Likewise, proportions of subjects who were falsely positive were compared between the groups (table 1).

Table 1

Results of testing the proportions of positive response in each of the questions

Those questions that did not have a significant difference between the hospital group and the community group for parkinsonism and no significant difference between the groups for absence of parkinsonism were chosen for consideration in the final model. The questions that fulfilled the criteria were 3, 4, 5, 6, and 11. There was at least one significant difference between the hospital group and community group for the other questions.

For the five questions (3,4,5,6,11) in the final model, an arbitrary cut off point value of 0.5 was used to predict parkinsonism in the final analysis. The purpose of a cut off point was such that when an estimated response function value was greater than a cut off point value, the person was classified as having parkinsonism. An alternative cut off point (acquired by minimising a measure which is the absolute value of the difference of sensitivity and specificity divided by the overall correct classification rate) was tested as well. Using the cut off point value, the sensitivity, specificity, and overall correct classification could then be calculated. Fisher's exact test was used to obtain the p value of the association. Both multivariate and stepwise logistic regression6 were used to build the model for predicting whether a person had parkinsonism or not and a reduced model which consisted only of questions 3, 4, and 5 was formed. The result from this reduced model is given in table 2.

Table 2

Multivariate logistic regression models with two different cut off points

The likelihood ratio test was used to compare this reduced model with the full model (comprising questions 3, 4, 5, 6, and 11) for significant difference. The hypothesis was that the coefficients for the variables (or questions) excluded were equal to zero. As the p value was 0.5122, exceeding 0.05, we concluded that the reduced model was as good as the full model. Therefore, the stepwise (or reduced) model would be preferred (that is, questions 3, 4, and 5).

Questions 4 and 5 were important variables in the full (five questions) model based on Wald statistics.6 If we attempted to reduce the model further by choosing these two questions only, we found that a higher sensitivity of 90.6% was achieved (cut off point of 0.0311). However, this could not replace the full model based on the likelihood ratio test at the 5% significance level because the p value was 0.0399. An appealing aspect of this model was its higher sensitivity albeit with lower specificity and overall correct classification (table2).

Discussion

Epidemiology is labour and time intensive. Tools or questionnaires that can act as a sensitive filter mechanism may enhance efficiency, especially in conditions with subtle early clinical features (for example, Parkinson's disease).

Screening questionnaires currently available have been evaluated in hospital (or clinic) based samples with questionable applicability for community studies. The number of questions has varied from two to 13 with little knowledge whether the selection of questions may cause any difference in a community setting. The present study has sought to overcome both these shortcomings. The comparison of questionnaire responses with clinical impressions from blinded assessors has meant that the clinical relevance of questions could be tested. In addition, blinded examiners could not assume that any of the subjects automatically had definite Parkinson's disease or did not have the disease. Care was taken in making the diagnosis of possible Parkinson's disease in the community group and if in doubt, an independent neurological opinion was sought. It should be stressed that the diagnosis of Parkinson's disease is only about 80% accurate even among experts.7

Although the full set of 11 questions would yield 100% sensitivity for detecting parkinsonism in both hospital and community settings, there were significant differences between responses to many individual questions between the two groups, thus casting doubt on earlier studies which employed hospital validated tools with few questions and applied them to community studies.

The present questionnaire shows that only questions 3, 4, 5, 6, and 11 were equally relevant to both groups. When subjected to multivariate stepwise logistic regression, only questions 3, 4, and 5 were relevant. Thus the adaptation of the initial questionnaire could be refined to a choice of three questions which were sufficiently statistically robust to be relevant to both hospital and community studies of Parkinson's disease. To achieve a high sensitivity (84.4%) and specificity (86.3%), a positive response was required to at least two of the three questions (cut off point 0.1216). Higher sensitivity can be achieved (90.6%) albeit at a cost of lower specificity (80.6%) if a positive response means “yes” to either questions 4 or 5 (cut off point 0.0311).

Tremor was not found to be a good discriminating question, which is in contradistinction to the results of Mutch and Meneghini and others.8 9 These authors validated their questionnaires using hospital or clinic subjects and found tremor and gait problems as key questions in ascertaining Parkinson's disease. Tremor was not a reliable predictor in the community group possibly because this symptom is more likely to alert people to present earlier and therefore they are likely to be diagnosed earlier than those without. This explanation is confirmed by the result in table 1. In the hospital group 93.7% of people with Parkinson's disease identified tremor, by contrast with only 62.5% of the community group. Anderson et al 10 claimed that unrealistic testing (for example, the use of patients in hospital only) tended to indicate a better performance of the screening method than would happen in an actual survey.

Our validation study yielded a higher percentage of parkinsonism (just over 10%) than generally quoted in the literature. This may be partly explained by the fact our lower age cut off point is older (aged 52). Furthermore, a large percentage (seven out of 14 patients, five with Parkinson's disease and two with vascular causes) were newly diagnosed. These new patients may have been missed by the traditional methods, such as mail survey of general practitioners. Our result accords with a recent community study by Bennett et al 11 which reported that parkinsonian signs are extremely common in elderly people (of the order of 15% for people aged 65–74, 30% from 75–84, and 50% aged 85 years and over).

In conclusion, to our knowledge this is the first study that has attempted to determine how well screening questions for parkinsonism actually perform in a community setting. It also used statistical methods to build up a useful model of predicting parkinsonism which may be adopted for future epidemiological studies.

Appendix

Acknowledgments

This study is supported by the NSW Health Department, the ACMA, and the RACP.

References

View Abstract

Request permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.