Article Text

Download PDFPDF

Review
The use and misuse of short cognitive tests in the diagnosis of dementia
  1. Jeremy Brown
  1. Correspondence to Dr Jeremy Brown, Department of Neurology, Addenbrooke's Hospital Memory Clinic and Queen Elizabeth Hospital King's Lynn NHS Trust, Box 83, Addenbrooke's Hospital, Hills Road, Cambridge, CB2 2QQ, UK; jmb75{at}medschl.cam.ac.uk

Abstract

Short cognitive tests are widely used in medicine to assess patients with memory problems but their role in the assessment of patients with cognitive problems is often misunderstood. They are a part of the examination of the patient and not tests for dementia or ‘case-finding tools’. This misunderstanding leads to widespread misconceptions concerning short cognitive tests and could lead to major over diagnosis or under diagnosis of dementia. Their use in clinical practice particularly in response to national directives aimed at increasing diagnosis rates in dementia needs a clear understanding of their role and limitations. A new classification of short cognitive tests is proposed with guidance on their use in clinical medicine.

Keywords
  • Alzheimer's Disease
  • Cognition
  • Dementia
  • Neuropsychology

Statistics from Altmetric.com

Keywords

Aim

To clarify the role of short cognitive tests in the diagnosis of cognitive disorders.

Background

There has been an explosion of interest in the diagnosis of dementia in the past 5 years fuelled in part by the ageing world population.1 This growth in interest has been encouraged by the UK2 and other Governments: 10 countries in Europe have followed France in developing National Dementia strategies and others are promised,3 details of these national strategies are published on line.3 Reports such as the UK National Dementia Audit of 20104 have highlighted shortcomings in the assessment of patients with dementia.

There is political involvement in this issue, the UK National Dementia Strategy: Living Well with Dementia in 2009, contained two forewords written by Government ministers.2 Objective 2 of the strategy was “Good-quality early diagnosis and intervention for all”. The strategy noted that only a third of patients with dementia received a diagnosis of dementia at any time in their illness and aimed to formulate a plan to improve diagnostic rates. This political drive has continued with the current UK Government with the Prime Minister's dementia challenge in March 20125 which aimed to greatly increase dementia diagnostic rates by 2015. The political drive, which encourages screening for dementia, is powerful but not evidence-based.6 The UK Government has developed initiatives in primary and secondary care to increase rates of diagnosis of dementia.7–9 The health minister has launched a map showing dementia diagnostic rates for different regions of the UK, increasing pressure on health professionals to increase rates of dementia diagnosis.10 Doctors are being asked to double the rate of diagnosis of dementia in the UK with an approach relying on minimal history and very short cognitive tests to ‘case-find’ patients with dementia.

This is not a purely UK strategy, in the USA the annual wellness Medicare visit requires detection of any cognitive impairment.11 The political drive culminated in the G8 dementia summit leading to a declaration of Global Action against Dementia.12

Outline of formal UK recommendations

In the UK, the latest Government Directed Enhanced Services (DES) for general practitioners (GPs) aims to reward them for testing for dementia in at-risk groups.7 The 2013 dementia Commissioning for Quality and Innovation document (CQUIN) offers payments to National Health Service (NHS) hospital trusts for ‘case-finding’ patients with dementia—this includes a diagnostic assessment.8 ,9 These programmes promote short cognitive tests to identify patients with possible dementia. The DES specifies the use of a short cognitive test validated in general practice—such as the General Practitioner Assessment of Cognition (GPCOG).7 The CQUIN includes the Mental Test Score (MTS) as part of the assessment.8 ,9 Both initiatives promote very short assessments, probably to improve compliance. There is a danger that misunderstanding of these tests and their role in the diagnosis of cognitive problems may lead to massive over diagnosis or under diagnosis of dementia. The larger risk is that over reliance on short cognitive tests in the diagnostic process will lead to patients with fixed cognitive problems or delirium receiving a diagnosis of dementia; but there is also a major risk that using easy short cognitive tests will fail to recognise patients with mild dementia.

Screening for cognitive deficits or diagnosing dementia?

CQUIN guidelines issued by the UK Government conflate two separate processes: screening for cognitive deficits and diagnosing dementia. The difference is illustrated by the case of an elderly patient admitted to a hospital emergency unit at midnight. Screening for cognitive deficits is an important part of the initial assessment—if a patient is disorientated and drowsy then the patient may be delirious and this is important to recognise, investigate and treat.13 ,14 Dementia is a chronic condition and should only be diagnosed once acute causes of cognitive dysfunction have been excluded,15 so it is not an appropriate time to try to diagnose or ‘case find’ dementia. Dementia diagnosis needs to be carried out once the patient has had their medical problems treated, is properly rested and in a calm environment.

How is a diagnosis of dementia made?

The Diagnostic and Statistical Manual Fifth Edition (DSM-5) definition of dementia (re-named Major Neurocognitive Disorder in this classification) is of a significant cognitive decline from a previous level of performance in one or more cognitive domains. The decline should be sufficient to undermine independence in everyday activities and not occur exclusively in the context of delirium. The decline should not be explainable by another mental disorder.15

The diagnosis of dementia can be complex and requires experience and time. National Institute for Health and Care Excellence (NICE) guidelines recommend that the diagnosis is made by specialist services16 and this implies that specialist referral should be considered in all cases of dementia. As some dementias are reversible, early referral is important; early diagnosis is recommended in the National Dementia Strategy.2

NICE recommends several stages in the diagnosis of dementia16

  1. History: As many patients with dementia have reduced insight they may underestimate the problem so it is important to take a history from a suitable informant as well as the patient. The contrast between the story from a close informant and the patient's account is often the most useful part of the assessment. It is important to ensure that medication is not contributing to their problems.

  2. Physical examination: Physical signs such as a visual field defect suggest a secondary dementia and may contribute positive information such as parkinsonian features in patients with dementia with Lewy bodies (DLB).

  3. Cognitive examination: It is fundamental to the diagnosis of dementia that a deficit in cognition is identified.

  4. Investigations: Imaging and other investigations can exclude secondary causes of dementia but may also produce positive support for a clinical diagnosis—such as hippocampal atrophy in Alzheimer's disease (AD).17

It is also important to follow-up the patient. Patients with mild problems such as amnestic mild cognitive impairment (aMCI) may not progress.18

In the mid-to-late stages of dementia the diagnosis can be easy to make and it is possible to omit some of these stages. The earlier the diagnosis is attempted the more difficult it becomes and the higher the risk of misdiagnosis. The key parts of the assessment are the history from the patient and an informant and the performance of the patient on cognitive testing.

How are cognitive skills currently tested?

Cognition can be tested informally during a medical consultation by asking questions concerning recent events and autobiographical details. A more formal assessment using a short cognitive test may detect other problems and gives the patient a score. NICE guidelines recommend the use of a short cognitive test in the diagnosis of dementia.16 Formal neuropsychological testing is important in the assessment of patients with complex problems or very subtle deficits but is not available for most patients with dementia in the UK.

Short cognitive tests

The two most popular short tests are the MTS, also known as the Abbreviated Mental Test (AMT), published in 197219 and the Mini-Mental State Examination (MMSE) described in 1975.20 These tests are around 40 years old but despite limitations continue to dominate the assessment of patients with cognitive problems. Many newer tests have been published. NICE suggests the MMSE, GPCOG, six-item cognitive impairment test (6-CIT) or the 7 min screen. Department of Health guidance suggests the GPCOG in primary care7 and the MTS for dementia case finding in secondary care.8 ,9

In order for short cognitive tests to be accepted into clinical practice they need to be validated in studies. There are many issues in assessing validation studies of short cognitive tests. Most validation studies have had limited application to normal clinical practice:21 many lacked clear methodology, were underpowered, examined populations with a high prevalence of dementia or were essentially correlative (comparing two tests not validating against a clinical diagnosis). Many validation studies have screened populations with known dementia, such studies do not ‘validate’ their role in the more complex initial diagnosis of dementia.

The selection of the patients is crucial to the results.21 A different population of patients will be recruited from a hospital memory clinic to those recruited as hospital inpatients or from general practice. The severity and prevalence of dementia in the study population will influence the performance of the test so recruiting patients with established moderate dementia (as opposed to patients presenting with mild problems) will improve the specificity and sensitivity of a test. Virtually any ‘cognitive’ test (even how long it takes to eat an apple) should distinguish patients with moderate dementia from healthy volunteers. Patients in validation studies are often young compared to those seen in clinical practice and problems arise in defining ‘normal’ cognition in an aging population.

Studies use three main types of controls—relatives of the patient, healthy volunteers and other patients. Patient's relatives are from a similar educational and cultural background but it can be difficult to match patient and control groups. Healthy volunteers are more easily matched to the patient group than relatives but screened, healthy volunteers may be higher performing individuals exaggerating the difference between patients and controls.

Using other patients—such as patients presenting with memory problems but diagnosed as ‘worried well’ as controls mirrors clinical practice and provides a tougher assessment that will tend to lead to lower sensitivities and specificities. There are problems: it is hard to be certain that patients presenting with memory problems really are ‘worried well’ and not misdiagnosed and how to analyse the group of patients in whom the diagnosis is unclear. Some studies include patients with aMCI as part of the control (‘non-demented’) group,22 potentially favouring less sensitive tests over more sensitive ones.

As the specificity and sensitivity of short tests depends so much on the design of the study comparing these measures across different studies has limited scientific validity.21 The unreliability of such comparisons is shown in a summary of specificity and sensitivity data tabulates the results for various studies and shows that the sensitivity of the MMSE in the diagnosis of dementia varies from 37% to 100% and the specificity from 48% to 100% in different studies.23

Current short cognitive tests

Most current short cognitive tests can be divided into three different classes:

  1. Short questionnaires

  2. Highly selective tests

  3. Multi-domain tests

Short questionaires

The MTS is the archetypal short questionnaire, many others have been published since for example, 6-CIT,24 4-point Abbreviated Mental Test,25 Six-item screener.26 At least 50% of the total score in these tests is for orientation with single points for other skills. They do not examine many cognitive skills. A major advantage of short orientation dominated questionnaires is that they take only 2–3 min to administer.

The MTS is a shortened version of an earlier test, the Roth-Hopkins test27 and consists of 10 questions, most of them testing orientation. The MTS was originally validated in a study as to detect ‘mental impairment’ in elderly inpatients.19 Most subsequent studies have been in elderly medical inpatients.

The MTS has disadvantages, these include that scoring is surprisingly difficult and subjective,28 (one word answers are either right or wrong—so a series of nearly right answers can score 0) and there is no direct patient input to the score sheet, so a clinician looking at the sheet later cannot judge the patient's performance directly.

Six-ITEM CIT24 is another short questionnaire recommended by NICE for the diagnosis of dementia although validation studies have concentrated on the recognition of established dementia24 ,29–30 rather than on the initial diagnosis.

Use as screening tests

Short questionnaires have been validated in the recognition of established dementia19 ,22 ,24 ,29–30 and are useful when there is little time and there is a high prevalence of cognitive problems among the patients—for example in assessing elderly medical inpatients. The emphasis on orientation is useful in screening for delirium, in which disorientation is a cardinal feature.

However, short questionnaires may be less sensitive than longer tests in detecting cognitive problems: the MTS is inferior to the MMSE in screening for cognitive problems in hospitals and the community.23 ,31

Use in the diagnosis of dementia

Short orientation dominated questionnaires are not generally used for the diagnosis of mild dementia. Orientation is preserved in patients with mild neurodegenerative disease and patients with mild, organic problems will pass these tests. One of the largest studies of the MTS found that the positive predictive value of the MTS for a diagnosis of dementia was only 25% and was therefore insufficient to help the diagnosis of dementia.32

Highly selective tests

Patients with established AD have specific difficulty recalling newly learnt words or drawing a clock, two tests easily administered in a clinic. Some short tests concentrate on one or both of these tasks. The most widely used such tests are the GPCOG,33 Memory Impairment Screen,34 Mini-cog35 and various Clock Drawing Tests.36–37 These tests take 2–8 min to administer.

Use as screening tests

Highly selective tests have been validated as screening tests for patients with moderate AD.32–37 They are designed for detecting AD and may be less good at detecting other forms of dementia, although patients with DLB may score poorly on clock drawing tests.38 The original authors found that the GPCOG was superior to the MTS in detecting dementia.33

Use in the diagnosis of dementia

There are very few studies of highly selective tests in the initial diagnosis of dementia. Empirically, highly selective tests are likely to be of limited use in the patient presenting with mild memory problems because they test too few cognitive skills. The pattern of mild AD is that episodic memory is affected but many other cognitive skills are preserved; therefore it is important that a test examines many skills.

This issue is important because the validation of the GPCOG in primary care has promoted its use in the diagnosis of dementia in primary care in the UK.7 The original GPCOG validation study33 is largely a study of elderly patients with a high prevalence of established cognitive problems, the study was flawed by a high drop-out rate and incomplete patient histories. The study supports the use of the GPCOG as a short tool for recognising patients with moderate cognitive problems in primary care, but this study does not examine its role in the diagnosis of mild AD or dementia in primary care. The Italian GPCOG validation study examined patients presenting with cognitive problems.39 This study shows that the Italian GPCOG performs very similarly to the MMSE. However, in this study there was a high prevalence of dementia and patients scored an average MMSE of 21 suggesting established disease.

The GPCOG includes a brief informant questionnaire that improves its performance,33 however this lengthens the test and introduces the influence of a third (including the tester) individual. The history is better taken separately.

Multidomain tests

These tests aim to test many cognitive domains and can establish what a patient can do, as well as what they cannot do. The most popular is the MMSE. In 1975 Folstein et al20 designed a short standardised test for the assessment of the mental status of patients on psychiatric wards. The MMSE is a medical phenomenon with over 50 000 citations on Google Scholar, despite numerous critiques.23 ,40 ,41 The test was designed to detect cognitive deficits in psychiatric inpatients. It was not designed as a test for AD (which at that time was regarded as a rare cause of presenile dementia). The original validation study was underpowered for detection of dementia with only 38 patients, the patients were very severely affected (with an average MMSE score of 12) and little detail is given of how the diagnosis of dementia was made. Over 100 validation studies have been published since and while most have significant faults,23 the sheer number establishes the usefulness of the MMSE.

The MMSE is not sensitive to mild AD21 ,40 ,41 as many of the subtests are too easy. The Addenbrooke's Cognitive Examination–III (ACE-III),42 Montreal Cognitive Assessment (MoCA)43 and Test Your Memory (TYM)44 are more recent and more difficult tests in this group. These tests take 5–20 min to administer.

The ACEs42 ,45 ,46 examine a range of cognitive domains, they have been translated into several languages.47 The ACE-III (an adaption of the ACE-R45 to avoid copyright issues with the MMSE) takes a trained nurse or doctor 20 min to administer45 which limits its application outside specialist clinics. The Montreal Cognitive Assessment (MoCA) has a layout similar to the MMSE but is a more stringent test of cognition and is gaining popularity.43 The TYM is a recent addition that resolves the paradox of more thorough testing in less time through self-administration of the test by the patient under minimal supervision.44

Use as screening tests

The MMSE has been widely validated as a screening tool for dementia with variable results.23 The harder multidomain tests are unlikely to be suitable as screening tests in populations with a high prevalence of moderate or severe cognitive problems as they are too sensitive. They have been shown to be useful in particular circumstances. The ACE-R and MoCA are useful in the detection of cognitive problems in other diseases such as Parkinson's disease.47 ,48 The TYM test functions well as a screening test in a general neurology clinic.49

Use in the diagnosis of dementia

Multidomain tests establish what a patient can do as well as what they cannot and therefore are useful supportive tests in the diagnosis of mild dementia. However, the easiness of the MMSE makes it insensitive to early dementia and it is more useful as a screening tool than in aiding the diagnosis of mild dementia.23 The ACE-R, MoCA and TYM are more sensitive to mild AD than the MMSE,43–45 they were initially validated in memory clinics to aid the diagnosis of dementia and these validations have been replicated in subsequent studies.47 ,48 ,50 They can be recommended for this purpose. The ACE-III and TYM contain much information recorded or spoken by the patient themselves, this preserves an objective record of the patient's abilities and reduces the influence of the tester, it also allows a third party to assess the patient's performance independently later.

Limitations of short cognitive tests

Short cognitive tests are part of the examination of the patient, they are not diagnostic tests

When a professional asks a patient to perform various physical tasks, it is self-evident that they are examining the patient, not performing a diagnostic test. The same is true for cognitive tasks: short cognitive tests are part of the examination, albeit one aided by standardisation. Once this is understood then many of the problems associated with short cognitive tests and their interpretation are highlighted.

The limitations of short cognitive tests are the same as for any form of physical examination. Unless they are combined with a history they can be surprisingly unhelpful in making a diagnosis. This can be illustrated by comparing the diagnosis of a stroke to that of dementia.

When a patient presents with unilateral weakness, an examination alone gives much less information that a full assessment. The briefest history clarifies the diagnosis: the statement “Mrs Jones was fine last night but awoke this morning aphasic and weak down the right side” strongly suggests a stroke. In contrast a physical sign in isolation: such as an extensor right plantar response is of limited use and certainly should not be used alone to diagnose stroke.

Similarly a cognitive score alone should not lead to a diagnosis. A very brief history such as “Mr Jones has begun to ask the same question several times an hour” in addition to poor episodic memory on a short test strongly suggests AD, a score of 22/30 on the MMSE should not be used alone to diagnose AD.

Reducing human cognition to a single number

Most short cognitive test results are summarised by a number with a top score between 4 and 100, the scores are not points on an absolute scale and scoring schemes for short tests are subjective and subject to manipulation for external aims, such as ensuring that the total score adds up to a round number. The reduction of human cognitive function to a single figure has its uses: a numerical value can help assimilate information about a patient rapidly and experienced clinicians have an idea of what a MMSE of 15 or 25 suggests about a patient with AD. However, reductionism can lead to the belief that a score of 24 on the MMSE is normal while 23 signifies dementia (particularly in validation studies). Cut-offs are a rough guide and should not be determined in one clinical scenario and applied in another.

Interpreting results

The pattern of the scoring is often more important than the overall score. A native French speaker scoring 87/100 on the ACE-III administered in English, with reduced verbal fluencies and slight difficulty in naming line drawings probably has normal cognition. An English professor scoring 87/100 with all points lost on recall and recognition of material learnt within the test probably has early AD.

External circumstances can influence the score. In the MMSE 5 points (17% of the total) are awarded for orientation in place20 so a patient tested in hospital may score fewer points than at home.

Education and knowledge of the test language will influence the scores.21 Short cognitive tests cannot be entirely independent of educational influence whatever the authors claim (a person who has not been taught to draw a clock will not be able to do one in a test). Devising easy tests in the native language in countries with universal secondary education should minimise the effect of education and language as shown in the original TYM study.44 In lower income countries the level of education is likely to have a larger influence on the score of cognitive tests, this was shown in the Greek TYM validation study conducted largely in rural areas with limited education.51 Minority ethnic groups could be disadvantaged in short tests unless they are fluent in the local language and familiar with local culture.52

Anxiety, attention, motivation and physical handicaps will influence results.

A single test will not suffice, a test far too easy for a graduate with mild problems in clinic, may be far too difficult for an elderly inpatient with learning difficulties.

Applying short cognitive tests to major screening programmes for dementia

The potential pitfalls of official campaigns to boost dementia diagnosis can be illustrated by analogy considering how a government might screen for multiple sclerosis (MS) in all hospital inpatients. It is impractical to employ neurologists to examine all patients admitted to hospital, select those who they feel may have MS and order MRI and cerebrospinal fluid analyses on this group.

If a similar approach was taken to that used in dementia then a single question could be formulated such as: “Have you in the last year experienced any short-lived episodes of numbness or visual disturbance?” This would be followed by a shortened neurological examination.

The objections to such a scheme mirror the objections to mass screening for dementia. A single question is no substitute for a proper history. It is likely that patients with migraine and benign phenomena would say yes to this question and a sizeable proportion of patients with progressive MS would say no.

If the examination is used as the main discriminator then many patients with other causes of (eg leg weakness), will be misdiagnosed. These patients may far outnumber the patients with MS. Using a highly abbreviated examination will compound these problems and may lead to a diagnosis based on a single equivocal sign. Mass screening will lead to mass diagnosis and mass misdiagnosis. These patients would overwhelm local neurology and radiology services. Current dementia initiatives risk a similar problem with dementia diagnosis. Dementia patients need the same care in diagnosis as that shown in MS and other diseases.

The CQUIN and National Dementia audit highlight extremely important issues. There are valid reasons to screen for cognitive problems in some individuals, for example, elderly hospital inpatients.14 Screening of unselected populations for dementia does not fulfil Wilson's criteria for a screening programme53 because of the lack of an effective treatment. Mass, early, unreliable diagnosis of dementia is not justified.6

Conclusion

Short cognitive tests allow the identification of cognitive deficits in patients. Recognition of cognitive deficits is fundamental to the management of patients with cognitive problems but the identification of cognitive deficits in patients must not be confused with the diagnosis of dementia. Doctors using short tests are examining the patient not performing a diagnostic procedure. Over-reliance on very short, poorly validated tests may lead to mass over or under diagnosis of dementia. There is a need for good studies of short cognitive tests that follow the normal patient pathway.

Recommendations

Short cognitive tests can be divided into three main categories with different uses (see table 1):

  1. Short questionnaires such as the MTS are useful for large scale screening for cognitive deficits in populations with a high prevalence of cognitive problems—such as medical inpatients. They should be regarded as a minimal requirement in the assessment of elderly medical inpatients for cognitive deficits. Short questionnaires are not suitable for supporting a diagnosis of mild dementia.

  2. Highly selective tests such as the GPCOG are useful tests for patients with established AD in general practice. They are less helpful in mild dementia.

  3. Multidomain tests such as the ACE-111 are useful to support the initial diagnosis of dementia in primary or secondary care. Completion of a multidomain test should be a minimum requirement in the diagnosis of dementia or prior to referral to a specialist for diagnosis. Multidomain tests are too sensitive to be useful in large scale screening of patients for cognitive deficits.

Table 1

Classes of short cognitive tests, their advantages and disadvantages and suggested uses

Key messages

  • Short cognitive tests are part of the examination of the patient not diagnostic tests.

  • Recognising cognitive deficits (which may be temporary) needs to be clearly separated from diagnosing dementia.

  • National campaigns need to aim at improving clinical skills in recognising cognitive problems, not mass ‘case-finding’ patients with dementia using protocols. The current tests recommended for national screening programmes have not been shown to be appropriate.

  • Clear guidelines are needed on the choice and interpretation of short tests.

  • Very short tests have serious limitations.

  • Good comparative studies between short cognitive tests are lacking.

  • A single test will not suffice for all assessments.

  • Different classes of short cognitive tests are better suited to different tasks—short questionnaires are useful in rapid screening of patients but multidomain tests are more useful to support a clinical diagnosis of dementia.

Acknowledgments

The author would like to thank Drs Philip Buttery, Peter Martin and John Thorpe for reading and making comments on an earlier draft of this article.

References

Footnotes

  • Short cognitive tests are part of the examination of the patient not diagnostic tests; failure to understand this can lead to massive over or under diagnosis of dementia.

  • Competing interests JB is the inventor of one of the tests discussed, the TYM test.

  • Provenance and peer review Not commissioned; externally peer reviewed.

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.