The recently published paper ‘Abnormal pain perception is associated with thalamo-cortico-striatal atrophy in C9orf72 expansion carriers in the GENFI cohort’ by Convery et al.[1] draws attention to a topic of great importance in the field of frontotemporal dementia (FTD) research. In this study, Convery and colleagues investigated differences in pain responsiveness within a group of patients with genetic FTD. Changes in pain responsiveness compared to baseline were captured using a scale designed by the group, and patients were scored from 0-3 (0 = no change, 0.5 = questionable or very mild change, 1 = mild change, 2 = moderate change, 3 = severe change). Within the sample, symptomatic C9orf72 mutation carriers (9/31) experienced greater changes in pain responsiveness than symptomatic MAPT (1/10) and GRN (1/24) mutation-carriers or normal controls (1/181). Within the C9orf72 mutation carriers, these changes were associated with thalamo-cortico-striatal atrophy.
This research brings attention to an important but little-investigated clinical feature of FTD. Changes in pain responsiveness, including both increases and decreases, have now been reported in both sporadic and genetic FTD, along with other somatic complaints.[1–4] However, the changes are not widely captured in either clinical or research settings, and the field lacks standardized and objective measurements to do so. The ability to measure changes in pain responsiveness may be a useful clinical marker to differentiate FTD from other neurodegenerative diseases,[4] and, if the C9orf72 results of Convery et al. are replicated, as an indicator of possible genetic underpinnings.
Recent findings raise the possibility that changes in pain responsiveness differ between FTD phenotypes. Increased pain responsiveness has been reported in patients with semantic-variant primary progressive aphasia (svPPA), particularly in those with right-temporal atrophy, which stands in contrast to decreased pain responsiveness observed in behavioral-variant FTD (bvFTD).[2–5] These findings, in conjunction with those of Convery et al., highlight the importance of capturing directionality of change as well as severity. Similarly, analyses of different phenotypes within the FTD spectrum will be critical to broaden the clinical relevance of this research to sporadic FTD, as some phenotypes are rarely genetic (e.g., svPPA). In the Convery et al. paper, the overwhelming majority of symptomatic participants had bvFTD, which is typical for genetic cohorts. However, the extension of this research into sporadic cases raises the exciting question of whether changes in responsiveness to pain can distinguish between FTD phenotypes, which implicate overlapping but distinct neuroanatomical circuits. This question has great theoretical, as well as clinical, importance.
The Convery et al. paper highlights that altered responsiveness to pain was present in symptomatic but not presymptomatic genetic mutation carriers. It thus remains unclear whether altered pain responsiveness is an early feature of the disease or develops later. Elucidating this timeline will clarify the clinical utility of this research: whether it is useful for early diagnosis or for distinguishing between phenotypes after the dementia syndrome has developed.
As we continue to expand this line of research, it will be essential to develop both subjective and objective measurements of pain responsiveness and other somatic changes in patients with FTD. Refining our understanding of these changes has the potential to be useful in clinical and research settings alike.
 
1 Convery RS, Bocchetta M, Greaves CV, et al. Abnormal pain perception is associated with thalamo-cortico-striatal atrophy in C9orf72 expansion carriers in the GENFI cohort. J Neurol Neurosurg Psychiatry Published Online First: 5 August 2020. doi:10.1136/jnnp-2020-323279
2 Barker MS, Silverman HE, Fremont R, et al. ‘Everything hurts!’ Distress in semantic variant primary progressive aphasia. Cortex J Devoted Study Nerv Syst Behav 2020;127:396–8. doi:10.1016/j.cortex.2020.03.002
3 Snowden JS, Bathgate D, Varma A, et al. Distinct behavioural profiles in frontotemporal dementia and semantic dementia. J Neurol Neurosurg Psychiatry 2001;70:323–32. doi:10.1136/jnnp.70.3.323
4 Fletcher PD, Downey LE, Golden HL, et al. Pain and temperature processing in dementia: a clinical and neuroanatomical analysis. Brain J Neurol 2015;138:3360–72. doi:10.1093/brain/awv276
5 Ulugut Erkoyun H, Groot C, Heilbron R, et al. A clinical-radiological framework of the right temporal variant of frontotemporal dementia. Brain J Neurol 2020;143:2831–43. doi:10.1093/brain/awaa225

Russell et al. (1) published a retrospective cohort study with a population of former professional soccer players with known high neurodegenerative mortality. Findings showed that they are at lower risk of common mental health disorders and have lower rates of suicide than a matched general population. These findings are surprising and different from previous studies, which have used first-hand clinical accounts of ex-athletes who have lived with neurodegeneration (1). We suggest there may be reasons for this disparity and welcome critical dialogue with the authors of this research.

Cohort Comparison
Russell et al. has compared their soccer cohort with a matched population cohort. However, the matched cohort may also include those who have experienced repetitive head impacts, such as amateur soccer players, rugby players or boxers. Therefore, the study represents differences of elite versus non-elite rather than sport versus non-sport. While Russell recognises the healthy worker effect (2), it may have a greater influence in this study than presented.

Soccer Stoicism
Men’s engagement in health-seeking behaviours has been a long-standing concern in health care and is often attributed to factors such as stigma, hypermasculinity and stoicism (3). Furthermore, working-class sports such as soccer, require the acceptance of pain, suffering, and physical risk, so these players are more likely to ‘suffer in silence’ than the general population (4). Given the effectiveness of masculine socialisation through sport participation, the absence of elevated medical reporting between male athletes and non-athletes does not indicate an actual absence of a larger disease profile. The lack of ex-elite athletes engaging with mental health support at a hospital may rather be indicative of health-avoidance behaviours.

Mental Health Concerns Defined by Hospital Admission
Using hospital admission records as the primary definition for mental health concerns is problematic. Hospital admission is reserved for the most severe acute psychiatric concerns. Therefore, such records miss many mental health concerns that are better managed in primary and community care settings. This may be particularly pertinent for this study, given that many of the sample have diagnosed dementia, and may be fully supported with their neuro-psychiatric needs by health care professionals outside the hospital context.

The Bigger Picture
It appears that only a selective subsection of data, or part of the picture, has been reported. No information on the length of visit to hospital, extent and nature of medical interventions, public health burden, number or frequency of visits by an individual and any further care has been provided. This information may illuminate other explanations for why there is a difference in common mental health disorders for soccer and match control samples.

Concluding thoughts
Russell et al assert research, “… has placed greater emphasis on psychiatric symptomatology in CTE. Nevertheless, data supporting this association are weak”. This study does little to support or contest this position. While the results presented are novel, they are not necessarily a significant contribution to the debate on the relationship between soccer participation, common mental health disorders and other neurological

References

(1) Russell, E. R., McCabe, T., Mackay, D. F., Stewart, K., MacLean, J. A., Pell, J. P., & Stewart, W. (2020). Mental health and suicide in former professional soccer players. Journal of Neurology, Neurosurgery and Psychiatry.

(2) Li CY, Sung FC. A review of the healthy worker effect in occupational epidemiology. Occup Med 1999;49:225–9.

(3) Wang Y, Hunt K, Nazareth I, Freemantle N, Petersen I. Do men consult less than women? An analysis of routinely collected UK general practice data. BMJ Open. 2013;3(8):e003320

(4) Anderson, E., & White, A. (2017). Sport, theory and social problems: A critical introduction. Routledge.

We read Larrabee and colleagues’ e-letter response to our systematic review on Performance Validity Testing (PVT). Whilst we welcome debate, and we recognize that some clinicians will disagree with our conclusions, we were disappointed that they misrepresented our paper in formulating their response:

1. The authors state “Throughout the paper, the authors refer to PVTs as “effort tests”, a characterization that is no longer in use in the United States..”. In reality we used the term “effort test” only twice in our paper; in the introduction: “(PVTs), also historically called effort tests” and once in the methods in describing our search terms. By contrast we use the term PVT on 45 occasions.

2. We are concerned that they then go on to misrepresent the results of our review. We found a wide variation in results in different clinical groups and in different tests. We noted that failure rates for some groups and some tests exceeds 25%. We did not conclude that all failure rates were as high as this, but rather that failing a PVT was not a rare phenomenon but was reasonably common in a range of clinical groups.

We presented results to support our conclusion that the PVT literature is problematic with regards to blinding to diagnosis and potential for selection bias.

We also uphold our speculation that an alternate explanation for failure on forced choice tests at above chance cutoffs may result from attentional deficit related to other symptoms. We were explicit that there are likely to be many reasons that a person might fail a PVT, and invite further research and discussion of the various causes of PVT failure that might help to explain the observed variation in failure rates across a range of diverse clinical conditions.

3. We did not ignore the importance of using multiple PVTs. On the contrary we explicitly stated “the manner in which we have described PVT failure rates does not necessarily reflect how they are used in practice by skilled neuropsychologists” and that “Guidance documents recommend that multiple performance validity measures should be used, including both free-standing and embedded indicators ...” But that was not the subject of the study which was the performance of individual tests.

4. They allege that we did not mention any of the previous published meta-analyses summarizing data from multiple investigations. However, the Sollman and Berry review they suggest has a quite different focus from our study, and the majority of studies included were of mixed clinical populations (e.g. ‘psychiatric’, or ‘neurological’ or ‘head injury’ without severity specified)(1). It was not relevant to our question.

Larrabee et al. raise three valid points of criticism:

1. That we were inaccurate in our portrayal of the Novitski paper. We have reviewed the Novitski et al. paper and although % fail rate was accurately transcribed we agree the study was erroneously included, as the included mTBI patients had already failed the WMT(2). Although interestingly, 36% of the amnestic MCI sample in this paper (who were not administered the WMT) also failed RBANS digit span <9. Of note, the mTBI group in the Novitski paper was not included in our Figure 2, due to the rather higher-than-usual cutoff score of 9. This isolated error does not alter our overall conclusions.

2. They challenge our statement that there is little consensus amongst experts on the use of PVTs. However Kemp et al, in further correspondence with us, stated explicitly that the view of these tests as ones of malingering was old fashioned and largely no longer accepted, whereas Larrabee and colleagues refer to them as just that. The British Psychological Society refer to them as ‘effort’ tests throughout their guidance whereas Larrabee and colleagues criticize such nomenclature. It seems to us even from the responses to our paper that there is a lack of consensus.

3. Finally, while Larrabee et al. report that our results are “not representative of the research database” we counter with an assurance that this data was just that: a systematic extraction of all available date from the research base on PVTs.

References:

1. Sollman MJ, Berry DTR. Detection of Inadequate Effort on Neuropsychological Testing: A Meta-Analytic Update and Extension. [cited 2020 Jan 15]; Available from: https://academic.oup.com/acn/article-abstract/26/8/774/4085

2. Novitski J, Steele S, Karantzoulis S, Randolph C. The Repeatable Battery for the Assessment of Neuropsychological Status Effort scale. Arch Clin Neuropsychol [Internet]. 2012;27(2 PG-190–195):190–5.

We read with interest Kemp and colleagues response to our recent systematic review on Performance Validity Testing (PVT). In response to specific criticisms raised:

1- The searches and data extraction were conducted by one investigator. We agree this is a potential limitation although only if papers were missed, or data was erroneously transcribed, and it can be demonstrated this would have changed the conclusions. Although Kemp and colleagues place great weight on this, the evidence they put forward to support their contention was limited. Of the four citations in their letter, reference 2 and reference 4 were in fact included (see our supplementary tables and our reference 57)(1,2). Reference 3 was, by coincidence, published simultaneously with our manuscript submission and was not available to us(3).

Reference 1 did not fit the terms of our search strategy and was not included although it would have been eligible(4). It was an unblinded study of the ‘coin in hand test’, a brief forced choice screening test for symptom exaggeration, administered to 45 patients with mixed dementias. It found 11% scored at or above a 2 error cut off and the authors proposed a new set of cut offs for interpretation; it was in keeping with our conclusions. We’d be happy to consider any other specific omissions or quality assessment issues not discussed which the authors consider would have altered the conclusions of the review.

2- The authors criticise our understanding of how PVTs are used in clinical practice, stating that PVTs should not interpreted on a stand-alone basis but in combination as part of a wider assessment. We agree and made that point explicitly in the paper. Nonetheless an understanding of the accuracy of individual tests remains of key importance in the weight accorded to individual tests in that wider assessment. They state that the way we presented single test failure rates is not the way the tests should be used. Again, we agree and pointed this out in the paper. However, they also misrepresent us, as we did not score or interpret the tests ourselves, nor did we conflate different forms of testing - we reported all the available data as the authors presented it.

3- The third point they make is that we are not saying anything new. We agree in as much as we have methodically documented and grouped in one paper data that was in the public domain. In particular, the authors suggest that ‘base rate failure is well understood’ but in our experience that is not the case; indeed the British Psychological Society’s own guidelines state “Further evidence on UK base rates of cognitive impairment and failure on effort tests in a range of clinical presentations and service settings is needed”(2). There are no other papers that synthesise this data in clinical populations to give readers an overview of these base rates. More importantly, the evidence we found, showed a wide variety of use and interpretation of PVTs. Kemp and colleagues go on to describe how studies should ideally be done to compare clinical populations. We agree and discuss this in our closing remarks. The problem is that evidence to date falls far short of this ideal.

As we made clear in the paper, we agree that PVTs may be useful in the correct context and with an understanding of their limitations. We were not “dismissive” of the developing PVT literature and we believe this should be a literature open to scrutiny by all.

1. Sieck BC, Smith MM, Duff K, Paulsen JS, Beglinger LJ. Symptom validity test performance in the Huntington Disease Clinic. Arch Clin Neuropsychol. 2013;28(2 PG-135–43):135–43.
2. British Psychological Society. Assessment of Effort in Clinical Testing of Cognitive Functioning for Adults. 2009.
3. Sherman EMS, Slick DJ, Iverson GL. Multidimensional Malingering Criteria for Neuropsychological Assessment: A 20-Year Update of the Malingered Neuropsychological Dysfunction Criteria. Arch Clin Neuropsychol. 2020;00:1–30.
4. Schroeder RW, Peck CP, Buddin WH, Heinrichs RJ, Baade LE. The Coin-in-the-Hand Test and Dementia. Cogn Behav Neurol. 2012 Sep;25(3):139–43.