Background In 2004, the WHO Collaborating Centre for Neurotrauma, Prevention, Management and Rehabilitation Task Force on Mild Traumatic Brain Injury published a supplement to the Journal of Rehabilitation Medicine containing a best evidence synthesis of the scientific literature on the epidemiology, diagnosis, prognosis and treatment of MTBI. This systematic review focused on the English literature from 1980 to 2002. Of 38 806 citations identified, 671 contained relevant scientific data. An additional 72 studies were identified by other means, totalling 743 studies. Only 42% (313) of these studies were deemed to be scientifically valid.

Aims My aim is to highlight the main methodological problems observed by the Task Force.

Methods The MTBI Task Force documented the strengths and limitations of the 743 reviewed studies. Information about the research question, study design, statistical methods, results, biases, strengths and limitations was collected. In addition, the Task Force collected specific information on studies that addressed the topics of risk, prognosis, diagnosis and treatment. This information addressed the unique aspects of study design (ie, randomised controlled trial, cohort, case-control, and cross-sectional) and important biases for each topic of study and design (eg, selection bias, information bias and confounding).

Results Studies on the incidence and risk of MTBI gave highly variable estimates that were dependent on the case definition (information bias) and the selection of cases (selection bias). Most studies gave an incidence proportion rather than a rate. Many studies were rejected because there was no denominator or population at risk identified. There were few studies that calculated the independent effect of risk factors. Many were rejected because of confounding. Diagnostic studies suffered from several biases. Few studies independently and blindly applied a gold standard reference test in a representative sample of patients (selection bias). Ceiling and floor effects were present in various case definitions. Prognostic studies tended to be underpowered and not capable of detecting independent effects of prognostic factors. There were few randomised controlled trials of interventions for MTBI and most were small in size and inconclusive. Of the 41 published guidelines reviewed, only five reported a methodology and only three were scientifically acceptable.

Conclusions The scientific quality of studies on MTBI was poor up to 2002. Cases and populations at risk were poorly defined, studies of risk and prognostic factors did not properly control for confounding factors, statistical analyses were crude and underpowered, and there were very few high-quality intervention studies.