Article Text

Download PDFPDF

Kinematical analysis of emotionally induced facial expressions: a novel tool to investigate hypomimia in patients suffering from depression
  1. R Mergl1,
  2. P Mavrogiorgou2,
  3. U Hegerl1,
  4. G Juckel2
  1. 1Laboratory of Clinical Neurophysiology, Department of Psychiatry, Ludwig-Maximilians-Universität München, Nussbaumstr. 7, D-80336 München, Germany
  2. 2Department of Psychiatry, Charité, Wilhelm-von-Humboldt-Universität zu Berlin, Schumannstr. 20–21, D-10117 Berlin, Germany
  1. Correspondence to:
 Dr R Mergl
 Department of Psychiatry, Ludwig-Maximilians-Universität München, Nussbaumstr. 7, D-80336 München, Germany;


Objective: A novel technique for the kinematic analysis of emotionally induced facial expressions was applied to detect subtle mimic dysfunction in patients with depression.

Methods: Using ultrasound markers at certain points on the face, facial movements were exactly measured while subjects watched a witty sketch (“Mr Bean”). Twenty five medicated patients with depression (11 men, 14 women; mean age, 55.8 years; mean total Hamilton Depression Rating Scale score, 17.1) and 25 healthy controls, matched by sex distribution and handedness, were studied.

Results: Depressed patients were characterised by abnormally slow velocity at the beginning of laughing and voluntary facial movements, in addition to reduced laughing frequency. A higher severity of symptoms of depression was significantly associated with slow initial velocity of laughing movements of the left mouth angle (r  =  −0.45).

Conclusion: The execution of voluntary and non-voluntary facial movements is abnormally slow in depressed patients, reflecting hypomimia. This mimic slowing is closely associated with the severity of depression. The response of depressed patients to emotional stimuli is also abnormally low, but emotional estimation of the stimuli is similar to normals. This pattern parallels the motor–emotional features known from patients with Parkinson’s disease.

  • ERT, emotional reaction time
  • HAMD, Hamilton Depression Rating Scale
  • IV, initial velocity
  • PD, Parkinson’s disease
  • VAS, visual analogue scale
  • depression
  • facial expression
  • kinematic analysis

Statistics from

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

In many depressed patients, emotionally induced facial expressions are greatly reduced.1 Studies (for example, Katsikitis and Pilowsky2) have demonstrated a reduced number of facial movements in depressed patients. However, this finding is unspecific. Therefore, methods are needed that can separate the facial abnormalities in depression from those in other diseases and that can distinguish between drug induced and disease related facial abnormalities. A new computer aided method3,4 for the exact measurement of the initial velocity (IV) of laughing movements triggered by emotional (humorous) stimuli might be useful in this respect. Using kinematic analysis, Juckel et al separated unmedicated schizophrenic patients who had an abnormally fast IV of laughing from schizophrenic patients treated with typical neuroleptics, such as haloperidol, who had an abnormally slow IV.5

In view of these promising results, we aimed to investigate facial movements elicited by humorous film stimuli in depressed patients and healthy subjects, using a computer based approach. Facial activity was expected to be abnormally reduced in depression.


Twenty five adult inpatients suffering from depression (14 women; all right handed; mean age, 55.8; SD, 14.8 years) and 25 healthy controls (14 women; two left handed; mean age, 46.0; SD, 13.7 years) participated in our study. Patients and controls were comparable with regard to sex distribution and percentage of left handedness. However, the controls were significantly younger than the patients (p  =  0.02; t test). Our study was approved by the local ethics committee. All subjects gave written informed consent according to the Declaration of Helsinki.6

The severity of depression was assessed using the Hamilton Depression Rating Scale (21 item version; (HAMD-21)7 (mean total score, 17.1; SD, 10.5)).

All patients were diagnosed according to ICD-10 (F31.x, five patients; F32.x, three; and F33.x, 17).8 The mean age at the onset of depression (as determined by asking the patients for the onset of their first depressive episode) was 42.9 (SD, 12.6) years, the mean duration of disease was 12.9 (SD, 10.0) years, and the mean number of relapses was 2.5 (SD, 1.4). Five patients had a history of attempted suicide.

Fourteen patients were treated with psychotropic drugs, namely: tricyclic antidepressants (nine), atypical antidepressants (such as mirtazapine) (three), and lithium or carbamazepine (nine). Benzodiazepines were prescribed in two cases. No patient had undergone electroconvulsive treatment.

Exclusion criteria were: neurological disease, severe organic disease, reactive depression, depression with organic aetiology, substance abuse, low level of intelligence, and neuroleptic drugs.

An active device for the three dimensional measurement of movements (CMS 70; Zebris Ltd, Tübingen, Germany) was used to analyse facial movements elicited by humorous film stimuli.4,9 High frequency ultrasonic signals emitted by ultrasonic markers were registered online with a high sampling rate (200 Hz divided by the number of ultrasonic markers). We fixed one reference marker to the forehead by tape, two markers to the mouth angles to register the activity of the zygomaticus major and minor, risorius, and depressor anguli oris muscles, and two other markers to the medial inferior rims of the eyes to register the activity of the orbicularis oculi muscle. These muscles are innervated by the nervus facialis.

All subjects sat in front of a device for the registration of ultrasonic signals, a video camera, and a television screen (for presentation of one “Mr Bean” sketch). The subjects assessed how funny the sketch was, using a visual analogue scale (VAS) (minimum value, 0 mm; maximal value, 167 mm).

We registered facial activity by applying the ultrasonic measurement system and simultaneous video recording of the face. To separate involuntary facial movements (laughing) from voluntary ones, subjects had to stretch their mouth angles and to close their eyes tightly after watching the sketch. The film, video recording, and facial activity were synchronised using a frame code generator.

Using the software 3DA, the digitised signals of the markers were filtered using a special algorithm.10 “Laughing” was evaluated in five especially funny film sequences by identifying synchronous changes of facial activity in the video recordings and ultrasonic markers.

We computed the following:

  • Frequency of laughing (number (N) of laughing reactions to the sketch in relation to its length (N/min movie)).

  • IV (mm/s) of the markers at onset of laughing (mean slope through the start and maximum point of the correspondent movement based on three recordings) and that of voluntary facial movements (velocity of the markers at the beginning of these movements  =  mean slope through the start and maximum point of them).

  • Emotional reaction time (ERT) (time (in seconds) between a funny film stimulus and laughing (average onset of correspondent movements of the mouth angles based on five humorous stimuli)).

Using SPSS for Windows (version 11.5), the abovementioned variables were submitted to a multivariate analysis of variance (MANOVA) for group comparisons regarding the mouth angles and the rims of the eye (left, right). The variables (left, right) were correlated (Pearson’s correlation for IV: minimum, 0.65; maximum, 0.96; p < 0.001). For ERT, rp  =  0.98 (p < 0.001) was computed. Post hoc t tests were performed to investigate task specific group differences. Mean VAS scores in patients and controls were compared by the Mann-Whitney U test.

Associations between metric variables (such as age) and kinematic parameters were investigated using Pearson’s correlation. HAMD-21 scores were correlated with kinematic variables by Spearman-Brown correlation. T tests were applied to compare patients who had previously attempted suicide with those who had not. p Values ⩽ 0.05 were considered significant. Alpha correction was applied, if necessary.


The IV of laughing was abnormally low in patients (fig 1A), as shown by MANOVA (table 1) and post hoc tests (mouth angles, p  =  0.02; t test). The IV of eye movements during laughing was an exception (fig 1A) (left eye, p  =  0.05; right eye, p  =  0.56; ANCOVA; covariate, age). Patients also exhibited an abnormally slow IV of voluntary facial movements (fig 1B; closing of the left eye, p  =  0.004; closing of the right eye, p  =  0.001; stretching of the right mouth angle, p  =  0.02; t tests).

Table 1

 Results of analysis of variance for comparison of depressed patients and healthy controls regarding facial parameter scores for the right and left side

Figure 1

 (A) Mean (SD) initial velocity of involuntary (laughing) movements triggered by watching a humorous film (“Mr Bean”) and (B) mean (SD) initial velocity of voluntary movements of the face in 25 patients suffering from depressive disorders and 25 healthy controls, as measured using ultrasonic markers at different positions on the face (right and left angle of the mouth; medial inferior rims of the left and the right eye). Asterisks indicate that healthy subjects conducted involuntary and voluntary movements of the face significantly faster than depressed patients in the experimental situation.

MANOVA revealed no significant group differences in ERT (p  =  0.26).

Patients laughed significantly less often than controls (p  =  0.006; ANCOVA; covariate, age). However, patients and controls did not significantly differ regarding the VAS scores for “degree of funniness” (p  =  0.12; Mann-Whitney U test).

In patients, no significant association between facial and emotional parameters (VAS score, ERT, laughing frequency, IV) and age, age at onset of depression, duration of depression, and number of relapses was found. The effects of previous suicide attempts on facial activity were small. However, patients with previous suicide attempts were characterised by significantly slower IV of movements of the right mouth angle during laughing (mean, 2.41; SD, 1.08 mm/s), compared with patients who had not attempted suicide (mean, 5.25; SD, 3.35 mm/s) (p  =  0.005; t test).

In patients, higher HAMD-21 scores were significantly associated with slower IV, especially regarding the left side (left mouth angle (laughing), r  =  −0.45; p  =  0.03; left eye (closing eyes), r  =  −0.67; p  =  0.006 (also significant after Bonferroni correction for multiple comparisons); and right eye (closing eyes), r  =  −0.56; p  =  0.03).


Depressed patients exhibited an abnormally slow IV of involuntary laughing and voluntary facial movements. Slower reactions on emotional stimuli in patients were more pronounced in the mouth corner region than at the inferior rims of the eyes.

ERT and VAS judgements of the funniness of the sketch cannot explain this phenomenon because they were normal. However, depressed patients laughed significantly less frequently than did controls.

The laughing frequency and IV of laughing movements were not significantly associated in depression. Therefore, they seem to represent different aspects of abnormal facial expression, namely: low susceptibility regarding humorous stimuli and abnormally slow execution of adequate facial reactions.

The transformation of the feeling “funny” into the motor reaction “laughing” was undisturbed in depression (normal ERT). The substrate of this transformation process is thought to be the nucleus accumbens.11 In view of normal ERT, functional disturbances of this structure probably do not occur in depressed patients.

Instead, their general slowing of facial movements may be interpreted as hypomimia (low degree of facial movements), as part of a parkinsonism with subclinical intensity, because the slowing of voluntary and involuntary movements combined with “normal” processing of emotional stimuli parallels the motor–emotional features seen in patients with Parkinson’s disease (PD).12

It can be assumed that basal ganglia dysfunction underlies these disturbances; several studies suggest that basal ganglia disturbances are a relevant factor in the pathophysiology of depression.13 Moreover, similar to patients with PD, decreased dopamine metabolism has been postulated to occur in patients with depression.14

The IV of facial movements was not associated with age and general clinical variables in depression. Instead, the slower IV of laughing movements of the right mouth angle was significantly associated with suicide attempts. This effect should be regarded with caution: the suicidal patients were hospitalised and the suicide attempts were recent. Therefore, low effect and low attention may have affected the results. The severity of depression was correlated with a slower IV, especially with regard to left sided facial movements.

These findings are encouraging, but preliminary. To support them, it will be essential to compare depressed patients directly with patients who have PD.


We gratefully acknowledge Dr B Graf for evaluating the kinematical data as part of her MD thesis (Ludwig-Maximilians-Universität München, Germany).



  • Competing interests: none declared