Methods

Study design and population

This cross sectional study consisted of two parts. The first part focused on spasticity in the elbow flexor muscles and the second part on spasticity in the knee extensor muscles.

Patients with self-reported spasticity in the upper arm and/or upper leg following an upper motor neuron lesion were recruited from the inpatient and outpatient departments of a local rehabilitation centre. To take part in the study patients had to be able to understand simple commands. The presence of pain or severe contractures of the elbow or knee were exclusion criteria.

The study was approved by the local medical ethics committee. Patients signed informed consent before participation in the study.

Procedure

Each patient was measured by three raters. Patients were measured in random order, with 30–45 min of rest in between two ratings. The raters, three physicians and a physiotherapist involved in daily spasticity care and experienced with using the AS, were instructed in detail about the measurement protocol a few weeks prior to assessment during a group session. Before the actual measurement, the full passive range of motion was assessed by the rater, starting from the position with maximum shortened muscles and rotating to the position of full muscle stretch at a low velocity. Subsequently, raters performed the movement twice at fast velocity throughout the entire range of motion. Raters noted the AS score on a form. In case of doubt, the lower score obtained was documented.

For measurement of the elbow flexors (part 1), patients were seated comfortably in the supine position with the affected arm on a supporting scale, with the shoulder abducted to about 20° (fig 1). The rater rested one hand on the upper arm of the patient and the other on the palmar side of the forearm just proximal to the wrist, simultaneously holding the dynamometer. From the starting position of full elbow flexion, the rater extended the elbow to maximum extension. For measurement of the knee extensors (part 2), patients were positioned lying on the non-tested side, with ±45° of hip flexion on the tested side (fig 2). The raters were instructed to hold the tested leg with one hand just above the knee and the other around the distal part of the lower leg, holding the dynamometer that was fixed ventrally above the ankle. From the starting position of maximum knee extension, the rater flexed the knee fully. The standardised positions were maintained without forceful stretching or discomfort for the patient. Patients were explicitly instructed to relax fully and not to react to stretches. Raters and patients were blinded to the sEMG and dynamometry output and other raters’ scores.

• Download figure
• Open in new tab
• Download powerpoint

Figure 1

Demonstration of the patient's positioning for measurement of the elbow flexors.

• Download figure
• Open in new tab
• Download powerpoint

Figure 2

Demonstration of the patient's positioning for measurement of the knee extensors.

Instrumentation

Surface EMG signals were obtained using bipolar, pre-gelled circular (diameter 10 mm) electrodes (ARBO H93, solid gel), with an inter-electrode distance of 24 mm. A reference electrode was placed around the wrist.

Electrodes were placed on the elbow flexor muscles biceps brachii (BB) and brachioradialis (BR), or on the knee extensor muscles rectus femoris (RF) and vastus lateralis (VL). Sensors on BB, RF and VL were placed according to the electrode placement recommendations of the SENIAM based protocol.20 BR electrodes were placed at one quarter of distance on the line between the fossa cubit and the distal radius (so on the proximal side). A reference electrode was placed at the medial aspect of the contralateral wrist.

For force measurements, a handheld dynamometer (Biometrics M500) was used which was positioned at the palmar aspect of the wrist or anterior aspect of the lower leg, such that the rater could easily hold it during AS scoring. In addition, the elbow or knee angle was registered using a mono-axial electronic goniometer, placed on the lateral side of the joint, to enable the characterisation of the movement performed during measurement by determining the start and end of the movement. All sensors were connected with the Mobi measurement device (Mobi, TMSi, The Netherlands), a blue tooth recording and processing system that allowed temporal synchronisation of the sEMG signals with analogue data from the angle and force sensors. Sample frequency was 512 Hz and data were bandpass filtered at 15–256 Hz using a second order Butterworth filter.

Data analysis and outcome parameters

The AS was scored according to the original scale (score 0–4).4

Outcome parameters were selected in order to reflect: (1) reflex muscle activity on stretch while rating the AS, using sEMG; and (2) the total resistance felt by the raters while rotating the limb, using a dynamometer.

From sEMG recordings, root mean square values (μV) of each muscle during joint rotation were calculated, representing the average muscle activity during stretching of the muscle. Subsequently, the total amount of EMG activity was calculated by integrating the root mean square during the whole joint rotation (area under the curve, AUC_muscle (μV×s)). The latter was considered to give better representation of the intensity of muscle activity during the whole rotation.

Similarly, the area under the force curve was calculated from the dynamometer data, representing the intensity of applied force or the resistance during joint rotation (resistance (N×s)).

Goniometry recordings were used to determine the start and end points of the muscle stretching phase, from which movement characteristics during joint rotation, such as range of movement (°), duration (s) and mean angular velocity (velocity (°/s)), were derived.

The parameters AS score and AUC_muscle were used as primary outcome measures.

Statistical analysis

Descriptive statistics were used for sociodemographic and outcome variables. Mean values of each instrumentally obtained parameter were computed out of the two performed elbow extension or knee flexion movements. In a few cases only one cycle was used for analysis because of failure to perform one of the movements properly, indicated by the rater, or for technical reasons (eg, recording problems).

Construct validity of AS for each rater separately was investigated, by calculating the association between AS and AUC_muscle, using Spearman’s correlation coefficient. Additionally, the correlation coefficient was calculated between AS, resistance and velocity.

Inter-rater reliability of the AS was evaluated first by calculating raw overall agreement between raters. Subsequently, generalised kappa for the three raters was calculated. In addition, the intraclass correlation coefficient (ICC) for absolute agreement was calculated.21 A linear mixed model analysis was performed to obtain insight into the weight of each independent variable (AUC_muscle, resistance and covariate velocity) in explaining the dependent variable AS. In order to explore the variability between raters, the factor rater was also added to the model. The “eyeball test” was used to test the assumption of normally distributed residuals. The percentage of explained variance (first level R²) for the model was calculated according to the formula of Snijders and Bosker.22

For statistical analysis, SPSS 11.5 was used. For calculation of the generalised kappa, Excel software was used. Alpha was set at 0.05 for statistical significance.

Results

In total, 30 patients were included in this study, of whom 19 patients participated in each part. The characteristics of both groups are summarised in table 1. In the first part of the study, all 19 patients were measured by rater 1, 18 by rater 2 and 16 patients by rater 3. Some dynamometry and sEMG data were missing in three patients. In part 2, all 19 patients were measured by raters 1, 3 and 4. Some dynamometry and sEMG data were missing in one patient; dynamometry data were missing in another patient.

AS score “0” was rated in 22% of the measurements, “1” in 44%, “2 in 23%, “3” in 11% and “4” in 1% of the measurements, with comparable distributions in both parts of the study. During slow stretch no severe contractures were found.

Figure 3 shows an example of muscle activation patterns of knee extensors and resistance during AS scoring by rater 1. The medians and ranges of the movement characteristics during scoring for each rater are shown in table 2.

• Download figure
• Open in new tab
• Download powerpoint

Figure 3

Example of surface electromyography, knee angle and dynamometry output during measurement of a patient with hereditary spastic paraparesis, measured by rater 1. The goniometry signal between the vertical lines corresponds to the two consecutive fast knee flexion movements and represents the stretch phase of the knee extensor muscles.

Construct validity

Part 1 (elbow flexor muscles)

The results for each rater are shown in table 3. For all raters AS was significantly moderately associated with resistance. AS was not associated with sEMG parameters, except for rater 2, for whom a positive association was found with AUC_BB, but not with AUC_BR.

Part 2 (Knee extensor muscles)

Table 3 shows the results of the knee extensor measurements. A consistent finding was the marked positive association between AS and resistance across the three raters. In addition, all three raters showed a moderate positive correlation between AS and AUC_VL, rater 3 and 4, and also between AS and AUC_RF.

Reliability

Part 1 (elbow flexor muscles)

Overall agreement of AS scores between raters 1 and 2 and between raters 1 and 3 was 44%. Overall agreement between raters 2 and 3 was 38%. The generalised kappa for the three raters (n  =  16) was 0.20. The ICC for absolute agreement concerning the AS scores of the three raters was 0.58 (95% CI 0.30 to 0.81).

Part 2 (knee extensor muscles)

Overall agreement of AS scores between raters 1 and 3 was 32%. Between raters 1 and 4 it was 53% and between raters 3 and 4 it was 42%. The generalised kappa for the three raters (n  =  19) showed very low agreement: κ = 0.16. ICC for absolute agreement was 0.63 (95% confidence interval 0.39 to 0.82).

The role of angular velocity in the variability between raters

Part 1 (elbow flexor muscles)

Table 3 shows negative associations between AS and velocity in raters 2 and 3 (p<0.01). The association between velocity, sEMG parameters and resistance was explored further (table 4). In all raters, velocity was highly negatively associated with resistance. In rater 2, velocity was also moderately negatively associated with AUC_BB; in raters 2 and 3 with AUC_BR.

Table 5 shows the results of linear mixed model analysis. None of the parameters AUC_muscle, resistance or velocity appeared significantly associated with AS. However, the factor rater significantly affected AS outcome (p<0.05). The model explained 34% of the variance in AS of the elbow flexor muscles.

Part 2 (knee extensor muscles)

The negative correlation between AS and velocity was marked and statistically significant for each of the three raters (see table 3). Table 4 shows a marked and statistically significant negative association between velocity and resistance, except for rater 4. The negative association with muscle activity of the knee extensors was moderate and statistically significant in all cases.

Table 6 shows the results of linear mixed model analysis for the knee extensor measurements: resistance and velocity were significantly associated with AS while AUC_RF and AUC_VL were not. However, the factor rater also showed a highly significant association (p<0.001). This model explained 65% of the variance in AS of the knee extensors.

Discussion

This is the first study investigating both the construct validity and inter-rater reliability of the AS using real-time sEMG and dynamometry recordings. The results of this study show that the methodological characteristics of AS are unsatisfactory for the assessment of spasticity.

Construct validity

The contribution of muscle activity parameters, representing involuntary muscle activity when the muscle is stretched, on the variability in the AS score appeared to be low, in particular in the elbow flexor muscles. The association between the AS and resistance was generally stronger than the association between AS and AUC_muscle. This finding is in agreement with most other studies7 but a recent study on the agreement between various measures of spasticity23 found no consistent relationship of biomechanical measures with the MAS. In the same study, sensitivity of the MAS, when compared with sEMG outcomes, was low.

In present study, the association between resistance and AUC_muscle was also not strong, indicating that even objectively measured resistance is not a good representation of reflex muscle activity. Resistance was calculated as force–time integrals, instead of the more commonly used slope of the force–angle curve.24 Force–time integrals are influenced by the duration of joint rotation, which was assumed to be one of the features that contributed to the eventual score for resistance perceived by the individual rater. Remarkable is the large inconsistency between raters. It appeared that the large variation in angular velocity within and between raters, which was associated highly with both AS and resistance, acts as a confounder in AS assessment.

The results of our study differ significantly from the study of Sköld and colleagues25 who recorded sEMG while scoring the MAS of knee flexors and extensors in patients with spinal cord injuries. Although comparison with the present study is problematic because of differences in measurement procedures, they found that the majority of sEMG parameters were strongly associated with the MAS. However, 50% of the measurements in the study were rated as MAS scores “0”. For calculation of the Spearman rank correlation coefficient, these scores all become assigned to the same rank, causing misleading results and overestimation of the association.

Reliability

There is little consensus in literature and among statisticians about what statistical methods are best to analyse rater agreement.6 The overall agreement between raters gives a general idea about the agreement of the individual scores in the same patient group. In this study, rather low values, up to a maximum of 53%, were found. The kappa coefficient was very low in both upper and lower limb measurements. However, the kappa coefficient can be considered less appropriate for this study because it is very stringent for a scale with five categories.21 A weighted kappa, which takes partial agreement into account, could have been used. It is, however, identical to the ICC when the most commonly used weighing scheme is applied.21 The ICC for absolute agreement between the three raters was 0.58 for AS scoring of elbow flexors, and 0.63 for AS scoring of knee extensors. However, overestimation of the ICC is a major problem when the between subjects variance is high,26 which is likely in this study population. Another limitation of using ICC is that it assumes equal spacing between categories, which is probably not the case in the AS.

Several explanations can help in understanding the findings in this study. Most importantly, raters may differ in defining the specific rating levels of the AS or in the definition of the measured construct itself (ie, spasticity or perceived resistance against passive movement). The actual rating levels of the AS (0–4) can be viewed as an arbitrary categorisation of the underlying construct spasticity, which is a continuous trait. The perceived resistance to passive movement is an aggregate composed of various physical features and weights attached to each feature. Raters may vary in terms of which feature they notice and the weights they associate with each. This is well illustrated by the results of this study. As shown in table 2, the movement characteristics of the rotations differed largely between and within raters. The differences between raters were shown to play a significant role in the eventual AS score, which gives ground for stratified presentation of the results per rater.

There is no standard on how much reliability is “good enough”.21 An ICC of 0.63 can be considered insufficient for use in scientific research, in particular when used as a single outcome measure (eg, see Zajicek and colleagues27). When the test is used for individual judgement, even higher reliability is required.21

Study limitations

For the assessment of the contribution of each variable to the AS, we used a linear mixed model. This model assumes that the dependent variable is a continuous variable, which is not the case for the AS. In addition, sample size for the study was rather small. Nevertheless, for illustration purposes only we considered the model sufficiently robust.

Although there is some heterogeneity in our population, there is no reason to assume that the nature of the upper motor neuron lesion influences the results in this cross sectional study although it might have increased the between subjects variance. Finally, in the upper limb study, some additional error might have been introduced by the fact that gravitational force opposed elbow extension during the first part of the movement and assisted during the second part. In the lower limb study, this effect was reduced due to limb rotation in the horizontal plane. In addition, the brachialis muscle, another important elbow flexor, was not measured with sEMG because of its deep position.

In summary, the results of this study show that the methodological characteristics of the AS are unsatisfactory and that the AS should not be used as a single outcome measure for the assessment of spasticity. It is essential that both researchers and clinicians are very well aware of the limitations of this scale.

Conclusion

Ashworth Scale outcomes appeared to be poorly related to simultaneously measured reflex muscle activity. In its current form, the Ashworth Scale is therefore insufficiently valid and reliable as a measure for spasticity. We should therefore stop using it as a single outcome measure and focus on newer and promising methods, preferably including sEMG applications.