The present study was conducted to investigate the hypothesis that subjects with MS have a disproportionate impairment in the phonological loop of working memory and whether such impairment is domain-specific. This was assessed by comparing performance in subjects with MS across both the visual (using the Paced Visual Serial Addition Test: PVSAT) and auditory (using the Paced Auditory Serial Addition Test: PASAT) domains. In addition, we assessed the role of processing speed in mediating performance deficits. The results showed that: (1) Although all subjects performed better on the PVSAT versus the PASAT, no disproportionate modality-specific differences were observed in the MS group relative to controls; and (2) the rate of decline across trials in the MS and healthy control groups did not differ on either the PVSAT or PASAT. Implications of the results are discussed within the context of a working memory model.