Viewed as a neural network disorder, mesial temporal lobe epilepsy (mTLE) may cause widespread deficits in human brain functions. Impairments in cognitive functions such as memory and language have been well addressed, but perceptual deficits have only been considered in terms of behavioral data. Little imaging research on perceptual deficits in mTLE has been reported. The present study is expected to reveal impairments in the perceptual networks in patients with mTLE using fMRI. The fMRI-based independent component analysis (ICA) was applied to 33 patients with mTLE and 33 matched healthy controls. In light of the resting-state networks (RSNs) corresponding to the basal functions of visual, auditory, and sensorimotor systems, the ICA data of functional connectivity within these RSNs were compared between the patients and controls. Compared with the controls, the mTLE patients presented decreased functional connectivity within the regions of the auditory and sensorimotor networks, as well increased functional connectivity in the primary visual cortex and decreased functional connectivity in the bilateral MT+ areas of the visual network. Our neuroimaging results are in agreement with the previous findings that specific perceptual functions are impaired in mTLE. Furthermore, our findings in the visual network support the belief that the primary visual function is not impaired and that there may be deficits in the high-order visual function in mTLE. Our fMRI study may contribute to the understanding of neuropathophysiological mechanisms underlying perceptual impairments in mTLE.