Dopaminergic control of the striatum for high-level cognition

Dopamine has long been implicated in a wide variety of high-level cognitive processes, ranging from working memory to rule learning and attention switching. Notable progress has been made in the past decades, but the mechanisms underlying effects of dopamine on high-level cognition remain unclear. This article reviews evidence for an important role of the striatum and its interaction with the prefrontal cortex and suggests a variety of ways by which changes in dopamine transmission can bias high-level cognition.

Introduction

Dopamine has long been implicated in behavioral control. Particularly well known are its contributions to reward learning [1, 2••, 3, 4]. More specifically, dopamine has been associated with a class of learning that, in the instrumental domain, echoes Thorndike's habit learning of automatized responses through reinforcement. This form of dopamine-dependent habit learning is thought to be regulated by a model-free system that has been associated with the dorsolateral parts of the striatum. It is defined based on its insensitivity to changes in outcome value and instrumental contingency and is often contrasted with a form of goal-directed behavior that is regulated by a model-based system [5, 6]. Unlike habitual behavior, which is hardwired by reinforcement and directly based on experience, goal-directed behavior involves flexible, forward planning using internal representations (models) of the environment [7] and is directly sensitive to changes in outcome value and contingency [8]. While signals associated with model-based, goal-directed control have been found throughout the brain, including the prefrontal cortex, hippocampus, and dorsomedial striatum [9, 10••, 11, 12••, 13, 14], current formal theories of reinforcement learning offer no obvious role for dopamine in model-based control [5, 15, 16].

This conceptualization of dopamine as serving exclusively model-free behavior is apparently at odds with empirical evidence demonstrating effects of dopamine on high-level cognitive control processes, such as working memory, complex rule learning and attention switching [17, 18••, 19, 20, 21]. Indeed high-level cognitive deficits are core to many dopamine-related disorders, such as addiction [22] and Parkinson's disease (PD) [23]. Performance on tasks that typically involve model-based forward planning, such as the one-touch Tower of London and self-ordered spatial search tests, is sensitive to dopamine manipulation in PD patients [24], healthy volunteers [25], and nonhuman primates [26].

Furthermore, certain effects of dopaminergic drugs on tasks of learning are difficult to account for by modulation of a model-free, habitual system, and rather seem to involve behavior that depends on explicit models of the environment [19, 20, 27]. For example, Cools et al. [28^••] have shown effects of dopamine receptor stimulation and dopamine synthesis capacity on a deterministic form of one-trial reward and punishment prediction learning. Although it is tempting to interpret these effects in relation to the standard framework of model-free reinforcement learning, performance on the task probably does not involve any model-free control, but rather depends on the ability to update explicit ‘cognitive’ predictions of future reward or punishment.

In addition, the body of neurophysiological work that originally inspired the hypothesis that dopamine is involved in model-free, habit learning has recently been extended with new data showing that even the midbrain dopamine neurons themselves encode signals that could support model-based, goal-direct control [29, 30]. For example, Bromberg-Martin and Hikosaka [29] have shown that midbrain dopamine neurons that encode reward expectation also encode information expectation, suggesting that dopamine plays a role not just in reward seeking but also in information seeking.

These different lines of evidence suggest that effects of dopamine go beyond the modulation of model-free, habitual behavior [31], and extend to high-level cognitive processes. In this review, a variety of mechanisms will be addressed by which high-level effects of dopamine may arise. Two factors should be kept in mind. First, midbrain dopamine neurons are known to project to brain regions associated with model-based control, for example, the prefrontal cortex, the dorsomedial striatum and the hippocampus. Second, although there might be separate model-free and model-based systems for behavioral control [5], these systems are unlikely to act in isolation. In particular, the prefrontal cortex is well known to interact with the striatum in part-segregated, part-interactive fronto-striatal circuits [32, 33, 34]. Accordingly, dopamine might affect high-level cognitive function by altering flow through these circuits. Various instantiations of such fronto-striatal circuit effects are discussed.