Emotion and decision-making: affect-driven belief systems in anxiety and depression
Abstract
Emotion processing and decision-making are integral aspects of daily life. However, our understanding of the interaction between these constructs is limited. In this review, we summarize theoretical approaches to the link between emotion and decision-making, and focus on research with anxious or depressed individuals that reveals how emotions can interfere with decision-making. We integrate the emotional framework based on valence and arousal with a Bayesian approach to decision-making in terms of probability and value processing. We then discuss how studies of individuals with emotional dysfunctions provide evidence that alterations of decision-making can be viewed in terms of altered probability and value computation. We argue that the probabilistic representation of belief states in the context of partially observable Markov decision processes provides a useful approach to examine alterations in probability and value representation in individuals with anxiety and depression and outline the broader implications of this approach.
Overview
Emotions are an integral part of a person’s internal state and, thus, have profound influences on the choices one makes; yet, our understanding of how emotions interact with decision-making (see Glossary) is surprisingly incomplete [1]. Decision-making integrally depends on the computation of the value of available options [2], which, in turn, are a function of the environment and the internal state of the individual [3]. Recent studies have examined how choices are computed in dynamic environments in which the value of the options changes as a consequence of the actions selected (see [4], for a review). Thus, how emotion interacts with the selection of an option based on value is further complicated by the fact that valuation is highly dynamic and can be a function of the number of available options [5], the individual’s cognitive state [6, 7], effort [8], time [9], information about the options [10], and the presence of other individuals [11].
More recently, there have been a number of attempts to determine and integrate the influence of emotions on decision-making (see [12], for a review). Moreover, significant advances have been made in understanding the behavioral [2], neural [13–15], and computational [16–18] basis of reward-related decision-making. In the light of this recent work, the integration of emotion and decision-making can be conceptualized as a dynamic iterative process aimed to help the individual adapt to his or her environment, taking into account the internal state of the individual, the characteristics and determinants of the valuation process, and the characteristics of the environment.
In this review, we narrow our focus on existing studies of decision-making in populations with emotional problems, but broaden the scope to incorporate recent insights from Bayesian approaches to decision-making. The focus on clinical populations provides insight into how emotions can interfere with decision-making. The adoption of a Bayesian perspective on the interaction between emotion and decision-making extends a previously proposed framework of understanding decision-making as a homeostatic process [3]. More specifically, we propose that affect-driven belief systems profoundly affect the transformation of actions into choices, as well as the modification of future expectations. This modulation by affect-driven belief systems can be cast into a Bayesian learning framework, that is, an iterative dynamic probabilistic system, and can be used to provide a quantitative as well as a heuristic basis for measuring and explaining dysfunctions of decision-making in individuals with affective disorders.
Decision-making and emotion
Decision-making is a process that unfolds over time. This temporal structure can be used to identify three component processes. Specifically, choosing among options initially involves the process of assessing the available options. This is followed by the selection of an option based on the value that has been associated with the option. Lastly, the outcome associated with the selected action is evaluated. The influences of emotions on these specific component processes have begun to be considered only recently. In part, this may be because of the difficulty to incorporate emotions into computational models, on the one hand, while retaining a connection to the rich literature of the phenomenology [19], physiology [20], and psychology of emotion [21], on the other.
Traditionally, emotions have been conceptualized along valence and arousal dimensions (for a review, see [21]). However, others have categorized emotions along different facial expressions [22] or appraisal dimensions [23]. These approaches acknowledge that emotions have both quantitative dimensions (i.e., degrees of feeling good or bad) and qualitative dimensions (i.e., feeling sad or angry). Within a dual-system framework of reasoning [24], emotions are thought to affect the relative extent to which an intuitive, heuristic System 1 and an analytic, deliberate System 2 contribute to decision-making. Moreover, based on a temporal perspective of delineating decision-making into component processes, emotions can be both factors that contribute to the modulation of assessment, selection, and outcome evaluation of options and consequences that emerge from these component processes. For example, a sad mood may overvalue the negative attributes of an option during its assessment, but can also be the consequence of an undesired outcome.
The ground-breaking work of Tversky and Kahneman [25] established that both value and the probability of an outcome are represented as nonlinear functions, such that higher values have decreasing marginal gains, losses are valued greater than gains, low probabilities are overweighted, and high probabilities are underweighted. Several investigators have developed models that conceptualize the effect of emotions as modulating these nonlinear value and probability functions [26–28] within a dual-system framework. For example, Mukherjee [29] developed a model to account for individual differences and emotion influences on decision-making based on experimental findings by Hsee [6] and others [30]. In these experiments, individuals who were instructed to engage affective processes when processing value were less sensitive to magnitude changes of the available options and showed greater distortions of the S-shaped probability weight function [7]. In the computational model developed by Mukherjee [29], the value of an option is obtained from a mixture of System 1 and System 2 processes, which is parameterized by a mixture coefficient quantifying the degree to which affect influences decision-making. This mixture parameter can be conceptualized as individual differences in decision-making dispositions, context-dependent outcome processing, and affective construction of the decision-making situation.
Other investigators have departed more radically from the traditional utility model to explain differences in choices as they relate to emotion processing. Specifically, Kusev [31] has proposed that individuals do not calculate utilities explicitly. Instead, people construct preferences based on their experiences [32]. For example, people overweight small, medium-sized, and moderately large probabilities and they also exaggerate risks. However, neither of these findings is anticipated by prospect theory or experience-based decision research. This suggests that people’s experiences of events leak into decisions, even when risk information is explicitly provided [33]. As a consequence, choices depend strongly on context, the type of options, implicitly the degree of affect associated with these options, and the nature of the presentation of the available options in the decision-making situation. Similarly, Vlaev and colleagues [34, 35] have suggested that individuals do not have a common representation of value across different domains. For example, people will offer to pay more money when stakes are high in a pairing of low versus medium pain than when stakes are low, which is thought to be due to the fact that individuals are not able to compare qualitatively different options or outcomes on a single value dimension. Although these investigators do not deny the evidence provided by descriptive theories of utilities associated with options in decision-making situations, they challenge the universality of the shape of the observed value and probability weight functions and propose a much more dynamic formalism to explain behavioral observations in decision-making situations.
The experimental and theoretical accounts of the influence of emotion on decision-making we have discussed reveal that: (i) emotions can influence the value and weight computation of available options; (ii) these computations are dynamically adjusted based on the environment and the individual’s internal state; (iii) it is not yet clear whether there are valence- or arousal-specific influences on this dynamic iterative process.
Evidence for disruption of decision-making in anxiety and depression
Studies of decision-making in clinical populations are of immense value, because they can help to establish brain-behavior relationships, clarify the nature of dysfunctional process(es) in a disorder group, and point toward the development of potential treatments for disorders. However, when studying decision-making in individuals with a particular disorder, it is important to note that the observed differences between the psychiatric target population and the comparison group are typically the result of a complex set of factors that include pre-disease characteristics, disease-related (e.g., treatments) and disease-unrelated (e.g., life events) developments, and the particular state the individual finds him-/herself in at the time of testing. Moreover, most studies have utilized a limited number of decision-making paradigms to examine anxiety and mood related effects on decision-making. Given these complexities and limitations, it should not be surprising that it can be difficult to link a particular mood or anxiety state to decision-making dysfunctions.
Altered belief systems [36–38] (for a review, see [39]) play a critical role in the conceptualization of both anxiety and depression. In particular, individuals with anxiety disorders show an increased bias towards threat-related content [40] and an intolerance of uncertainty [41]. In comparison, depressed individuals show reduced responsiveness to reward [42] and an increased negative evaluation of self [43]. There is some consensus that anxiety makes individuals more sensitive, and thus more aversive, to options with large negative consequences [44]. This heightened sensitivity to options with large negative consequences can hurt them when an option occasionally associated with a highly negative outcome is actually on average the best option, but may help them on tasks with intermittent large punishments that signal the need for representational overhaul, such as the Iowa Gambling Task (IGT). In particular, individuals with generalized anxiety disorder learn to avoid decisions with high immediate gain but high long-term loss significantly faster than comparison subjects [45]. On this decision-making task, better performance is associated with poorer ability to regulate emotions [46], which may be due to a more prolonged visceral response [47]. However, other groups have found impaired performance on the IGT as a function of both high and low trait anxiety [48, 49]. In particular, high trait anxious individuals generate an increased anticipatory physiological response prior to low reward, low punishment, or advantageous choices. These investigators have argued that anxiety may result in distraction from task-relevant processing, inefficient processing of relevant vs irrelevant cues, and interference from increased verbal processing (ruminations). Post-traumatic stress disorder subjects do not show an impairment of learning the contingencies of the IGT, but show reduced activation of reward-related areas [50], which is consistent with the finding that stress interferes with the acquisition of advantageous response selection on this task [51]. Patients with obsessive compulsive disorder show decision-making dysfunctions that have been attributed to their inability to appropriately process emotions [52]. On the whole, the effects of anxiety disorder on decision-making are complex and may be related to other, more anxiety-specific dysfunctions, such as avoidance of threat-relevant information [53], interference by negative distracters [54], or a general deficit of inhibitory processing in the presence of limited availability of controlled processing resources [55].
Anhedonia [56, 57], that is, the inability to experience pleasure, and deficits in reward-related processing [58] have been considered to be the critical components that contribute to dysfunctions in decision-making in depressed individuals. Consequently, it is not surprising that individuals with depression show reduced responsiveness to reward [42]. On the IGT, individuals with major depressive disorder show poorer performance: fewer advantageous card selections [59], fewer selections of risky card decks [60], and less shifting [61]. Moreover, these individuals select more cards from decks with high frequency, low-magnitude punishment contingencies [61]. Surprisingly, other research has showed that acutely depressed individuals make better choices relative to controls or those recovering from depression [62], which is consistent with the finding that depressive individuals learn to avoid risky responses faster than control participants [60]. In general, depressed individuals appear to experience an increase in decisional conflict in a number of different decision-making situations [63, 64], attenuated processing of counterfactual outcomes [65], and a prolonged attenuation of temporal discounting of rewards [66]. In addition, individuals with depressive symptoms fail to develop a response bias towards rewarded stimuli [67–70], in tasks in which which subjects must categorize a briefly presented stimulus as belonging to category A or B. In comparison, manic, depressed, and euthymic bipolar subjects select significantly more cards from risky decks and prefer decks that yield infrequent penalties over those yielding frequent penalties [71]. The percentage of trials on which subjects choose the more likely of two possible outcomes is also significantly impaired in depressed bipolar patients [72]. Others have reported that bipolar manic individuals show impaired [73] or erratic [74] decision-making ability, which has been termed suboptimal [75] decision-making. Depressed and manic individuals show slower deliberation times, a failure to accumulate as many points as controls, and suboptimal betting strategies [75]. These findings point towards both trait-dependent (i.e., disorder related and presumably long-term) and state-dependent (i.e., internal state and mood-related) decision-making dysfunction. This dysfunction may be due to decision-specific alteration in value-related, probability-related, or temporal processing of available options.
In sum, anxiety, depression, and mood swings exert complex effects on decision-making as measured by performance on the IGT. Specifically, increased sensitivity to losses, attenuated processing of reward, and differential selection based on the history of rewards and punishment point towards emotion-related modulation of both value and probability in anxiety and depression. In the following section, we integrate these findings with a computational approach aimed at quantifying belief systems.
Understanding the influence of affect-driven belief systems on decision-making in anxiety and depression
Bayesian models of decision-making and control
In recent years, the understanding of the behavioral and neural processes underlying decision-making has benefited significantly from neuroeconomic models [2, 76, 77] and reinforcement learning models [13, 78, 79]. These approaches have provided insights into how individuals quantify the value of options, what brain systems play a key role in this process, as well as how the underlying neural substrates give rise to behavioral phenomena. However, one critical aspect of goal-directed action selection that does not typically receive explicit treatment in neuroeconomic or reinforcement learning models is the subjective uncertainty individuals have about both the state of the world and the eventual consequences associated with the different action choices in that state. Uncertainty arises from a multitude of sources, for example, noisy sensing, imperfect motor control, neuronal communication errors, intrinsic stochasticity, and non-stationarity in the environment. Moreover, selection and action may lead to complex or indirectly observable changes in the state of the world or that of an individual in the world, in addition to any direct reward/punishment consequences. There are two ways in which uncertainty complicates decision-making: one is the representation and propagation of imprecise information, the other the translation of that information into a goal-directed course of action via a decision policy.
In the past decade, significant progress has been made in understanding how the brain represents uncertain information with the help of Bayesian statistical models, which offer a mathematically precise language for describing probabilistic knowledge, as well as normative procedures for computation based on such knowledge. In particular, Bayesian models provide a way of formalizing how beliefs are linked to observations to arrive at estimations of the status of the world, which guide the decision-making process. In the context of Bayesian models, emotions can be conceptualized as modulating the mathematical representation of these beliefs and the processing of observations. As a generalization of the classical notion of an ideal observer in psychology [80], Bayesian models have helped to demonstrate that humans perform close to Bayesian ideal (optimum) in a number of simple experimental tasks (e.g., multimodal cue integration [81–83], reward learning [84], attentional selection [85], and motor adaptation [86]) and shed light on the neural representation of value and uncertainty [84, 87].
However, even more revealing than when the brain performs optimally is when it does not. Idiosyncratic failures of the brain to represent/process probabilistic information, by making certain implicit statistical assumptions (biases), can reveal important neural design principles that underlie intelligent behavior. For example, it has been shown that the apparently irrational tendency for subjects to extract transient stimulus patterns in a truly random stimulus sequence may be due to the inappropriate, reflexive engagement of neural mechanisms necessary for adapting to changing environmental contingencies [88].
One approach to link conceptually the emotional state of the individual to his or her decision-making is to assume that feelings affect the way probabilities of gains or losses are transformed to weights and values (see Figure 1 and its description for an example). In the context of Bayesian statistical models, this approach translates into modifications of belief (prior probabilities) and evidence (likelihood) as a function of mood states. A number of different approaches have been developed to formalize this link between belief/evidence and mood states. Among these are partially observable Markov decision processes (POMDPs, see Figure 2 for a schematic description) [89, 90]. In this approach, individuals do not have precise knowledge of the state in which they are; instead they maintain a probability distribution of being in a particular state, which is the result of previous experiences. The probability distribution of belief states can be used to estimate the expected reward or cost associated with an action and can, thus, be used to determine how good or bad an action is. Observations (or outcomes) resulting from the selected action, in turn, update the probability distribution of the belief states via a so-called belief-state estimator. The updating occurs using the standard Bayes’ rule. The optimal decision is determined by the distribution of the current belief states, which, in turn, is determined by the probability of transitioning from one state to another via a particular action and the probability of observing an outcome when selecting a particular action. In addition, the optimal decision is determined by the nature of the costs and rewards, the degree to which the decision-maker looks ahead and estimates costs and rewards of subsequent actions, and the probability of experiencing a cost or reward when transitioning from one state to another. A rich computational [89, 91], applied [92], and even neural-systems level [93] literature is emerging based on this heuristic scheme.
This theoretical approach to the substrates of information representation and goal-directed decision-making can be fruitfully applied to understanding the influence of emotions on decision-making. The experience of an outcome and, in particular, the differences between the expected and observed outcome provide an opportunity to improve one’s beliefs about the consequences (value) of the available options and adopt a better decision policy in the future. Huys and Dayan [94] have developed layered notions of control in order to formalize dysfunctions of these processes in the context of decision-making for individuals with anxiety or depression. In this framework, three layers of control are constructed to explain the relationship between emotional dysfunction and decision-making: (i) beliefs about the control of the reliability of outcomes (i.e., how likely a specific outcome is), which can be quantified by the action-dependent outcome entropy; (ii) the degree to which an action is associated with a specific outcome, which can be quantified by assessing the precision of the action-outcome relationship in transition probabilities or the entropy of the transition probability matrix; (iii) the degree to which desirable outcomes can be achieved via reinforcement learning processes, which is a function of the entropy and value of the marginal outcome distribution. Huys and Dayan suggest that controllability of reinforcement is a critical factor of decision-making dysfunction in depression.
A conceptual framework for the influence of affect-driven belief systems on decision-making in anxiety and depression
Within the context of Bayesian inference, POMDPs (Figure 2) provide a particularly useful heuristic framework to examine alterations in decision processes due to altered affect-driven belief systems. Specifically, the belief-state estimator represents processes that determine an individual’s expectations of the current state. The probabilistic formalism can support the general hypothesis that individuals with anxiety and depression represent information and action outcomes in such a way that probabilities are misrepresented and outcome values are biased. Thus, in both disorders, transformation of probabilities into decision weights may result in greater weighting of low probability events, as suggested by Mukherjee [29]. Moreover, the altered valuation process that leads to attenuated sensitivity to magnitude changes of the value of available options [6] can contribute to inappropriate valuation of expected outcomes. As a consequence, altered reward perception affects the belief-state estimator processes. More importantly, the process of updating belief states via Bayes’ rule may also be affected, due to suboptimalities in probabilistic inference and value function computation/comparison, and/or the representation of priors or the likelihood function. Consequently, an anxious or depressed individual may select options based on an altered representation of the current belief states, which can be the result of biased transition probabilities due to altered processing of costs and rewards. For example, low probabilities with high threat or cost value may be over-represented in the decision-making processes of an anxious individual, which may aid performance in a situation with infrequent but highly costly outcomes such as the IGT.
In addition to updating the current belief states, individuals also have to determine how far ahead they need to look when computing the optimal policy in a decision-making situation. In anxious individuals, intolerance of uncertainty may lead to avoidance of looking ahead [53], therefore rushing into action with suboptimal choices, which is also consistent with an increased susceptibility to interference by negative distracters [54]. In depressed individuals, the attenuated ability to represent the differential value of available options may affect decision-making when the outcomes of options are very similar. Moreover, these individuals may not be able to adequately update the value structure and, therefore, show diminished learning and updating of the belief systems that provide the basis for making optimal decisions, which is consistent with failure to learn a bias towards rewarded stimuli [67–70] and with the use of suboptimal betting strategies [75].
In sum, altered probability and value processing can have profound effects on the current belief system of the individual’s state. Affect-driven belief systems can lead to a misrepresentation of the chances and potential outcomes that an individual perceives when selecting an action. For example, the avoidance of the initiation of a social interaction by an anxious individual may be guided by an over-representation of the state that leads to a rejection by the other individual. Similarly, the difficulty to select actions associated with rewards by a depressed person may be due to the attenuated effects of the observed rewards, which leads to under-representation of the state associated with actions leading to rewarding outcomes. However, the precise effects of emotion on decision-making, in general, and in the case of anxiety or depression, in particular, need further empirical study.
Concluding remarks
The integration of emotion processing and decision-making is challenging on a number of different levels (Box 1). Conceptually, emotions have been defined along arousal and affective valence dimensions, which are difficult to integrate within value and probability frameworks of decision-making. Phenomenologically, emotions are often viewed as highly introspective, whereas decision-making has been quantified along a few external variables. Computationally, emotion processing has been conceptualized in terms of associative networks, but also as a result of conditioning processes, whereas decision-making has been conceptualized as a neuroeconomic process. Pathologically, emotional disorders have been approached from the perspective of altered view of self, belief, and conditioning, whereas decision-making dysfunctions have been described in terms of altered preference structures. The Bayesian approach we have proposed in this article may help to develop a common framework for emotion processing and decision-making, by providing a quantitative approach to measuring the influence of emotions on choices (and vice versa) in terms of altered belief states.
Glossary
Bayes’ rule | a mathematical rule that relates the odds of event A1 to event A2, before and after conditioning on event B. Essentially, Bayes’ rule explains how existing beliefs change in the light of new evidence |
Bayesian statistical model | a theoretical approach whereby prior probability distributions, which quantify beliefs, are modified by likelihood functions, which result from observations, to form posterior probabilities via the Bayes’ rule [95] |
Decision-making | a complex process of transforming options into actions by making choices based on some metric that represents the importance of these options to an individual [2] |
Dual-system theory, a theory of human reasoning, which holds that individuals engage two different systems when processing options in a decision-making situation [96] | System 1, which is conceptualized as automatic, unconscious, implicit and associative, and System 2, which is viewed as explicit, rule-based, rational and analytic |
Neuroeconomics | a field that studies the processes that connect the assessment of options to the selection of actions, based on the conceptual approaches developed in economics and the mechanistic frameworks used in neuroscience [77] |
Iowa Gambling Task (IGT) | a simple decision-making task that pits options with high short-term gains associated with long-term losses against options with low short-term gains but higher longer-term gains. The task is thought to rely on value computation in the ventromedial prefrontal cortex [97] |
Partially Observable Markov Decision Process (POMDP) | an agent-based decision process in which the precise current state is unknown and, therefore, a probability distribution over the set of possible states is maintained. The probability distribution is modified according to Bayes’ rule based on a set of observations [90] |
Prospect theory | a formal description of how individuals transform probabilities into weights and losses or gains into values to estimate the utility of an option in a decision-making situation [98] |
Reinforcement learning | the processes of behavioral change that underlie how an individual decides in an environment so as to maximize rewards [79] |
Utility | the degree of preference that an individual assigns to an option in a decision-making situation [99] |
Footnotes
Publisher’s Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
References
Published online 2012 Aug 13. doi: 10.1016/j.tics.2012.07.009