Estradiol Modulates Effort-Based Decision Making in Female Rats

Disorders of the dopamine system, such as schizophrenia or stimulant addiction, are associated with impairments in different forms of cost/benefit decision making. The neural circuitry (ie

amygdala, prefrontal cortex, nucleus accumbens) underlying these functions receives dopamine input, which is thought to have a central role in mediating cost/benefit decisions. Estradiol

modulates dopamine activity, and estrogen receptors (ERs) are found within this neurocircuitry, suggesting that decision making may be influenced by estradiol. The present study examined the

contribution of estradiol and selective ERα and β agonists on cost/benefit decision making in adult female Long-Evans rats. An effort-discounting task was utilized, where rats could either

emit a single response on a low-reward lever to receive two pellets, or make 2, 5, 10, or 20 responses on a high-reward lever to obtain four pellets. Ovariectomy increased the choice on the

high-reward lever, whereas replacement with high (10 μg), but not low (0.3 μg), levels of estradiol benzoate reduced the choice on the high-reward lever. Interestingly, both an ERα agonist

(propyl-pyrazole triol (PPT)) and an ERβ agonist (diarylpropionitrile (DPN)) increased choice on the high-reward lever when administered independently, but when these two agonists were

combined, a decrease in choice for the high-reward lever was observed. The effects of estradiol, PPT, and DPN were more pronounced 24 h post-administration, suggesting that these effects may

be genomic in nature. Together, these results demonstrate that estradiol modulates cost/benefit decision making in females, whereby concomitant activation of ERα and β receptors shifts the

decision criteria and reduces preference for larger, yet more costly rewards.

Disorders of the dopamine system, such as schizophrenia, Parkinson's disease, or stimulant addiction, are associated with impairments in different forms of cost/benefit decision making

(Rogers et al, 1999; Murphy et al, 2001; Walton et al, 2002; Shurman et al, 2005; Hoffman et al, 2006). One form of decision making that has received considerable attention entails

situations where an organism chooses between options that yield either smaller, easily accessible rewards, or more valuable rewards that may be obtained with considerable more effort. It is

well established that effort-based decision making is critically dependent on mesolimbic dopamine transmission, as systemic treatment with dopamine antagonists or dopaminergic lesions of the

ventral striatum markedly reduces preference for larger rewards associated with a greater effort cost (Salamone et al, 2007; Floresco et al, 2008b; Salamone et al, 2009). Furthermore

different nodes of dopaminergic circuits are important in mediating effort-related judgments, including the basolateral amygdala (Floresco and Ghods-Sharifi, 2007; Ghods-Sharifi et al,

2009), anterior cingulate of the prefrontal cortex (Walton et al, 2002), and nucleus accumbens core (Floresco, 2007; Botvinick et al, 2009).

Studies investigating the neurochemical underpinnings of effort-based decision making have almost exclusively employed male subjects. However, it is important to note that ovarian hormones,

in particular, estradiol, exert complex modulatory control over dopamine (Segarra et al, 2010; Zhao and Becker, 2010; Jacobs and D’Esposito, 2011a) that may in turn alter cost/benefit

evaluations. A wide range of behaviors and neurobiological mechanisms are modulated by estradiol, including neuroprotection of dopamine neurons (Kuppers et al, 2000). Estradiol affects

dopamine autoreceptors, enhances binding on D2 dopamine receptors, and increases excitability of receptors on dopamine terminals (Thompson and Moss, 1994; Becker, 1999; Becker and Hu, 2008).

All of these mechanisms result in enhanced and/or prolonged effects of dopamine. Additionally, the rapid onset as well as the enduring effects of estradiol suggests both genomic and

non-genomic mechanisms mediate estradiol's ability to enhance dopamine transmission (McEwen, 1991; Simoncini and Genazzani, 2003).

Estradiol also exerts dose-dependent effects on cognition mediated by regions thought to be critical for normal decision making including the hippocampus, prefrontal cortex, striatum, and

amygdala (Luine et al, 1998; Galea et al, 2001; Holmes et al, 2002; Wide et al, 2004; Sherwin, 2006; Sinopoli et al, 2006; Barha et al, 2010; Barha and Galea, 2010). Specifically low levels

of estradiol facilitate, whereas high levels impair prefrontal and hippocampus-dependent learning (Holmes et al, 2002; Wide et al, 2004; Sinopoli et al, 2006; Barha et al, 2010). Indeed some

of estradiol's effects on cognition are modulated by its effects on dopaminergic transmission within these regions (Quinlana et al, 2008). Given these findings, estradiol may also impact

effort-related judgments that are critically dependent on mesocorticolimbic dopamine circuitry.

There are two known estrogen receptors (ERα and ERβ) encoded by different genes and are widely expressed in the mammalian central nervous system (Shughrue et al, 1997; Shughrue and

Merchenthaler, 2000; Warembourg and Leroy, 2004). For example, mRNA expression for both ERα and ERβ was found in the female hippocampus, although more expression of ERα mRNA is detectable in

the basolateral amygdala, and only ERβ mRNA in the nucleus accumbens and prefrontal cortex (Shughrue et al, 1997; Shima et al, 2003). Estradiol works genomically and non-genomically on both

ER subtypes, and synergistically with neurotransmitters (ie dopamine) to exert their effects. Many of estradiol's effects on neuronal properties are fast and easily reversible, suggesting

non-genomic actions, whereas the genomic effects typically occur 24–48 h after administration (Woolley, 1999).

Given estradiol's effects on cognitive domains (Galea et al, 2001; Galea et al, 2008; Barha et al, 2010; Barha and Galea, 2010) the interplay of estradiol and dopamine (Chiodo et al, 1986;

Luine et al, 1998; Becker, 1999), and the presence of ERs within key brains regions implicated in cost/benefit decision making (Shimizu and Bray, 1993; Shughrue et al, 1997; Shughrue and

Merchenthaler, 2000; Shima et al, 2003), the present study examined the influence of ovarian hormones in female Long-Evans rats using a well-established effort-discounting task (Floresco et

al, 2008b). In the present study we used intact and ovariectomized subjects and administered either estradiol, an ERα agonist, ERβ agonist, or both and recorded choice preference during an

effort-based decision making task. Given the potentiating effects of estradiol on dopamine along with the presence of ERs within the neurocircuitry implicated in mediating cost/benefit

decision making, we hypothesized that alterations in estradiol activity would exert a modulatory effect on effort-based decision making in female rats.

Adult female Long-Evans rats (210–244 g, N=16; Charles River, Montreal, QC, Canada) were single housed and maintained to ∼85% of free feeding weight, allowing a gain of 3 g per week for

natural growth, and given ad-libitum water. The colony room was maintained on a 12 12 hours light–dark cycle (on at 0700 hours), and testing occurred between 1000–1400 hours. Experimentation

was in accordance with the Canadian Council of Animal Care and was approved by the University of British Columbia's Animal Care Committee.

Two sound-attenuating operant chambers (30.5 × 24 × 21 cm; Med-Associates, St. Albans, VT, USA) were used. Each chamber was fitted with two retractable levers and a food receptacle in

between to allow for food reinforcement (45 mg sugar pellets; Bioserv, Frenchtown, NJ, USA). A single light located in the top-center of the wall opposite the levers illuminated the chamber.

Experimental data were recorded using MedPC software.

Training protocols adapted from Cardinal et al (2000) have been described previously (Floresco et al, 2008b; Ghods-Sharifi and Floresco, 2010). Rats were trained under a fixed-ratio (FR) 1

schedule to a criterion of 50 presses in a 30-min session (one session per day), first for one lever then for the other (counterbalanced left/right between subjects) across 2 consecutive

days. Later training occurred on a simplified version of the full task, where rats were randomly presented with one of the two levers over 90 training trials and one press on the lever

delivered a single sugar pellet. Before the commencement of the trial, the levers were retracted and the house light turned off. Every 40 s, the house light would illuminate and one of the

two levers would be inserted into the chamber. Omissions were scored when subjects failed to respond to the extended lever within 10 s, whereby the lever would retract and the chamber would

darken. However, a response on the extended lever would result in the retraction of the lever and the immediate delivery of a single sugar pellet. The house light remained illuminated for

another 4 s. For each set of trials, the left or the right lever would be presented once randomly. Before moving on to the full task, a criterion of 80 or more successful trials (⩽ 10

omissions) was reached by each subject for at least four consecutive training sessions.

Effort discounting: Figure 1 illustrates the basic procedures in this task. Each day, animals completed a 32 min session consisting of 48 discrete trials, each separated into four blocks.

Each block of trials began with two forced-choice trials. On these trials, only one of the two levers was randomly presented. During the next 10 trials, both levers were presented and rats

had a free choice between the two levers. Throughout the inter-trial state, the chamber was in darkness and both levers were retracted. At 40 s intervals, a new trial began along with the

illumination of the house light, followed by the extension of both levers 3 s later. One lever was designated as the low reward (LR) lever, and the other as the high reward (HR) lever. These

levers were counterbalanced (left/right) between animals, and remained constant for the duration of the experiment. Once the levers were presented, the animal had 25 s to make a response;

failing to do so is counted as an omission, and the chamber was reset. A single press of the LR lever resulted in the retraction of both levers and the immediate delivery of two pellets.

However, after the first response on the HR lever, the LR lever was immediately retracted, and the HR lever remained inserted in the chamber until completion of the required FR of presses.

The FR requirement for the HR lever increased within the session (described below). Upon completion of the FR requirement for the HR, the HR lever retracted, four pellets were immediately

delivered 0.5 s apart, and the house light remained on for another 4 s. The chamber then darkened, and was reset to the inter-trial state.

Effort-based decision-making task. An illustration of the basic procedures in this task. Each session is comprised of 48 discrete trials. A house light comes on, both levers are presented,

and the animal has a free choice between the two levers: the low-reward (LR) lever, and the high-reward (HR) lever. Once presented, the animal has to make a response within 25 s; failing to

do so is counted as an omission, and the chamber is reset to the intertrial state (darkness). A single press of the LR lever resulted in the retraction of both levers and the immediate

delivery of two pellets. However, after the first response on the HR lever, the LR lever is immediately retracted, and the HR lever remains inserted in the chamber until completion of the

required fixed ratio (FR) of presses. Upon completion of the FR requirement for the HR, the HR lever retracts, four pellets are immediately delivered. The chamber then darkens, and is reset

to the intertrial state. Additionally, the FR requirement for the HR lever increases within the session. The FR of lever presses required to obtain the HR increases discretely over the four

blocks of trials, beginning with 2 presses, then 5, 10, and finally 20 presses, respectively.

The FR of the lever presses required to obtain the HR increased over the four blocks of trials, beginning with 2 presses, then 5, 10, and finally 20 presses. On the rare occurrence when a

rat failed to complete the required number of presses on the HR lever within 25 s after its extension, the lever retracted without delivery of food, and the chamber was reset. However, the

animal's choice was still incorporated into the data analysis. The number of omissions and the average rate of lever pressing (number per second) from each trial block were added together

and the total across all four blocks was used for statistical analysis. Training on the task continued until rats as a group (1) chose the HR lever during the first trial block (FR 2) on at

least 70% of successful trials and (2) demonstrated stable baseline levels of choice. Stable baseline performance was determined using a procedure described by (Winstanley et al, 2004) in

which data from three consecutive sessions were analyzed using a repeated-measures ANOVA with two within-subjects factors (day and trial block). If the effect of trial block was significant

at the p