Brain stimulation reward

In 1953, James Olds and Peter Milner, of McGill University, observed that rats preferred to return to the region of the test apparatus where they received direct electrical stimulation to the septal area of the brain. From this demonstration, Olds and Milner inferred that the stimulation was rewarding, and through subsequent experiments, they confirmed that they could train rats to execute novel behaviors, such as lever pressing, in order to receive short pulse trains of brain stimulation. According to B.F. Skinner, operant reinforcement occurs when a behavior is followed by the presentation of a stimulus, and it is considered essential to the learning of response habits. Olds' and Milner's discovery enabled motivation and reinforcement to be understood in terms of their underlying physiology, and it led to further experimentation to determine the neural basis of reward and reinforcement. In one oft-cited example, in 1972, Heath's subject known as "B-19" reported "feelings of pleasure, alertness, and warmth" and "protested each time the unit was taken from him, pleading to self-stimulate just a few more times". Among ethicists, early "direct brain stimulation" or "psychosurgery" experiments have been criticized as "dubious and precarious (even) by yesterday's standards". In a case published in 1986, a subject who was given the ability to self-stimulate at home ended up ignoring her family and personal hygiene, and spent entire days on electrical self-stimulation. By the time her family intervened, the subject had developed an open sore on her finger from repeatedly adjusting the current. == Brain stimulation reinforcement ==

Early studies on the motivational effects of brain stimulation addressed two primary questions: 1. Which brain sites can be stimulated to produce the perception of reward? and 2. Which drugs influence the response to stimulation and via what mechanism? Investigation of the brain reward circuitry reveals that it consists of a distributed, multi-synaptic circuit that determines both BSR and natural reward function. Often, animals that work to initiate brain stimulation will also work to terminate the stimulation. == Relationship to natural rewards and drives ==

The relationship between BSR and natural rewards (e.g. food, water and copulation) has long been debated, and much of the early research on BSR is focused on their respective similarities and differences. BSR is facilitated through the same reinforcement pathway activated by natural rewards. Self-stimulation can exert robust activation of central reward mechanisms due to more direct action than natural rewards, which initially activate peripheral nerves. Similar to self-administration behavior, responding for intracranial brain stimulation has a highly compulsive component characteristic of an addicted state. BSR is hypothesized to be so effective in establishing compulsive habits due to its more direct activation of the reward pathway, bypassing transmission through sensory pathways in response to natural rewards. Delayed reinforcement following a response for BSR decreases how strongly this behavior is reinforced and to what extent it continues. A delay of one second, for example, between a lever-press and reward delivery (stimulation) can reduce response levels. BSR offers insights into the neural circuitry involved in reinforcement and compulsive behavior. == Anatomy of reward==

Mapping and lesion studies on BSR were designed to determine the location of reward-relevant neurons as well as determine the signal pathways that are directly affected by brain stimulation. The site of intracranial self-stimulation leads to substantially different behavioral characteristics. Sites along the length of the medial forebrain bundle (MFB) through the lateral and posterior hypothalamus, the ventral tegmental area (VTA), and into the pons are associated with the strongest reward effects of stimulation. It is a major target for the dopaminergic projections from the VTA, a group of neurons located close to the midline on the floor of the midbrain. The VTA is the origin of dopaminergic cell bodies that comprise the mesocorticolimbic dopamine system. Its capacity in this regard is equivalent to the medial forebrain bundle (MFB) and ventral tegmental area (VTA). Indirect activation Electrophysiological data suggest stimulation of the MFB or VTA does not directly activate dopaminergic neurons in the mesolimbic reward pathway. These data suggest BSR is facilitated by initial excitation of descending, myelinated neurons, which then activate the ascending, unmyelinated neurons of the VTA. Excitatory, cholinergic inputs to the VTA are thought to play a role in this indirect activation, but the neuroanatomical components of this circuit have yet to be fully characterized. == Intracranial self-stimulation (ICSS) procedures ==

Initial training Since the initial demonstration of BSR by Olds and Milner, experiments in rodents record ICSS responding to quantify motivation to receive stimulation. Subjects undergo stereotactic surgery to permanently implant either a monopolar or bipolar electrode to the desired brain region. Electrodes are connected to a stimulating apparatus at the time of the experiment. The first portion of an ICSS experiment involves training subjects to respond for stimulation using a fixed-ratio 1 (FR-1) reinforcement schedule (1 response = 1 reward). In experiments involving rats, subjects are trained to press a lever for stimulation, and the rate of lever-pressing is typically the dependent variable. which can be manipulated experimentally using the independent variables of stimulation amplitude, frequency, and pulse duration. Each discrete trial consists of non-contingent stimulation at a certain amplitude followed by a brief window during which the animal can respond for more stimulation. Effective currents for BSR elicit responding above a certain rate (3 out of 4 trials, for example). The lowest current the animal responds sufficiently to is deemed the minimum effective current. This is done at a constant frequency, typically at the higher end of the frequency range employed in ICSS studies (140–160 Hz). Frequency-rate responding At a constant minimum effective current, ICSS responding is recorded over a series of trials, which vary in stimulation frequency. Each trial consists of a short priming phase of non-contingent stimulation, a response phase where responses are recorded and rewarded with stimulation, and a short time-out phase where responses are not recorded and no stimulation is delivered. This is repeated for a series of 10-15 different ascending or descending frequencies, in 0.05 log-unit increments, which range anywhere from 20 to 200 Hz. While the amplitude of the stimulation influences which neurons are stimulated, the frequency of stimulation determines the firing rate induced in that neuronal population. Generally, increasing stimulation frequency increases the firing rate in the target population. This is associated with higher ICSS response rates, eventually reaching a maximum level at the maximum firing rate, limited by the refractory properties of neurons. Other factors The independent variables of stimulation train and pulse duration can also be varied to determine how each affects ICSS response rates. Longer train durations produce more vigorous responding up to a point, after which rate of responding varies inversely with train length. This is due to lever-pressing for additional stimulation before the previously earned train has finished. The reinforcement schedule can also be manipulated to determine how motivated an animal is to receive stimulation, reflected by how hard they are willing to work to earn it. This can be done by increasing the number of responses required to receive a reward (FR-2, FR-3, FR-4, etc.) or by implementing a progressive-ratio schedule, where the number of required responses continually increases. The number of required responses increases for each trial until the animal fails to reach the required number of responses. This is considered the "break-point" and is a good indication of motivation related to reward magnitude. Curve-shift analysis Stimulation intensity, pulse duration, or pulse frequency can be varied to determine dose-response functions ICSS responding using curve-shift analysis. This approach generally resembles traditional pharmacological dose-response curve where the frequency of stimulation, rather than the dose of a drug, is examined. This method allows for quantitative analysis of reward-modulating treatments on response rates in comparison to baseline conditions. Lower stimulation frequencies fail to sustain ICSS responding at a probability above chance. Response rates increase rapidly over a dynamic range of stimulation frequencies as the frequency increases, until a maximum response rate is reached. Changes in the rate of response over this range reflects changes in the magnitude of the reward. Rate-frequency, rate-intensity, or rate-duration functions make inferences about the potency and efficacy of stimulation, as well as elucidate how drugs alter the rewarding impact of stimulation. Curve-shift analysis is often used in pharmacological studies to compare baseline response rates to those following drug administration. The maximum response rate during baseline conditions is typically used to normalize data in a frequency-rate curve to a maximum control rate (MCR). More specifically, the number of responses for any given trial is divided by the highest number of responses recorded in a baseline condition trial, which is then multiplied by 100. In an experimental condition, if the MCR falls below 100% at the highest stimulation frequencies, it is thought to reflect an impacted capability or motivation to respond, potentially induced by a drug with sedative or aversive properties. Shifts above 100% of the MCR indicate improved ability or motivation to respond, potentially induced by a drug with rewarding or stimulant properties. Sensitivity of the neural circuitry to the rewarding properties of stimulation is assessed by analyzing left- or right-shifts in the M50, or the frequency at which 50% of the maximum number of responses was recorded. Reaching 50% of the MCR at a lower frequency is characteristic of a left-shift in the frequency-rate curve and sensitization of the reward circuitry to stimulation. An increase in the M50 indicates that a greater stimulation frequency was required to reach 50% of the MCR, and the reward circuitry has been desensitized by the experimental manipulation. Another way of analyzing the frequency-rate curve between control and experimental conditions is to do a linear regression through the ascending data points in a plot of raw data (which has not been normalized to the MCR). The point where y=0, or the x-intercept, is called the threshold frequency or theta zero (θ0). This is the frequency at which ICSS response rates are equal to 0 (and any frequency above this will theoretically elicit ICSS responding). == Modulation with drugs ==

Several major drug classes have been studied extensively in relation to ICSS behavior: monoaminergic drugs, opioids, cholinergic drugs, GABAergic drugs, as well as a small number of drugs from other classes. This effect is generally potentiated following administration of drugs that themselves increase the amount of extracellular dopamine in the nucleus accumbens, such as cocaine, which inhibits re-uptake of dopamine to the intracellular space by blocking its transporter. Conversely, these levels are decreased and the rewarding properties of BSR are blocked following administration of drugs that antagonize dopamine receptors or reduce the amount of extracellular dopamine, by promoting either degradation or re-uptake of the neurotransmitter. While dopamine is generally considered to be the main neurotransmitter implicated in the reward system, it is often not the only neurotransmitter affected by addictive, monoaminergic drugs. Importantly, the circuitry involved in BSR is multi-synaptic and not exclusively dopaminergic. High potency MOR agonists like morphine have a somewhat varied effect on ICSS responding despite having high abuse potential, resulting in both potentiation and depression. The effect these drugs have on ICSS responding has been found to be highly dependent on dose, pretreatment time, and previous opioid exposure. Various studies on the effect of MOR-selective drugs including morphine, heroin, fentanyl, methadone, and hydrocodone have found mixed effects on ICSS responding. Low doses of these drugs have been found to elicit weak facilitation of ICSS, while high doses result in a biphasic ICSS profile, consisting of a higher threshold for ICSS at lower frequencies followed by ICSS potentiation at higher frequencies. Upon chronic administration of high-potency MOR agonists at low doses, there is no tolerance to ICSS facilitation. Opioid receptor antagonists, such as naloxone, can reverse the effects of both opioid receptor agonists on ICSS responding and the potentiating effects of psychostimulants like methamphetamine. Naloxone, which is a competitive antagonist of all opioid receptor sub-types, does not influence ICSS responding when administered on its own. KOR agonism, typically associated with dysphoric states, more consistently results in a depression of ICSS responding. The KOR agonist salvinorin-A, for example, causes an overall decrease in ICSS response rates at lower stimulation frequencies. Repeated administration does not produce tolerance to ICSS depression. Many studies have confirmed that low doses of nicotine result in ICSS facilitation, while higher doses result in ICSS depression. Alcohol has several other pharmacological targets other than GABAergic vectors alone, and its metabolite acetaldehyde can activate the mesolimbic dopamine system, particularly at lower levels of drinking. GABAB receptor agonists and positive allosteric modulators have been found to result in ICSS depression and have been found to inhibit the reinforcing effects of several drugs, including cocaine, methamphetamine, and nicotine, reversing the ICSS facilitation these drugs typically cause. == Clinical and pre-clinical evidence ==

Mechanisms of BSR offer a tool that provides insight into the way the brain governs behavior through motivation and reinforcement, especially in regard to addictive and compulsive behaviors. ICSS studies of BSR have proven to be a robust measure of reward sensitivity, and have potential to help assess the abuse liability of various future therapeutics. Drugs found to prevent ICSS facilitation have potential to be developed and therapeutically implemented to reduce the risk of addictive disorders in a clinical setting. == See also ==