Hebbian theory

Hebbian theory provides an explanation for how neurons might connect to become engrams, which may be stored in overlapping cell assemblies, or groups of neurons that encode specific information. Initially created as a way to explain recurrent activity in specific groups of cortical neurons, Hebb's theories on the form and function of cell assemblies can be understood from the following: Research conducted in the laboratory of Nobel laureate Eric Kandel has provided evidence supporting the role of Hebbian learning mechanisms at synapses in the marine gastropod Aplysia californica. Because synapses in the peripheral nervous system of marine invertebrates are much easier to control in experiments, Kandel's research found that Hebbian long-term potentiation along with activity-dependent presynaptic facilitation are both necessary for synaptic plasticity and classical conditioning in Aplysia californica. While research on invertebrates has established fundamental mechanisms of learning and memory, much of the work on long-lasting synaptic changes between vertebrate neurons involves the use of non-physiological experimental stimulation of brain cells. However, some of the physiologically relevant synapse modification mechanisms that have been studied in vertebrate brains do seem to be examples of Hebbian processes. One such review indicates that long-lasting changes in synaptic strengths can be induced by physiologically relevant synaptic activity using both Hebbian and non-Hebbian mechanisms. ==Principles==

In artificial neurons and artificial neural networks, Hebb's principle can be described as a method of determining how to alter the weights between model neurons. The weight between two neurons increases if the two neurons activate simultaneously, and reduces if they activate separately. Nodes that tend to be either both positive or both negative at the same time have strong positive weights, while those that tend to be opposite have strong negative weights. The following is a formulaic description of Hebbian learning (many other descriptions are possible): :\,w_{ij}=x_ix_j, where w_{ij} is the weight of the connection from neuron j to neuron i , and x_i is the input for neuron i . This is an example of pattern learning, where weights are updated after every training example. In a Hopfield network, connections w_{ij} are set to zero if i=j (no reflexive connections allowed). With binary neurons (activations either 0 or 1), connections would be set to 1 if the connected neurons have the same activation for a pattern. When several training patterns are used, the expression becomes an average of the individuals: :w_{ij} = \frac{1}{p} \sum_{k=1}^p x_i^k x_j^k, where w_{ij} is the weight of the connection from neuron j to neuron i , p is the number of training patterns and x_{i}^k the k -th input for neuron i . This is learning by epoch, with weights updated after all the training examples are presented and is last term applicable to both discrete and continuous training sets. Again, in a Hopfield network, connections w_{ij} are set to zero if i=j (no reflexive connections). A variation of Hebbian learning that takes into account phenomena such as blocking and other neural learning phenomena is the mathematical model of Harry Klopf. Klopf's model assumes that parts of a system with simple adaptive mechanisms can underlie more complex systems with more advanced adaptive behavior, such as neural networks. ==Relationship to unsupervised learning, stability, and generalization==

Because of the simple nature of Hebbian learning, based only on the coincidence of pre- and post-synaptic activity, it may not be intuitively clear why this form of plasticity leads to meaningful learning. However, it can be shown that Hebbian plasticity does pick up the statistical properties of the input in a way that can be categorized as unsupervised learning. This can be mathematically shown in a simplified example. Let us work under the simplifying assumption of a single rate-based neuron of rate y(t), whose inputs have rates x_1(t) ... x_N(t). The response of the neuron y(t) is usually described as a linear combination of its input, \sum_i w_ix_i, followed by a response function f: :y = f\left(\sum_{i=1}^N w_i x_i \right). As defined in the previous sections, Hebbian plasticity describes the evolution in time of the synaptic weight w: :\frac{dw_i}{dt} = \eta x_i y. Assuming, for simplicity, an identity response function f(a)=a, we can write :\frac{dw_i}{dt} = \eta x_i \sum_{j=1}^N w_j x_j or in matrix form: :\frac{d\mathbf{w}}{dt} = \eta \mathbf{x}\mathbf{x}^T\mathbf{w}. As in the previous chapter, if training by epoch is done an average \langle \dots \rangle over discrete or continuous (time) training set of \mathbf{x} can be done:\frac{d\mathbf{w}}{dt} = \langle \eta \mathbf{x}\mathbf{x}^T\mathbf{w} \rangle = \eta \langle \mathbf{x}\mathbf{x}^T\rangle\mathbf{w} = \eta C \mathbf{w}.where C = \langle\, \mathbf{x}\mathbf{x}^T \rangle is the correlation matrix of the input under the additional assumption that \langle\mathbf{x}\rangle = 0 (i.e. the average of the inputs is zero). This is a system of N coupled linear differential equations. Since C is symmetric, it is also diagonalizable, and the solution can be found, by working in its eigenvectors basis, to be of the form :\mathbf{w}(t) = k_1e^{\eta\alpha_1 t}\mathbf{c}_1 + k_2e^{\eta\alpha_2 t}\mathbf{c}_2 + ... + k_Ne^{\eta\alpha_N t}\mathbf{c}_N where k_i are arbitrary constants, \mathbf{c}_i are the eigenvectors of C and \alpha_i their corresponding eigen values. Since a correlation matrix is always a positive-definite matrix, the eigenvalues are all positive, and one can easily see how the above solution is always exponentially divergent in time. This is an intrinsic problem due to this version of Hebb's rule being unstable, as in any network with a dominant signal the synaptic weights will increase or decrease exponentially. Intuitively, this is because whenever the presynaptic neuron excites the postsynaptic neuron, the weight between them is reinforced, causing an even stronger excitation in the future, and so forth, in a self-reinforcing way. One may think a solution is to limit the firing rate of the postsynaptic neuron by adding a non-linear, saturating response function f, but in fact, it can be shown that for any neuron model, Hebb's rule is unstable. Therefore, network models of neurons usually employ other learning theories such as BCM theory, Oja's rule, or the generalized Hebbian algorithm. Regardless, even for the unstable solution above, one can see that, when sufficient time has passed, one of the terms dominates over the others, and :\mathbf{w}(t) \approx e^{\eta\alpha^* t}\mathbf{c}^* where \alpha^* is the largest eigenvalue of C. At this time, the postsynaptic neuron performs the following operation: :y \approx e^{\eta\alpha^* t}\mathbf{c}^* \mathbf{x} Because, again, \mathbf{c}^* is the eigenvector corresponding to the largest eigenvalue of the correlation matrix between the x_is, this corresponds exactly to computing the first principal component of the input. This mechanism can be extended to performing a full PCA (principal component analysis) of the input by adding further postsynaptic neurons, provided the postsynaptic neurons are prevented from all picking up the same principal component, for example by adding lateral inhibition in the postsynaptic layer. We have thus connected Hebbian learning to PCA, which is an elementary form of unsupervised learning, in the sense that the network can pick up useful statistical aspects of the input, and "describe" them in a distilled way in its output. ==Hebbian learning and mirror neurons==

Hebbian learning and spike-timing-dependent plasticity have been used in an influential theory of how mirror neurons emerge. Mirror neurons are neurons that fire both when an individual performs an action and when the individual sees or hears another perform a similar action. The discovery of these neurons has been very influential in explaining how individuals make sense of the actions of others, since when a person perceives the actions of others, motor programs in the person's brain which they would use to perform similar actions are activated, which add information to the perception and help to predict what the person will do next based on the perceiver's own motor program. One limitation of this idea of mirror neuron functions is explaining how individuals develop neurons that respond both while performing an action and while hearing or seeing another perform similar actions. Neuroscientist Christian Keysers and psychologist David Perrett suggested that observing or hearing an individual perform an action activates brain regions as if performing the action oneself. These re-afferent sensory signals trigger activity in neurons responding to the sight, sound, and feel of the action. Because the activity of these sensory neurons will consistently overlap in time with those of the motor neurons that caused the action, Hebbian learning predicts that the synapses connecting neurons responding to the sight, sound, and feel of an action and those of the neurons triggering the action should be potentiated. The same is true while people look at themselves in the mirror, hear themselves babble, or are imitated by others. After repeated occurrences of this re-afference, the synapses connecting the sensory and motor representations of an action are so strong that the motor neurons start firing to the sound or the vision of the action, and a mirror neuron is created. Numerous experiments provide evidence for the idea that Hebbian learning is crucial to the formation of mirror neurons. Evidence reveals that motor programs can be triggered by novel auditory or visual stimuli after repeated pairing of the stimulus with the execution of the motor program. Consistent with the fact that spike-timing-dependent plasticity occurs only if the presynaptic neuron's firing predicts the post-synaptic neuron's firing, the link between sensory stimuli and motor programs also only seem to be potentiated if the stimulus is contingent on the motor program. == Hebbian theory and cognitive neuroscience ==

Hebbian learning is linked to cognitive processes like decision-making and social learning. The field of cognitive neuroscience has started to explore the intersection of Hebbian theory with brain regions responsible for reward processing and social cognition, such as the striatum and prefrontal cortex. In particular, striatal projections exposed to Hebbian models exhibit long-term potentiation and long-term depression in vivo. Additionally, models of the prefrontal cortex to stimuli ("mixed selectivity") are not entirely explained by random connectivity, but when a Hebbian paradigm is incorporated, the levels of mixed selectivity in the model are reached. It is hypothesized (e.g., by Peter Putnam and Robert W. Fuller) that Hebbian plasticity in these areas may underlie behaviors like habit formation, reinforcement learning, and even the development of social bonds. ==Limitations==

Despite the common use of Hebbian models for long-term potentiation, Hebbian theory does not cover all forms of long-term synaptic plasticity. Hebb did not propose any rules for inhibitory synapses or predictions for anti-causal spike sequences (where the presynaptic neuron fires after the postsynaptic neuron). Synaptic modification may not simply occur only between activated neurons A and B, but at neighboring synapses as well. Therefore, all forms of heterosynaptic plasticity and homeostatic plasticity are considered non-Hebbian. One example is retrograde signaling to presynaptic terminals. The compound most frequently recognized as a retrograde transmitter is nitric oxide, which, due to its high solubility and diffusivity, often exerts effects on nearby neurons. This type of diffuse synaptic modification, known as volume learning, is not included in the traditional Hebbian model. == Contemporary developments, artificial intelligence, and computational advancements ==

Modern research has expanded upon Hebb's original ideas. Spike-timing-dependent plasticity (STDP), for example, refines Hebbian principles by incorporating the precise timing of neuronal spikes to Hebbian theory. Experimental advancements have also linked Hebbian learning to complex behaviors, such as decision-making and emotional regulation. Current studies in artificial intelligence (AI) and quantum computing continue to leverage Hebbian concepts for developing adaptive algorithms and improving machine learning models. In AI, Hebbian learning has seen applications beyond traditional neural networks. One significant advancement is in reinforcement learning algorithms, where Hebbian-like learning is used to update the weights based on the timing and strength of stimuli during training phases. Some researchers have adapted Hebbian principles to develop more biologically plausible models for learning in artificial systems, which may improve model efficiency and convergence in AI applications. A growing area of interest is the application of Hebbian learning in quantum computing. While classical neural networks are the primary area of application for Hebbian theory, recent studies have begun exploring the potential for quantum-inspired algorithms. These algorithms leverage the principles of quantum superposition and entanglement to enhance learning processes in quantum systems. Current research is exploring how Hebbian principles could inform the development of more efficient quantum machine learning models. Recent research on Hebbian learning has focused on the role of inhibitory neurons, which are often overlooked in traditional Hebbian models. While classic Hebbian theory primarily focuses on excitatory neurons, more comprehensive models of neural learning now consider the balanced interaction between excitatory and inhibitory synapses. Studies suggest that inhibitory neurons can provide critical regulation for maintaining stability in neural circuits and might prevent runaway positive feedback in Hebbian learning. In 2017, Jeff Magee and colleagues identified behavioral timescale synaptic plasticity (BTSP), a form of learning in hippocampal CA1 neurons in which synaptic inputs active several seconds before or after a dendritic plateau potential are strengthened, even without coincident postsynaptic spiking. This mechanism operates on a much longer timescale than traditional Hebbian or spike-timing-dependent plasticity and provides a means for linking events separated in time during behavior. ==See also==