[Paper Study] Fundamentals in Spike Response Model and SpikeProp

1. Spike Response Model (SRM)

In the world of spiking neural networks, Spike Response Model is a more generalized spiking neuron model which takes integrate-and-fire model, firing rate model as its special cases. The simplest SRM model can be formulated as

where hi(t) is the action potential of neuron i at time t. hiext(t) is the external input. tif is the last spiking time of neuron i. wij and dij are the weight and delay from neuron j to neuron i, respectively.

η is the refractory function in SRM, which denotes the repsonse of the firing neuron to its own spike. A typical choice of η can be

ε is the synaptic response function which depicts the effect of an incoming spike on the membrane potential at the soma of the post synaptic neuron. Below is a typical ε function.

2. SpikeProp

SpikeProp is a back-propagation algorithm for spike network. Akin to its ansatz in the traditional rate coding neural networks, it makes use of the gradient-descent method to minimize the error function as described below.

where tja is the actual spike time of neuron j in the output layer, and tjd is the desired spike time.

Using SRM as the neuron model, we can construct a three-layered feed-forward network with multiple delayed synaptic terminals in each single connection. The algorithm can be briefed as below.

(1) Calculate the local error field δj for all neurons in output layer.

(2) For each subsequent layer, calculate δi for all neurons.

(3) For the output layer,

For the subsequent layers,

Reference

[1] W. Gerstner, Time structure of the activity in neural network models, Phys. Rev. E 51 (1995) 738-758.

[2] Sander M. Bohte, Joost N. Kok, Han La Poutre, Error-backpropagation in temporally encoded networks of spiking neurons, Neurocomputing 48 (2002) 17-37.