Implantable Loop Recorder

Combining SNNs with Filtering for Efficient Neural

Decoding In Implantable Brain-Machine Interfaces

Zhou Biyan, Pao-Sheng Vincent Sun, and Arindam Basu∗

B. Zhou And P.S.V. Sun Have Contributed Equally

∗Author to whom any correspondence should be addressed

Arxiv:2312.15889V2 [Cs.Lg] 22 May 2025

Combining SNNs with Filtering for Efficient Neural Decoding in Implantable Brain-Machine Interfaces2 Abstract. While it is important to make implantable brain-machine interfaces (iBMI) wireless to increase patient comfort and safety, the trend of increased channel count in recent neural probes poses a challenge due to the concomitant increase in the data rate.

Extracting information from raw data at the source by using edge computing is a promising solution to this problem, with integrated intention decoders providing the best compression ratio.

Recent Benchmarking Efforts Have

shown recurrent neural networks to be the best solution. Spiking Neural Networks (SNN) emerge as a promising solution for resource efficient neural decoding while Long Short Term Memory (LSTM) networks achieve the best accuracy. In this work, we show that combining traditional signal processing techniques, namely signal filtering, with SNNs improve their decoding performance significantly for regression tasks, closing the gap with LSTMs, at little added cost.

Results With Different Filters Are Shown

with Bessel filters providing best performance. Two block-bidirectional Bessel filters have been used–one for low latency and another for high accuracy. Adding the high accuracy variant of the Bessel filters to the output of ANN, SNN and variants provided statistically significant benefits with maximum gains of ≈5% and 8% in R2 for two SNN topologies (SNN Streaming and SNN 3D). Our work presents state of the art results for this dataset and paves the way for decoder-integrated-implants of the future.

1. Introduction

Implantable Brain-Machine Interfaces (iBMI) (Figure 1(a)) are a promising class of assistive technology that enables the reading of a person’s intent to drive an actuator . It holds promise to enable paralyzed patients to perform activities of daily living with partial or total autonomy . While the first applications were in motor prostheses to control a cursor on a computer screen , or wheelchairs , or robotic arms , recent studies have shown remarkable results for speech decoding [5, 6], handwritten text generation and therapies for other mental disorders .

The majority of clinical iBMI systems have a wired connection from the implant to the outside world that restrict’s the user’s mobility. Many studies have found user independence to be a top priority for patients . In addition, the wired connection also entails a risk of infection leading to an increasing interest in wireless neural interfaces [1, 10, 11]. Another trend in the field has been the constant increase in the number of electrodes to increase the number of simultaneously recorded neurons (Figure 1(b)) which can increase the precision of decoding the intent of the user and enable dexterous control. The recently developed Neuropixels technology has increased the number of Combining SNNs with Filtering for Efficient Neural Decoding in Implantable Brain-Machine Interfaces3 recorded neurons to ≈1000. This can be problematic for wireless implants due to the conflicting requirements of high data rate and low power consumption . Hence, there are efforts to compress the neural data on the sensor by extracting information from it by edge computing (Figure 1(c,d)).

Different degrees of computing can be embedded in the implant, from spike detection, classification, to decoding .

When The Occurrence Of Spikes Is Only Of

interest, spike detection methods can be used to only transmit the occurrence of spikes and can provide compression rations between 100-1000x (depending on firing rate and signal to noise ratio) compared to the conventional data rate [14, 15]. However, this method removes all waveform information that could enable the detection of source neurons necessary in neuroscientific experiments. Spike sorting is another compression method in which in addition to the spike, an identifier of the firing neurons is also sent, resulting in slightly reduced compression rates [14, 16].

Ideally, Decoding On

implant can provide the maximum compression with the added benefit of patient privacy since the data does not need to leave the implant—only motor commands are sent out. Figure 1(c) compares the transmission data rate of three methods [17, 18] assuming 100/1000/10000 channels, neural firing rate of 100 Hz, two classes and decoder output rate of 250 Hz. As the number of channels increases, decoding offers the best compression, as its output data rate is fixed (albeit at the cost of increased decoder complexity). Traditional decoders have used methods from statistical signal processing such as Kalman filters and their variants [19, 20]. With the rapid growth of Artificial Neural Networks (ANN) and variants for many different applications due to their natural ability to model nonlinear functions and availability of special hardware for training, it is natural to explore the usage of such techniques for motor decoding and several such works have recently been published .

To fit on the implant, the decoder has to be extremely energy and area efficient, along with being accurate. Some specialized decoder integrated circuit using ANNs have been developed to achieve this purpose . Brain-inspired SNN are supposed to be even more energy-efficient due to their event-driven nature . They are also expected to be better at modeling signals with temporal dynamics due to their inherent “stateful” neurons with memory. However, detailed comparisons between SNN and ANN variants with controlled datasets and benchmarking procedures have been lacking. A recent effort has put together a benchmarking suite to address this gap and one chosen task is that of motor decoding. We use the same dataset for benchmarking and show additional results for more control cases.

Neurobench showed that streaming SNNs provide a good tradeoff in terms of accuracy vs computes while other methods could have similar memory footprint. Another recent work using the same benchmarking suite showed recurrent networks providing the best results with LSTMs outperforming SNNs in terms of accuracy. We make the following novel contributions in this paper: • We improve SNNs for regression by filtering the output using a Bessel filter from signal processing and close the gap in accuracy with LSTMs.

Combining SNNs with Filtering for Efficient Neural Decoding in Implantable Brain-Machine Interfaces4

(C)

Figure 1: (a) Conceptual figure of an iBMI that reads user intent and controls an effector. (b) Trend of exponential increase in the number of simultaneously recorded neurons . (c) Transmission data rates for the different cases (Raw data, spike sorting, decoding).

While spike sorting provides some compression, decoding on the implant can provide the best option, especially as the number of electrodes increases beyond 1000. (d) Integrating computing in the implant can reduce the wireless datarate enabling scalability of iBMI systems.

• We demonstrate that filtering the outputs of both ANN and SNN decoders with block Bidirectional Bessel filters improves decoding accuracy. • We demonstrate state-of-the-art decoding accuracy on this benchmark using LSTMs and filtered SNNs which occupy the higher and lower ends of the pareto optimal curve respectively.

• We show the effect of increasing training data that shows which models have potential for improvements in future. • We show improved accuracy when trained with better curated data; this points to an automated method of trial selection for training neural networks for neural decoding.

The rest of the paper is organized as follows. The following section discusses some of the related works while Section 3 describes the dataset, models and pre-processing used in this work. Section 4 presents the results comparing different models in terms of their performance-cost tradeoff using pareto curves. This is followed by a Section 5 that discusses the main results and provides additional control experiments. Finally, we summarize our findings and conclude in the last section.

Combining SNNs with Filtering for Efficient Neural Decoding in Implantable Brain-Machine Interfaces5

2. Related Works And Contribution

The current work on designing decoders for motor prostheses can be divided into two broad categories–those using traditional signal processing methods and more recent ones based on machine learning.

2.1. Traditional Signal Processing Decoders

An early decoder used in BMI system is the linear decoder, such as population vector (PV) algorithm . Optimal linear estimators (OLE), generalized from PV algorithm, has comparable performance in closed-loop BMI systems, Whereas Bayesian algorithms perform better . Inspired by estimation and communication theory, Wiener filter improved linear decoders by combining neuron history activation .

Kalman filter has an outstanding ability to cope with dynamic and uncertain environments and is suited in real-time applications. That makes Kalman filter one of the most widely used decoding algorithms in iBMI systems. However, the conventional Kalman filter is only optimal for linear variables and Gaussian noise .

Many

variants of Kalman filter have been proposed to be applied to different applications or environments, such as decoding for cursor movement , predicting the movement for clinical devices , controlling the robotic arms , speech decoding .

2.2. Machine Learning Decoders: Algorithms

Machine learning is widely used in various applications due to its powerful ability to process complex data. An SVM decoder could be trained to analyze rhythmic movements of Quadriplegia patients , or motor control of paralyzed limbs .

Recently, ANNs have attracted much attention among machine learning algorithms and have made great progress in BMI decoding. ELM-based intelligent intracortical BMI ( i2 BMI ) achieves an outstanding performance compared to traditional signal processing decoders . A multi-layer ANN is trained to decode the finger movement running in a real-time BMI system, which outperforms a Kalman filter .

Recurrent neural networks (RNN) were introduced since they are more skilled at capturing the relations between two variables using a hidden state with memory. For instance, there have been studies on decoding speech and on brain representation for handwriting . Long-term decoding achieved higher performance by using LSTM and Wiener filter . To decode speech for a paralyzed person, a natural-language model and Viterbi decoder are used .

Neuromorphic algorithms have emerged as an energy-efficient decoder and an effective tool for data compression . SNN is a brain-inspired neural network popular in neuromorphic applications due to its low energy. It can achieve nearly the same accuracy as ANN but with less than 10% memory access and computation of ANN .

Similarly, it was found that the SNN decoders could use far fewer computes compared to ANN, but with a performance penalty in accuracy, for the motor prediction of primates Combining SNNs with Filtering for Efficient Neural Decoding in Implantable Brain-Machine Interfaces6 in the Neurobench benchmark suite . Another recent work showed that SNNs offer an advantage of low-latency which is essential for closed-loop neuromodulation.

In this work, we show that the combination of traditional signal processing filters with SNNs results in one of the best decoders for iBMI systems.

2.3. Machine Learning Decoders: Hardware

Specialized hardware implementations are needed to fit machine learning decoders within the strict area and power budgets of implants. One of the earliest works used a hardware-algorithm co-design approach to exploit statistical variations in analog circuits to make a sub-microwatt decoder based on extreme learning machine, a variant of reservoir computing algorithms. It also used a configurable digital processing second stage to program distinct weights learned for each chip.

More Recent Work Has

used general purpose M0 processor and digital matrix acceleration units, where the power efficiency stems from usage of special features called spike band power (SBP) that require much lower sampling rate than conventional spike detection. While these earlier works demonstrated promising decoder hardware, they were not integrated with neural recording amplifiers as a system on chip.

This Has Been A Recent Focus

where multiplexed neural recording front-end circuits are integrated with a 31-class decoder for brain to text applications. In this case, the classifier was as simple linear discriminant analysis (LDA) but its performance is enhanced by a preceding feature extraction module that extracts distinct neural codes. While all of these works used traditional ANNs, one example used a SNN decoder to perform a closed loop decoding task of moving an object to a desired location using rodents. Based on the decoded outputs in each step, intra-cortical micro stimulation was delivered to close the loop and allow error correction over multiple time steps. This was a multi-component system not optimized for power dissipation. There is significant opportunity to improve SNN hardware and integrate with neural amplifiers to create a system on chip.

• D: Dropout Rate

• TGT: The fixed time interval for ground truth labels. Combining SNNs with Filtering for Efficient Neural Decoding in Implantable Brain-Machine Interfaces7

Trajectory

Figure 2: The experiment in the dataset has the NHP controlling the cursor and moving it to the target location. Once the NHP completes the action (referred to as a reach), the target location will move to a new location, and the subject will move the cursor accordingly.

3.1. Dataset

The primate reaching dataset chosen for this paper was gathered and released by , with the six files chosen for Neurobench being the files of interest. These six files are recordings of two non-human primates (NHP) (Indy and Loco), where each NHP accounts for three files (more details about this choice in ).

This dataset contains microelectrode array (MEA) recordings of the NHP’s brain activity while it is moving a cursor to the target location, as seen in Figure 2. The finger velocity is sampled at fGT = 250 Hz resulting in ground truth labels at a fixed interval of TGT = 4 ms. The target position changes once the monkey successfully moves the cursor to the intended target. We refer to this action as a reach. The dataset contains a continuous stream of the brain’s activity from one MEA with Nch = 96 probes (Indy) or two MEAs with Nch = 192 probes (Loco). In this work, we ignore sorted spikes since it has been shown that spike detection provides sufficient information for decoding and is more stable over time. Hence, the number of probes Nch is the input feature dimension N0 for the neural network models (except ANN 3D) that will be discussed in the following subsection.

Training NN models on time series-based data requires the data to be split apart into separate segments. In analogy with keyword spotting , each segment of neural data should correspond to separate keywords. By using the target positions in this dataset, we can separate the spike data into segments based on indices in the target position array where there is a change in values, as illustrated in Figure 3.

Such

consecutive indices forms the beginning and end of a reach, and then we can split the time series into training, validation, and test sets based on the number of reaches. The split ratio used in this paper follows that of Neurobench , which is 50% for the training set, and 25% each for validation and test sets. The total number of reaches recorded in each file can be seen in Table 1.

Combining SNNs with Filtering for Efficient Neural Decoding in Implantable Brain-Machine Interfaces8

0.6

...

-1.5 -1.5

...

(B)

Figure 3: How each reach is defined in this work: a) The start and end of a reach are marked by the index where there is a change in the target location array, indicating the monkey has moved the cursor to the previous target location.

B) A Sample

segment taken from the file indy 20160622 01, where we can see five consecutive reaches being segmented.

3.2. Network Models

To explore the potential of various neural network models as the neural decoder, five different model architectures with and without memory are tested: ANN, ANN 3D, SNN 3D, Streaming SNN and LSTM, which can be seen in Figure 4. These five models use NN architectures popular as neural decoders (e.g. ANNs used in , SNNs used in and LSTMs used in ) and have memory at the input layer or hidden layer. Every model except for LSTM has two versions of varying complexity (explained in Section 4.1) where complexity refers to the model size indicating the number of neurons. The larger model is henceforth referred to as the base model while the smaller model is dubbed the tiny variant. It was found that networks deeper than 3 layers performed poorly and hence deeper models were excluded from this study.

(I) Ann Or Ann 2D

The ANN model has an architecture of Nch −N1 −N2 −2, with rectified linear unit (ReLU) as the activation function for the first two-layers as well as batch normalization to improve upon the accuracy obtained by the model. Note that N0 = Nch indicates one feature extracted from each probe obtained by summing Combining SNNs with Filtering for Efficient Neural Decoding in Implantable Brain-Machine Interfaces9

Scaling (2)

Figure 4: Architecture of models used in this paper are a) ANN b) ANN 3D c) SNN 3D d) SNN Streaming and e) LSTM. the neural spikes over a fixed duration of TW as described in Section 3.3. Also, N3 = 2 corresponds to predicting the X and Y velocities. A dropout layer with a dropout rate of 0.5 is also added to the first two layers to help regularize the model.

In analogy with the naming convention of ANN 3D introduced next, this model can also be referred to as ANN 2D due to the shape of the input weight tensor.

(Ii) Ann 3D Or Ann Flat

The architecture of the ANN 3D or ANN flat model is m × Nch −N1 −N2 −2, i.e. it shares an identical architecture with ANN, except at the input layer. This model divides the TW duration of the input bin window into m sub-windows and creates a m-dimensional feature from each probe by summing spikes in each sub-window.

This mode of input will be further explained in Section 3.3. The input will then be flattened across the sub-windows, yielding a final input dimension of Nch × m; hence, the number of weights/synapses in the first layer is m times more than ANN.

It is referred to as ANN flat in ; we refer to it as ANN 3D here in keeping with the shape of the input weight tensor, which we feel is more intuitive.

(Iii) Lstm

The LSTM model contains a single LSTM layer of dimension NLSTM, followed by a fully-connected layer of dimension 2. The input of the model shares the same pre-processor as ANN (summing spikes in a bin-window of duration TW); however it uses a different TW. The input is first normalized with a layer normalization, before passing through the rest of the network.

(Iv) Snn 3D Or Snn Flat

The SNN 3D aims to achieve high accuracy and shares a similar architecture with ANN (Nch−N1−N2−2), with the following differences: 1) Instead of using standard activation like ReLU, the SNN 3D model uses the leaky integrate-and-fire (LIF) neuron after every fully-connected layer, 2) the input is first passed through layer normalization, similar to LSTM due to the recurrent nature of LIF, 3) at the final layer there is a scaling layer applied to the output LIF neurons and 4) the input Combining SNNs with Filtering for Efficient Neural Decoding in Implantable Brain-Machine Interfaces10 spikes are processed using the sub-window method similar to ANN 3D to capture finer temporal details of the spike rate variations. However, the dimension of the input layer is not m × Nch like ANN 3D since in this case, the spikes from the m sub-windows are fed over m time steps to Nch neurons in the first layer using a single weight synapse. The LIF neurons are governed by the following set of

(1)

where U[t] and X[t] are the membrane potential of the LIF neuron and the input at the t-th time step respectively, W is the synaptic weight of the fully-connected layer, β is the decay rate, Sout[t] is the output spike, Uthr is the membrane potential threshold, Usub is the subtracted value if the reset mechanism is reset-by-subtraction and θ is the reset mechanism. The LIF neurons for all layers shares the same Uthr and β. The first two layer uses the reset-to-zero mechanism while the last layer does not use any reset to allow the final output neurons to accumulate membrane potential to predict the velocity of the primate’s movement. For every stride of 4 ms, the membrane voltages are reset and the integration is restarted with fresh input to produce the next output. Due to the reset of the LIF neurons after every prediction, overlapping bin-windows (for TW > St) cause the SNN 3D to process the same input spikes for multiple predictions.

(V) Snn Streaming

The SNN Streaming model also consists of three fully-connected layers (Nch −N1 − N2 −2), with LIF neurons (Equation (1)) in each layer. Unlike SNN 3D, every LIF layer has its own unique Uthr and β. SNN 3D is designed to achieve the highest accuracy while SNN Streaming is designed to achieve the best tradeoff between accuracy and resource consumption. Accordingly, the two main differences between SNN 3D and SNN streaming are the usage of Layer Normalization and the input data processing. SNN Streaming avoids using Layer Normalization, which removes data sparsity (by subtracting the mean from the 0 values). The resulting model of SNN Streaming has higher sparsity and enables the construction of an energy- efficient model by reducing computations.

In Terms Of Input Spike Processing,

TW = TGT = St = 4 ms in this model and hence it does not require any additional pre-processing as seen in Section 3.3; hence, it is called a streaming mode since inputs can stream in directly and continuously to this model. Just like SNN 3D, Combining SNNs with Filtering for Efficient Neural Decoding in Implantable Brain-Machine Interfaces11

(C)

Figure 5: Input data pre-processing methods for feature extraction presented in this paper: a) Summation mode, where the number of spikes detected within a bin window TW for each probe is summed to create a feature. b) Sub-window mode, where the bin window is further divided into m sub-windows, and the number of spikes detected within each sub-window is summed. c) Streaming mode, where the input spike is gathered as it is.

the first two layer uses reset-to-zero while the last layer does not reset its membrane potential.

3.3. Feature Extraction By Input Spike Processing

The spikes generated by the NHP’s neurons are sparse in nature. SNNs can intrinsically accept sparse spiking input since they create an accumulation in the membrane potential variable. For ANNs however, the information over a past time period has to be explicitly accumulated in a feature extraction step.

Also, From The Biological Viewpoint, It Is

generally assumed that short term firing rates (as opposed to mean firing rates over a trial duration ) are important for motor control . Hence, we calculate firing rates, ri(tk) at the sample time tk from the spike waveforms Pi = P

Ts,I Δ(T −Ts,I) On The

i-th probe (1 ≤i ≤Nch) using the following equation:

(2)

where tk+1 −tk = TGT is the sampling time, ts,i denote neural spike times on the i-th probe and TW is the bin window duration. Three different pre-processing methods were used in this paper: the summation method, the sub-window method and the streaming method as illustrated in Figure 5. For all of them, the stride size, st is identical to the Combining SNNs with Filtering for Efficient Neural Decoding in Implantable Brain-Machine Interfaces12 sampling duration, which is TGT = 4 ms. They differ in the choice of TW and how to present the firing information in the bin window to the network as described next.

(I) Summation Method (Used In Ann And Lstm):

This is the simplest case where the firing rate in a bin window with duration of TW is directly used as a feature and input to the NN. We define the input feature

(3)

This method is depicted in Figure 5(a). This method is used by ANN and LSTM models, where ANN uses TW = 200ms while LSTM uses TW = 32ms.

Both

these bin window sizes were obtained after optimization using the training and validation data. Generally, shorter time windows are preferred to capture fine temporal structure of spike trains and reduce the latency of response .

However, with short time windows, LSTMs (or other recurrent models) can retain a memory about long-term history through their state variables while ANNs cannot. Hence, the time window for ANN needs to be longer than that of LSTM to retain sufficient information. In terms of hardware realization, while this accumulation of spikes is straightforward for non-overlapping bin windows, cases with overlap would require repeated operations with overlapping data in naive implementations.

Efficient implementation of such firing rate calculation with overlapping windows are shown in using recursion. (ii) Sub-Window Method (used in ANN 3D and SNN 3D): Similar to the summation method, the sub-window method uses information over the latest TW bin window. However, instead of summing all the spikes, it provides firing rate information at an even shorter time-scale (or with finer resolution) of TW/m. Thus, the feature computed from the i-th probe itself becomes a vector

I (Tk)...Rm

i (tk)] with m components corresponding to firing rates in each of the m sub-windows (duration of integration in Equation (2) is reduced to TW/m). The sub-window method is illustrated in Figure 5(b) and is used by the ANN 3D and SNN 3D models with TW = 200ms and m = 7. The feature vector x(tk) for ANN 3D is defined according to Equation (4) as follows:

(4)

where the dimension of x(tk) is Nch × m. For the SNN 3D, the firing rates in each sub-window are given as input feature to the SNN, which has m time steps. Thus the input feature vector for the SNN in the j-th time step (1 ≤j ≤m) is given by:

(5)

where the dimension of xj(tk) is Nch. Note that ‘j’ indexes time steps here and the SNN output at j = m is the prediction of motor velocity for sample time tk. Combining SNNs with Filtering for Efficient Neural Decoding in Implantable Brain-Machine Interfaces13



Figure 6: Frequency distribution of ground truth and two model outputs from for 2 sec of data from the fourth file in Table 1 shows higher frequency content in prediction. This indicates a strategy to estimate the filter cut off frequency.

(iii) Streaming Method (used in SNN Streaming): The streaming method, as the name suggests, processes the incoming spike data as a continuous stream as seen in Figure 5(c).

The streaming method aims to achieve a better tradeoff between accuracy and computations compared to SNN 3D. In this case, TW = st = TGT = 4 ms implying no overlap between consecutive windows.

This Allows For A Direct

interface between the probes and the model, without the need of adding additional compute cost to our network like the two methods mentioned before. The input feature vector x(tk) is given by the following equation:

(6)

where u() denotes the Heaviside function.

Can

replace multiply and accumulate (MAC) operations by selective accumulation (AC) operations.

3.4. Filters For Snn

Most of the NN models (with the exception of LSTM and SNN Streaming) introduced in Section 3.2 operate on a window or chunk of inputs; providing these windows in any order would result in the same prediction. However, in real life the motor output is a smooth signal with a continuous trajectory. To understand this, we plot in Figure 6 the frequency content of ground truth trajectories of a sample 2-sec waveform and compare it with predicted trajectories of two models from . It is clear that the predictions have much higher frequency content indicating ground truth trajectories are smoother.

In signal processing, this can be rectified by using a filter, which amounts to adding a memory of the past output. Combining SNNs with Filtering for Efficient Neural Decoding in Implantable Brain-Machine Interfaces14 Three types of filters are compared to further explore the effect of filtering– Bessel filter, Butterworth filter and Chebyshev filter.

Bessel Filters Provide A Linear Phase

response resulting in constant delay, but it requires higher filter order to achieve same attenuation at high frequency compared to others. Butterworth filter with no ripples in the passband provides a maximum flat response in the passband, which means the filter will introduce minimal variations to the desired signal amplitude. However, the Butterworth filter has a wide transition band, which makes its rolloff gentle, and hence, the Chebyshev filter becomes a third alternative.

In terms of digital filter implementation, three different techniques were tested in this work. First, we tried forward (Fwd) filtering, which can achieve real-time filtering, but cannot have zero phase shift.

The Improvements Achieved With This

method was marginal and we do not consider them in the following parts.

On The

contrary, bidirectional (Bid) filtering can effectively eliminate phase distortions, but it is generally applicable to offline filtering since the whole waveform is needed before processing begins. To achieve a compromise, block bidirectional filter with a sliding window is applied, such that only a latency penalty of half block size is applicable. We vary the block size between 16 −80, the order of filters between 2 −8 and their cutoff frequencies in the range of 0.05 −0.5 respectively to find the optimum for each model.

3.5. Metrics

In order to evaluate the performance of the models comprehensively in terms of cost vs performance, three metrics are used: (1) number of operations, (2) memory footprint, and (3) accuracy.

Three types of operations are considered for (1) – multiply, add and memory read (since the energy for memory access often dominates the energy for computations ). For most NNs, each synaptic operation comprises a multiply and add (MAC) since the neuron activations and weights are not binary.

On The Other

hand, for SNNs, the synaptic operations only involves accumulations (AC) because of the binary neuron activation. Note that the operations mentioned in this work refer to the operations between synaptic connections as detailed in Section 4.4, while the operations within neurons that determine the membrane potential are excluded. The number of operations is used as proxy for power/energy in this work since the actual energy ratio between these three operations depends on bit-width, process node and memory size; more accurate energy evaluations will be the subject of future work. For (2), memory footprint is evaluated from model size where every parameter is stored using a 32-bit float number. For (3), Coefficient of determination (R2) is a commonly used metrics for regression tasks , which is defined by Equation (7):

(7)

where the label and predictions are showing as yi and ˆyi respectively while ¯y is the mean

Y Are Computed For Predicting X And

Y velocities respectively and the final R2 is an average of the two. Combining SNNs with Filtering for Efficient Neural Decoding in Implantable Brain-Machine Interfaces15 Another set of important metrics for NN hardware are throughput and latency.

We have not considered them here since the considered NN models are small enough so that the total time taken for evaluating the prediction is dominated by the input data accumulation time shown in Section 3.3 and delay due to filtering. However, we do touch upon this point later in Section 3.4.

3.6. Training & Testing Details

All models are trained for 50 epochs using the SNNTorch framework, with a learning rate of 0.005, a dropout rate between 0.3 −0.5, and an L2-regularization value between 0.005 −0.2. AdamW is chosen as the optimizer, Mean Squared Error (MSE) loss is determined as the loss function, and a learning scheduler (cosine annealing schedule) is used after every epoch. For ANN, ANN 3D, and SNN 3D, data is shuffled with batch size of 512 in training. For SNN 3D, the membrane potential resets every batch, while reset occurs at the beginning of each reach for membrane potential in SNN Streaming and hidden states in LSTM. The distribution of reach durations show most reaches completed in less than 4 sec while some reaches being much longer, presumably due to the NHP not attending to the task. Similar to , reaches that exceed 8 seconds in length are removed to improve the training performance. Leaky Integrate-and-Fire Neuron is used in SNNs, where the threshold and β are learned during training and Arctan is applied as a surrogate function . The membrane potential of neurons ceases to reset in the last layer to enable regression. The velocity predicted by SNNs is determined by scaling the membrane potential of neurons with a learnable constant parameter. For validation and testing, data is input to the models in chronological order, and reset mechanisms only occur at the beginning. Filters are employed exclusively during the inference process.

4. Results

To comprehensively examine the capability of different models, we performed multiple experiments and evaluated models using the metrics mentioned in Section 3.5. All the results except memory access are obtained from the neurobench harness that does automated evaluation of the models; memory access is estimated based on theoretical equations of weight fetches based on experimentally observed sparsity multiplying the number of weights on a per layer basis. The findings are presented pictorially using two pareto plots, first comparing the accuracy versus operations trade-off and the second comparing accuracy versus memory footprint (e.g. see Figure 14 and Figure 16). A tabular summary of all the experiments performed for our base models can be found in Table 2. Table 2 also compares the results with other published work using statistical methods such as Steady State Kalman filter (SSKF), Unscented Kalman filter (UKF), recurrent Exponential-Family Harmonium (rEFH) etc.

Combining SNNs with Filtering for Efficient Neural Decoding in Implantable Brain-Machine Interfaces16 Table 2: Baseline Performance and Comparison with Prior Works using high accuracy filter configuration. Best performing filter in a class of SNN models is highlighted in bold font. Incremental accuracy improvement by using 80% data over 50% data is shown in parenthesis.

4.1. Model Size Search

As mentioned earlier, it was found that networks deeper than 3 layers performed poorly and hence deeper models were excluded from this study. The number of neurons in each of the two hidden layers was determined by searching within a certain range (N0 = Nch and N3 = 2 are fixed). We used the ANN to do this search due to its simplest network structure and resultant fast training. The results obtained by varying N1 and N2 are shown in the Figure 7. Here, model complexity is characterized by the number of synaptic weights. The text shown in the figure represents the different network architectures Combining SNNs with Filtering for Efficient Neural Decoding in Implantable Brain-Machine Interfaces17

64-64

Figure 7: Complexity versus R2 for different ANN models tested. N1 −N2 values of 16 −32 and 32 −48 were on the pareto curve and chosen as ‘tiny’ and ‘baseline’ variants. Here, model complexity is characterized by the number of synaptic weights.

(N1 −N2 combinations) tested. As expected, the R2 initially increases with increasing number of neurons but starts decreasing after the number of neurons reaches a certain value due to overfitting. The best trade-off between R2 and complexity is determined by the networks lying on the pareto curve shown in blue in Figure 7. Therefore, the two models with N1 −N2 values of 32 −48 and 16 −32 were selected as the ‘base’ and ‘tiny’ variants respectively for ANN. Same variant sizes were near optimal for ANN 3D and SNN 3D (we do not show these tradeoff curves for brevity), while for SNN Streaming, base and tiny variants represented N1 −N2 values of 32 −48 and 16 −48 respectively.

4.2. K-Fold Cross Validation

It is important to verify that the result will not vary significantly regardless of how the data is split. Hence, K-fold cross-validation is used to test all six files for three models (ANN, ANN 3D and SNN 3D). We divided the data into five parts, randomly selecting four parts as training and the other part was divided into validation and testing. The means and standard deviations of R2 for the 5-fold experiment are shown in Table 3.

Low variance of the results for all 3 cases implies using one-fold data split for our experiment is reasonable and will give dependable results. As a comparison, the results in Table 2 does show that without filter, the decoding accuracy for SNN 3D is the best and ANN is the worst with ANN 3D between the two. Hence, we just use the single data split in described earlier for the rest of the results.

4.3. Filtering: Performance improvement and Optimization First, we compare the performance of different types of filters; results for bidirectional filtering are shown here but similar conclusion holds for the Block bidirectional case as Combining SNNs with Filtering for Efficient Neural Decoding in Implantable Brain-Machine Interfaces18

No Filter

Figure 8: Comparison of different 4-th order filters using bidirectional technique and cut-off frequency of 0.05. Bessel filters show the best improvement. Statistical significance tested using paired t-test.

well. The comparison of these three filters with the original result is depicted in Figure 8. Here, the order of the filter is fixed at 4 and the cut-off frequency is fc = 0.05 based on the estimate from Figure 6. It is clear that the Bessel filter provides the maximum improvement of R2 both in SNN 3D and SNN Streaming model (statistical significance tested using paired t-test), which is about 8.2% higher than SNN 3D and 5.7% higher than SNN Streaming without using filters. The constant delay property of Bessel filters is crucial in not distorting the waveform shape. To visualize this qualitatively, X and Y velocities for one reach from one file after filtering by the three filters is shown in Figure 9(reaches from other files are shown in the Supplementary material). It can be seen that the velocity trajectories after Chebyshev filtering is not as smooth as the other two with Bessel filters producing the smoothest and most natural trajectories. Thus, we choose Bessel filters to perform the rest of the experiments in this paper.

To find the combination of filter order and block size with the best trade-off in terms of accuracy and latency, we evaluated the performance for SNN 3D and SNN Streaming Combining SNNs with Filtering for Efficient Neural Decoding in Implantable Brain-Machine Interfaces19



Figure 9: Comparison of three types of filters in terms of velocity and position based on one reach in the file ”indy 20160622 01”. The three columns indicate SNN with Bessel, Butter, and Chebyshev filter respectively. The first row shows the X velocity, the second row shows the Y velocity. 4-th order filter with block size of 32 and cut- off frequency of 0.05 is used.

using block Bid filtering of bessel filter with the cut-off frequency of 0.05. The block size is varied between 16 −80, while the filter order is swept between 2 −8. The results for both types of SNN models shown in Figure 10 indicate a large increase in R2 when the block size increases to 32 and marginal improvements from there on, making block size of 32 a good choice. For block sizes of 32, a filter order of 4 is optimal (statistically significant difference between order 2 and order 4 tested at the 5% level using paired t-test for both SNN 3D and SNN Streaming), while for higher block sizes, filter order of 6 gives better results. There are no results for block size of 16 with filter order larger than 4 because of insufficient samples to perform the filtering. In this case, filter order of 2 gives the best accuracy (statistically significant difference between order 2 and order 4 exists using the paired t-test at the 5% level for SNN Streaming and 10% level for SNN 3D). From these results, a block size of 32 and a filter order of 4 seems like the best choice. However, there is a direct tradeoff between filter order and latency of the output as discussed next; this may make block size of 16 more appropriate for some applications .

Latency between input and output is important for real-time applications with closed-loop operation such as motor decoding. The total time, Ttot, taken to produce an output by a NN decoder is given by Ttot = St + Tcomp where St is the stride to Combining SNNs with Filtering for Efficient Neural Decoding in Implantable Brain-Machine Interfaces20

(B)

Figure 10: Comparison of block size versus R2 in different filter order: a) The performance for SNN 3D using bessel filter with cutoff frequency of 0.05. b) The performance for SNN Streaming using bessel filter with cutoff frequency of 0.05.

capture the new input data (= TGT = 4 ms in this work) and Tcomp is the time taken to process the computations in the neural network. Given the very fast and energy- efficient In-memory computing (IMC) approaches to implement NN models prevalent now and the small networks considered in this paper, we can assume Tcomp << St making the throughput almost entirely dependent on St, i.e. time taken to capture new neural input spikes. Note that the bin window, TW does not add any extra penalty on latency of output generation; however, after every change of target, the prediction will be inaccurate for a time related to TW to allow enough relevant input to fill up the bin window. However, output filtering may induce an extra penalty on the latency. Bid filters produced best results as seen in the Table 2; however, they cannot be employed in real-time applications since they need to store the raw data in memory first and then Combining SNNs with Filtering for Efficient Neural Decoding in Implantable Brain-Machine Interfaces21

128

Figure 11: Latency versus R2 for block Bid filtering using 2-nd order Bessel filters with cut-off frequency of 0.05.The number denotes different block sizes. Block size of 32 gives an optimal tradeoff between accuracy and latency.

apply forward filtering two times in opposite directions. The block Bid filter is chosen as a compromise where the filter window is used to determine the length or block of samples that are filtered at one time, and the predicted point is located at the center of the sample window. Thus, the latency introduced by the block Bid filter is theoretically equal to half the length of the filter window.

Figure 11 Compares The Accuracy Of

2nd order Bessel filters for various block sizes. This shows that the minimum latency achievable by a 2nd order Bessel filter with block size 16 is 32 ms for this dataset with TGT = 4 ms. Hence, we select two Bessel filter configurations for further simulations– lower latency (2nd order, block size 16) and higher accuracy (4th order, block size 32).

Lastly, we have so far chosen the cutoff frequency fc in an adhoc fashion based on Figure 6. However, fc also can be optimized for the above two configurations.

SNN models are not directly involved in the training process; rather, the input data is processed by pre-trained SNNs and the model output is then passed to the Bessel filter. Two approaches were tried–one where the digital filter is treated as a single layer neural network and another where fc is directly optimized by using search algorithms, with the latter providing better results. Grid search is used first to determine a reasonably accurate range around 0.05 for fc with the potential maximum of R2. Then search step and range are halved, based on the preferred range found by grid search. This process is repeated independently for each of the six files until R2 stops increasing.

Figure 12 shows the optimization results for both SNN Streaming and SNN 3D for the two configurations. The results indicate a statistically significant difference between

Related Journal Articles & DOI Links

Selected peer-reviewed publications relevant to 12 Lead ECG Acquisition. Click the DOI to access the full paper (may require institutional access).

Why Choose Us?

Bangalore guidance for robotics, Spectre and autonomous systems projects.