All papers are available via the Strathclyde Open Access policy via the pureportal link in the footer of the page

Pre-Prints

• Keys to Accurate Feature Extraction Using Residual Spiking Neural Networks

Spiking neural networks (SNNs) have become an interesting alternative to conventional artificial neural networks (ANN) thanks to their temporal processing capabilities and their low-SWaP (Size, Weight, and Power) and energy efficient implementations in neuromorphic hardware. However, the challenges involved in training SNNs have limited their performance in terms of accuracy and thus their applications. Improving learning algorithms and neural architectures for a more accurate feature extraction is therefore one of the current priorities in SNN research. In this paper we present a study on the key components of modern spiking architectures. We empirically compare different techniques in image classification datasets taken from the best performing networks. We design a spiking version of the successful residual network (ResNet) architecture and test different components and training strategies on it. Our results provide a state of the art guide to SNN design, which allows to make informed choices when trying to build the optimal visual feature extractor. Finally, our network outperforms previous SNN architectures in CIFAR-10 (94.1%) and CIFAR-100 (74.5%) datasets and matches the state of the art in DVS-CIFAR10 (71.3%), with less parameters than the previous state of the art and without the need for ANN-SNN conversion. Code available at https://github.com/VicenteAlex/Spiking_ResNet.

• Unsupervised Spiking Instance Segmentation on Event Data using STDP

Spiking Neural Networks (SNN) and the field of Neuromorphic Engineering has brought about a paradigm shift in how to approach Machine Learning (ML) and Computer Vision (CV) problem. This paradigm shift comes from the adaption of event-based sensing and processing. An event-based vision sensor allows for sparse and asynchronous events to be produced that are dynamically related to the scene. Allowing not only the spatial information but a high-fidelity of temporal information to be captured. Meanwhile avoiding the extra overhead and redundancy of conventional high frame rate approaches. However, with this change in paradigm, many techniques from traditional CV and ML are not applicable to these event-based spatial-temporal visual streams. As such a limited number of recognition, detection and segmentation approaches exist. In this paper, we present a novel approach that can perform instance segmentation using just the weights of a Spike Time Dependent Plasticity trained Spiking Convolutional Neural Network that was trained for object recognition. This exploits the spatial and temporal aspects of the network’s internal feature representations adding this new discriminative capability. We highlight the new capability by successfully transforming a single class unsupervised network for face detection into a multi-person face recognition and instance segmentation network.

Publications

• Ultrafast neuromorphic photonic image processing with a VCSEL neuron

The ever-increasing demand for artificial intelligence (AI) systems is underlining a significant requirement for new, AI-optimised hardware. Neuromorphic (brain-like) processors are one highly-promising solution, with photonic-enabled realizations receiving increasing attention. Among these, approaches based upon vertical cavity surface emitting lasers (VCSELs) are attracting interest given their favourable attributes and mature technology. Here, we demonstrate a hardware-friendly neuromorphic photonic spike processor, using a single VCSEL, for all-optical image edge-feature detection. This exploits the ability of a VCSEL-based photonic neuron to integrate temporally-encoded pixel data at high speed; and fire fast (100 ps-long) optical spikes upon detecting desired image features. Furthermore, the photonic system is combined with a software-implemented spiking neural network yielding a full platform for complex image classification tasks. This work therefore highlights the potential of VCSEL-based platforms for novel, ultrafast, all-optical neuromorphic processors interfacing with current computation and communication systems for use in future light-enabled AI and computer vision functionalities.

• Neuromorphic Image Processing with a VCSEL Neuron

Photonic spiking VCSEL-neurons are used for high-speed neuromorphic image processing. We demonstrate that a VCSEL-Neuron system using sequential kernel operators successfully reveals all edge-features from 5000 complex images of the MNIST handwritten digit dataset.

• Neuromorphic technologies for defence and security

Despite the highly promising advances in Machine Learning (ML) and Deep Learning (DL) in recent years, DL requires significant hardware acceleration to be effective, as it is rather computationally expensive. Moreover, miniaturisation of electronic devices requires small form-factor processing units, with reduced SWaP (Size,Weight and Power) profile. Therefore, a completely new processing paradigm is needed to address both issues. In this context, the concept of neuromorphic (NM) engineering provides an attractive alternative, seen as the analog/digital implementation of biologically brain inspired neural networks. NM systems propagate spikes as means of processing data, with the information being encoded in the timing and rate of spikes generated by each neuron of a so-called spiking neural network (SNN). Based on this, the key advantages of SNNs are: less computational power required, more efficient and faster processing, much lower power consumption. This paper reports on the current state of the art in the field of NM systems, and it describes three application scenarios of SNN-based processing for security and defence, namely target detection and tracking, semantic segmentation, and control.

• Imaging from temporal data via spiking convolutional neural networks

A new approach for imaging that is solely based on the time of flight of photons coming from the entire imaged scene, combined with a novel machine learning algorithm for image reconstruction: a spiking convolutional neural network (SCNN) named Spike-SPI (Spiking – Single Pixel Imager). The approach uses a single point detector and the corresponding time-counting electronics, which provide the arrival time of photons in the form of spikes distributed over time. This data is transformed into a temporal histogram containing the number of photons per arrival time. A SCNN that converts the 1D temporal histograms into a 3D image (2D image with depth map) by exploiting the feature extraction capabilities of convolutional neural networks (CNNs), the high dimensional compressed latent space representations of a variational encoder-decoder network structure, and the asynchronous processing capabilities of a spiking neural network (SNN). The performance of the proposed SCNN is analysed to demonstrate the state-of-the-art feature extraction capabilities of CNNs and the low latency asynchronous processing of SNNs that offer both higher throughput and higher accuracy in image reconstruction from the ToF data, when compared to standard ANNs. The results of Spike-SPI show an increase in spatial accuracy of 15% over then ANN, using the Intersection of Union (IoU) for the objects in the scene. While also delivering a 100% increase over then ANN in object reconstruction signal to noise ratio (RSNR) from ~3dB to ~6dB. These results are also consistent across a range of IRF (Instrument Response Functions) values and photo counts, highlighting the robust nature of the new network structure. Moreover, the asynchronous processing nature of the spiking neurons allow for a faster throughput and less computational overhead, benefiting from the operational sparsity in the single point sensor.

• Perception understanding action: adding understanding to the perception action cycle with spiking segmentation

Traditionally the Perception Action cycle is the first stage of building an autonomous robotic system and a practical way to implement a low latency reactive system within a low Size, Weight and Power (SWaP) package. However, within complex scenarios, this method can lack contextual understanding about the scene, such as object recognition-based tracking or system attention. Object detection, identification and tracking along with semantic segmentation and attention are all modern computer vision tasks in which Convolutional Neural Networks (CNN) have shown significant success, although such networks often have a large computational overhead and power requirements, which are not ideal in smaller robotics tasks. Furthermore, cloud computing and massively parallel processing like in Graphic Processing Units (GPUs) are outside the specification of many tasks due to their respective latency and SWaP constraints. In response to this, Spiking Convolutional Neural Networks (SCNNs) look to provide the feature extraction benefits of CNNs, while maintaining low latency and power overhead thanks to their asynchronous spiking event-based processing. A novel Neuromorphic Perception Understanding Action (PUA) system is presented, that aims to combine the feature extraction benefits of CNNs with low latency processing of SCNNs. The PUA utilizes a Neuromorphic Vision Sensor for Perception that facilitates asynchronous processing within a Spiking fully Convolutional Neural Network (SpikeCNN) to provide semantic segmentation and Understanding of the scene. The output is fed to a spiking control system providing Actions. With this approach, the aim is to bring features of deep learning into the lower levels of autonomous robotics, while maintaining a biologically plausible STDP rule throughout the learned encoding part of the network. The network will be shown to provide a more robust and predictable management of spiking activity with an improved thresholding response. The reported experiments show that this system can deliver robust results of over 96 and 81% for accuracy and Intersection over Union, ensuring such a system can be successfully used within object recognition, classification and tracking problem. This demonstrates that the attention of the system can be tracked accurately, while the asynchronous processing means the controller can give precise track updates with minimal latency.

• SpikeSEG: Spiking segmentation via STDP saliency mapping

Taking inspiration from the structure and behaviour of the human visual system and using the Transposed Convolution and Saliency Mapping methods of Convolutional Neural Networks (CNN), a spiking event-based image segmentation algorithm, SpikeSEG is proposed. The approach makes use of both spike-based imaging and spike-based processing, where the images are either standard images converted to spiking images or they are generated directly from a neuromorphic event driven sensor, and then processed using a spiking fully convolutional neural network. The spiking segmentation method uses the spike activations through time within the network to trace back any outputs from saliency maps, to the exact pixel location. This not only gives exact pixel locations for spiking segmentation, but with low latency and computational overhead. SpikeSEG is the first spiking event-based segmentation network and over three experiment test achieves promising results with 96% accuracy overall and a 74% mean intersection over union for the segmentation, all within an event by event-based framework.

• A new spiking convolutional recurrent neural network (SCRNN) with applications to event-based hand gesture recognition

The combination of neuromorphic visual sensors and spiking neural network offer a high efficient bio-inspired solution to real-world applications. However, processing event- based sequences still remains challenging because of the nature of their asynchronism and sparsity behaviour. In this paper, a novel spiking convolutional recurrent neural network (SCRNN) architecture that takes advantage of both convolution operation and recurrent connectivity to maintain the spatial and temporal relations from event-based sequence data is presented. The use of recurrent architecture enables the network to have arbitrary length of sampling window allowing the network to exploit temporal correlations between event collections. Rather than standard ANN to SNN conversion techniques, the network utilizes supervised Spike Layer Error Reassignment(SLAYER) training mechanism that allows the network to directly adapt to neuromorphic(event-based) data. The network structure is validated on the DVS gesture dataset and it has achieved a 10 class gesture recognition accuracy of 96.59% and 11 class gesture recognition accuracy of 92.01%

.Deep convolutional spiking neural network based hand gesture recognition

Novel technologies for EMG (Electromyogram) based hand gesture recognition have been investigated for many industrial applications. In this paper, a novel approach which is based on a specific designed spiking convolution neural network which is fed by a novel EMG signal energy density map is presented. The experimental results indicate that the new approach not only rapidly decreases the required processing time but also increases the average recognition accuracy to 98.76% based on the Strathclyde dataset and to 98.21% based on the CapgMyo open source dataset. A relative comparison of experimental results between the proposed novel EMG based hand gesture recognition methodology and other similar approaches indicates the superior effectiveness of the new design.

• UAV detection: a STDP trained deep convolutional spiking neural network retina-neuromorphic approach

The Dynamic Vision Sensor (DVS) has many attributes, such as sub-millisecond response time along with a good low light dynamic range, that allows it to be well suited to the task for UAV Detection. This paper proposes a system that exploits the features of an event camera solely for UAV detection while combining it with a Spiking Neural Network (SNN) trained using the unsupervised approach of Spike Time-Dependent Plasticity (STDP), to create an asynchronous, low power system with low computational overhead. Utilising the unique features of both the sensor and the network, this result in a system that is robust to a wide variety in lighting conditions, has a high temporal resolution, propagates only the minimal amount of information through the network, while training using the equivalent of 43,000 images. The network returns a 91% detection rate when shown other objects and can detect a UAV with less than 1% of pixels on the sensor being used for processing.

• Cubesat-based passive bistatic radar for space situational awareness: A feasibility study

This paper proposes a low budget solution to detect and possibly track space debris and satellites in Low Earth Orbit. The concept consists of a space-borne radar installed on a cubeSat flying at low altitude and detecting the occultations of radio signals coming from existing satellites flying at higher altitudes. The paper investigates the feasibility and performance of such a passive bistatic radar system. Key performance metrics considered in this paper are: the minimum size of detectable objects, considering visibility and frequency constraints on existing radio sources, the receiver size, and the compatibility with current cubeSat’s technology. Different illuminator types and receiver altitudes are considered under the assumption that all illuminators and receivers are on circular orbits.