The ELM Neuron: An Efficient and Expressive Cortical Neuron Model Can Solve Long-Horizon Tasks

Aaron tells us about the Expressive Leaky Memory (ELM) neuron model, a biologically inspired phenomenological model of a cortical neuron.

The ELM Neuron: An Efficient and Expressive Cortical Neuron Model Can Solve Long-Horizon Tasks
  • Aaron Spieler
  • February 27, 2024 6:00 - 7:30 CET
  • Event

Biological cortical neurons are remarkably sophisticated computational devices, temporally integrating their vast synaptic input over an intricate dendritic tree, subject to complex, nonlinearly interacting internal biological processes.

With the aim to explore the computational implications of leaky memory units and nonlinear dendritic processing, we introduce the Expressive Leaky Memory (ELM) neuron model, a biologically inspired phenomenological model of a cortical neuron. Remarkably, by exploiting a few such slowly decaying memory-like hidden states and two-layered nonlinear integration of synaptic input, our ELM neuron can accurately match the aforementioned input-output relationship with under ten-thousand trainable parameters.

We evaluate the model on various tasks with demanding temporal structures, including the Long Range Arena (LRA) datasets, as well as a novel neuromorphic dataset based on the Spiking Heidelberg Digits dataset (SHD-Adding). The ELM neuron reliably outperforms the classic Transformer or Chrono-LSTM architectures on these tasks, even solving the Pathfinder-X task with over 70% accuracy (16k context length).

~ Share this Site ~

Upcoming Workshops

Accelerating Neuromorphic Inference and Training at the Edge - Rain
Maxence Ernoult
2024, Mar 5    6:00 - 7:30 CET
Advances in Neuromorphic Visual Place Recognition
Tobias Fischer
2024, Mar 20    6:00 - 7:30 CET
The ELM Neuron: An Efficient and Expressive Cortical Neuron Model Can Solve Long-Horizon Tasks

About the Speaker

Aaron Spieler is a computational neuroscientist passionate about exploring the intersection of deep learning and neuroscience. After earning his Bachelor’s in Computer Science from the University of Potsdam, he undertook an extended internship at Amazon Web Services working in deep learning based forecasting, before further specializing with a Master’s in Computational Neuroscience at the University of Tübingen. Throughout his Master’s thesis and a subsequent internship at the Max Planck Institute for Intelligent Systems, Aaron focused on phenomenological neuron modeling with applications to long-range prediction tasks. Pursuing this work allowed him to collaborate with excellent researchers from diverse backgrounds, including Prof. Bernhard Schölkopf and Prof. Anna Levina.

Hybrid Learning for Event-based Visual Motion Detection and Tracking of Pedestrians

Hybrid Learning for Event-based Visual Motion Detection and Tracking of Pedestrians

  • Cristian Axenie
  • 2024, January 15

Revolutionize traffic safety with neuromorphic visual sensing. Explore award-winning solutions for pedestrian detection and tracking, emphasizing sustainability and city-level deployment. Join Dr. Cristian Axenie in this groundbreaking AI exploration

Hands-On with snnTorch

Hands-On with snnTorch

  • Jason Eshraghian
  • 2023, March 2

Join Jason Eshraghian for an engaging hands-on session featuring snnTorch. Explore the world of neuromorphic engineering.

Hands-On with Nengo Applied Brain Research

Hands-On with Nengo Applied Brain Research

  • Trevor Bekolay
  • 2023, January 26

Dive into the world of applied brain research with Trevor Bekolay. Explore learning, memory, and neural simulations in this insightful recorded session