The ELM Neuron: An Efficient and Expressive Cortical Neuron Model Can Solve Long-Horizon Tasks

Aaron tells us about the Expressive Leaky Memory (ELM) neuron model, a biologically inspired phenomenological model of a cortical neuron.

Biological cortical neurons are remarkably sophisticated computational devices, temporally integrating their vast synaptic input over an intricate dendritic tree, subject to complex, nonlinearly interacting internal biological processes.

With the aim to explore the computational implications of leaky memory units and nonlinear dendritic processing, we introduce the Expressive Leaky Memory (ELM) neuron model, a biologically inspired phenomenological model of a cortical neuron. Remarkably, by exploiting a few such slowly decaying memory-like hidden states and two-layered nonlinear integration of synaptic input, our ELM neuron can accurately match the aforementioned input-output relationship with under ten-thousand trainable parameters.

We evaluate the model on various tasks with demanding temporal structures, including the Long Range Arena (LRA) datasets, as well as a novel neuromorphic dataset based on the Spiking Heidelberg Digits dataset (SHD-Adding). The ELM neuron reliably outperforms the classic Transformer or Chrono-LSTM architectures on these tasks, even solving the Pathfinder-X task with over 70% accuracy (16k context length).

Social share preview for The ELM Neuron: An Efficient and Expressive Cortical Neuron Model Can Solve Long-Horizon Tasks

Upcoming Workshops

The TSP1 Neuromorphic Chip: Advancing Brain-Inspired Computing
Chris Eliasmith, Danny Rosen
November 11, 2025
8:00 - 9:00 EST

About the Speaker

Aaron Spieler is a computational neuroscientist passionate about exploring the intersection of deep learning and neuroscience. After earning his Bachelor’s in Computer Science from the University of Potsdam, he undertook an extended internship at Amazon Web Services working in deep learning based forecasting, before further specializing with a Master’s in Computational Neuroscience at the University of Tübingen. Throughout his Master’s thesis and a subsequent internship at the Max Planck Institute for Intelligent Systems, Aaron focused on phenomenological neuron modeling with applications to long-range prediction tasks. Pursuing this work allowed him to collaborate with excellent researchers from diverse backgrounds, including Prof. Bernhard Schölkopf and Prof. Anna Levina.

Inspired? Share your work.

Share your expertise with the community by speaking at a workshop, student talk, or hacking hour. It’s a great way to get feedback and help others learn.

Related Workshops

Project Phasor - Kickoff

Project Phasor - Kickoff

Brian Anderson and others discuss the newly launched Project Phasor, aiming to organize efforts towards neuromorphic and NeuroAI virtualization and compilation.

Hands-On with Nengo Applied Brain Research

Hands-On with Nengo Applied Brain Research

Dive into the world of applied brain research with Trevor Bekolay. Explore learning, memory, and neural simulations in this insightful recorded session

Hands-On with Sinabs and Speck

Hands-On with Sinabs and Speck

Join Gregor Lenz for an engaging hands-on session featuring Sinabs and Speck. Explore the world of neuromorphic engineering and spike-based machine learning.