Lifelong Learning with a Biologically-inspired Neural Network (BiNN)

Reference implementation of "Enabling Lifelong Learning in AI with Biologically-inspired Neural Networks Based on Short-Term, Working, and Long-Term Memory" — 2025 IEEE 7th International Conference on Cognitive Machine Intelligence (CogMI).

---

The problem

Neural networks forget. Train one sequentially on shifting data and the newer distribution overwrites the older weights — catastrophic forgetting. It is one of the clearest gaps between machine and human cognition: people learn continuously for decades without wiping out what they already knew.

The idea

Humans don't avoid forgetting through a single memory system, so this architecture doesn't either. It combines three theories of human cognition — the Three-stage Memory Model, the Working Memory Model, and Complementary Learning Systems (CLS) theory — into one network with three interacting regions:

Region	Memory	Role
Cerebral Cortex	Long-term	One large, slow-learning CNN holding general knowledge. Updated only by consolidation.
Prefrontal Cortex	Working	A growing family of small CNNs, each an expert on one pattern-separated schema of the data.
Hippocampus	Short-term	Routes incoming items to a schema and fuses both systems' predictions.

New data only ever enters working memory. Because each schema owns a narrow slice of the feature space, learning a new slice doesn't disturb the others — the interference is avoided structurally, not regularised away. A schema that keeps proving relevant is eventually consolidated into long-term memory, which is how the model accumulates knowledge rather than merely partitioning it.

See docs/architecture.md for the full mapping from theory to code.

Results from the paper

On the CLEAR benchmark (real continual learning) and CIFAR-10 (artificial distribution shift), the BiNN was compared against two controls built from the same pre-trained weights:

• The constant-retrain control — today's AI — held 100% on CLEAR until roughly 7,000 images, then collapsed to ~50% as catastrophic forgetting set in.
• The BiNN never collapsed. Its accuracy kept climbing and overtook both controls, leading by up to 50 percentage points.

The trade-off is runtime: up to 1.98 s per prediction versus 0.07 s for a single model, since every schema votes. The current implementation is unoptimised.

Install

git clone https://github.com/HanavM/lifelong-learning.git
cd lifelong-learning
pip install -r requirements.txt

On Apple Silicon, substitute tensorflow-macos (and optionally tensorflow-metal) for tensorflow. A GPU is strongly recommended — the paper's runs used a Colab T4 with high-RAM.

Quickstart

# 1. Export CIFAR-10 and cluster it by visual similarity (paper III-A1).
#    Clustering is what creates the distribution shift the experiment measures.
python scripts/prepare_data.py \
    --out data/cifar10-clustered \
    --method bucketing \
    --projector-out models/projector

# 2. Pre-train the Cerebral Cortex on the pre-training pool.
python scripts/pretrain_cortex.py \
    --data data/cifar10-clustered \
    --out models/cerebral_cortex.keras

# 3. Stream the remaining data through the BiNN and both controls.
python scripts/run_experiment.py \
    --data data/cifar10-clustered \
    --cortex models/cerebral_cortex.keras \
    --projector models/projector \
    --out results/cifar10.json

# 4. Plot accuracy over time (paper Fig. 3 / Fig. 4).
python scripts/plot_results.py results/cifar10.json --out results/cifar10.png

Smoke test first. The full run takes hours. To check the pipeline end to end in a few minutes:

python scripts/prepare_data.py --out data/cifar10-clustered --limit-per-class 100
python scripts/pretrain_cortex.py --data data/cifar10-clustered \
    --out models/cerebral_cortex.keras --epochs 2
python scripts/run_experiment.py --data data/cifar10-clustered \
    --cortex models/cerebral_cortex.keras --stream-limit 400 --train-cycle 100

Library use

from binn import BiNNConfig
from binn.binn import BiNN
from binn.cortex import CerebralCortex
from binn.features import FeatureExtractor
from data.splits import build_splits

split = build_splits("data/cifar10-clustered")

extractor = FeatureExtractor(image_size=32, pca_components=50)
extractor.fit_projector(extractor.embed_batch([p for p, _ in split.pretrain[:1500]]))

cortex = CerebralCortex.load("models/cerebral_cortex.keras", num_classes=10)
model = BiNN(extractor, cortex, labels=split.labels, config=BiNNConfig())

for path, label in split.stream[:500]:
    model.learn(path, label)          # absorb into working memory
model.train_schemas()                 # train the schema networks

prediction = model.predict(some_path) # fused working + long-term answer
print(model.summary())

Every hyperparameter in the paper lives in BiNNConfig (binn/config.py) — the 3× working-memory weight, the 0.2 similarity threshold, the consolidation threshold of 15, and so on. Configs round-trip to JSON, so a run is reproducible from its saved config alone.

Repository layout

binn/
  config.py        every tunable from the paper, in one dataclass tree
  features.py      frozen VGG-16 → StandardScaler → PCA(50)
  centroids.py     Huber M-estimator centroids (pure NumPy, no TF)
  schemas.py       Schema + SchemaStore — working memory's contents
  models.py        the two Keras architectures
  cortex.py        Cerebral Cortex — long-term memory
  prefrontal.py    Prefrontal Cortex — working memory
  hippocampus.py   short-term memory + prediction fusion
  replay.py        hippocampal replay buffer
  binn.py          the orchestrator
  experiment.py    three-arm lifelong-learning harness
data/
  clustering.py    k-means / gradual bucketing over VGG-16 features
  splits.py        cluster-aware 45% pre-train / 55% stream split
scripts/           four-step CLI pipeline
tests/             36 tests over the deterministic core
notebooks/         the original Colab research notebooks, unmodified
docs/              architecture notes

Tests

pytest tests -q

The suite covers the deterministic core — centroid estimation, schema routing, replay sampling, prediction fusion, dataset splitting, config round-tripping — and needs neither TensorFlow nor a GPU.

Relationship to the original research code

The experiments were developed in Google Colab notebooks, preserved unmodified under notebooks/. This package is a restructuring of that work: the same algorithms and hyperparameters, reorganised into importable modules with tests, CLI entry points, and no hard-coded Google Drive paths.

Behavioural differences are documented in notebooks/README.md.

Citing

@inproceedings{modasiya2025lifelong,
  author    = {Modasiya, Hanav},
  title     = {Enabling Lifelong Learning in {AI} with Biologically-inspired
               Neural Networks Based on Short-Term, Working, and Long-Term Memory},
  booktitle = {2025 IEEE 7th International Conference on Cognitive Machine
               Intelligence (CogMI)},
  year      = {2025},
  pages     = {244--252},
  doi       = {10.1109/CogMI67134.2025.00035}
}

License

MIT — see LICENSE. The paper itself is © 2025 IEEE.