HM /
Let's talk

PART 17 / Advanced

Memory: biological & artificial

Which kind of memory does a task need, and what mechanism supplies it?

By Harshit Makwana · Edition 0.2

Illustrated introduction

What you will learn

Separate storage, retrieval, and learning, then test a hybrid memory system for interference.

Before you begin: From neurons to brains; Spiking neural networks

Ask what must be remembered

Memory can mean temporary task state, stored facts, an episode, a learned association or a changed synapse. These functions operate on different timescales and should not be collapsed into one mechanism. A system that retrieves a document has a different operation from a recurrent circuit that maintains activity after a stimulus disappears.

Associations and attractors

An attractor model uses recurrent dynamics to move nearby states toward stable patterns under particular assumptions. Hopfield’s work is an influential artificial-network example. The score calculation below is simpler: it compares a partial cue with stored vectors once. It does not implement attractor dynamics, capacity analysis or a hippocampal model.

Neural-network perspective

Transformer context, recurrent state, retrieval systems and external databases provide different memory interfaces. Retrieval selects information; it does not verify that the information is true. A robust system tracks provenance, update time and uncertainty. Evaluate interference, false recall, outdated records and forgetting rather than measuring only successful retrieval from an easy example.

SEE THE RELATIONSHIPS

From structure to behavior.

Network: a labelled conceptual diagram
Figure 17.1 · Network. Original educational diagram; synthetic, not a measured result.
The reasoning pathCue or observation then Retrieve / recurrent state then Check provenance then Use & update memory1. Cue or observation2. Retrieve / recurrent state3. Check provenance4. Use & update memory
A learning flow for this part. Follow the assumptions through to an observable result.

MAKE THE IDEA PRECISE

The mathematics.

score_j = sum_i cue_i pattern_(j,i)
Symbols and units
SymbolMeaning
cueQuery vector, with zero representing a missing component in this toy
pattern_jStored candidate vector j
score_jUnnormalized dot-product similarity
iVector component index

Build the equation, step by step

  1. For a cue and candidate of equal length, multiply corresponding entries and sum the products. This is a dot-product similarity, not a calibrated probability.
  2. With signed binary patterns, matching signs contribute positively and opposing signs contribute negatively.
  3. Rank the candidate scores. For general real vectors, vector length affects the result; normalization or a different distance may be needed. Retrieval is only one component of a memory system.

WORK THROUGH THE NUMBERS

Worked example

A partial cue [1, 0, 1] scores 2 against [1, -1, 1] and -2 against [-1, 1, -1]. The result identifies a closer pattern in this tiny constructed set, not a general memory capacity.

A small Python experiment.

Python 3 · Standard library only · Synthetic educational example. Run this in your own notebook or a local file.

cue = [1, 0, 1]
patterns = [[1,-1,1], [-1,1,-1]]
scores = [sum(a*b for a,b in zip(cue,p)) for p in patterns]
print(scores)

Expected output

[2, -2]

Change one quantity, predict the result, and then run the code again. Explain any difference before adding complexity.

Test your understanding.

Try it: Corrupt one nonzero cue component. Does the selected pattern remain reliable?

Self-check: Does selecting the most similar stored pattern prove that the answer is true?

No. Similarity retrieves candidates; it does not validate their content, origin or current relevance. Keep provenance and uncertainty, and check a retrieved claim against appropriate evidence before using it as a fact.

Where this model stops: Similarity does not establish truth, and biological memory categories are not one-to-one software modules.

A research question: How should a hybrid system resolve conflicting memories with different provenance?

Record evidence in your mastery worksheet →

THE NEXT LAYER

Chapter depth map.

This introduction provides a first model. The following chapters are outlines for deeper future editions; their full treatments have not yet been published.

Biological categories and mechanisms

17.1 Timescales and memory functions · Outline
  1. 17.1.1 Working, short-term and long-term memory
  2. 17.1.2 Episodic, semantic and associative memory
  3. 17.1.3 Behavioral categories versus neural mechanisms
  4. 17.1.4 Synaptic plasticity and consolidation
  5. 17.1.5 Species differences and limits of analogy
17.2 Dynamical memory · Outline
  1. 17.2.1 Recurrent attractors and persistent state
  2. 17.2.2 Noise, capacity and interference
  3. 17.2.3 Hippocampal-inspired models
  4. 17.2.4 Recall cues and pattern completion
  5. 17.2.5 Test mechanisms with perturbations

Artificial memory systems

17.3 Context, retrieval and storage · Outline
  1. 17.3.1 Transformer context and its boundaries
  2. 17.3.2 RAG and vector retrieval
  3. 17.3.3 External memory and agent state
  4. 17.3.4 Provenance, update policies and forgetting
  5. 17.3.5 Retrieval is not evidence that a claim is correct
17.4 Hybrid memory experiments · Outline
  1. 17.4.1 Fast state plus slower durable representations
  2. 17.4.2 Consolidation schedules as hypotheses
  3. 17.4.3 Retrieval accuracy and temporal consistency
  4. 17.4.4 Adversarial or conflicting memories
  5. 17.4.5 Compare with simpler task-specific state

Read with the original sources.

These readings support the concepts and deeper investigation. The worked numbers and illustrations above are teaching examples.

Complete source register →

Where next?