Home/Research/Nested Learning & HOPE
Research · NeurIPS 2025

A model is a hierarchy of nested optimization problems.

Nested Learning reads every component of a model — memory, projections, momentum, pre-training — as its own optimization problem running at its own frequency. HOPE applies it: a self-modifying Titans memory paired with a continuum memory system, where Level 1 keeps updating after training ends.

Talk to a solutions architect
The paradigm

Associative memory, all the way down.

Definition 1 frames memory as a mapping learned by minimising an objective — keys to values. Everything else in the model turns out to be the same shape at a different frequency.

M* = arg minM L̃(M(K); V) — memory learns to map keys to values by minimising L̃
MemoryA neural update caused by an input.
LearningThe process for acquiring effective and useful memory.
Definition 2

Update frequency is the hierarchy.

A higher level is a lower frequency: A ≻ B means fA > fB. Four levels, each with its own gradient flow and its own context.

Level 1Memory Mt — context compressionf = 1/token
Level 2Projections Wk, Wv, Wqf = 1/batch
Level 3Momentum — gradient compressionf = 1/batch
Level 4Pre-training — the outermost loopf = 1/epoch
In trainingAll levels update at their respective frequencies.
At inferenceOnly Level 1 updates. That is test-time learning — the model keeps learning without backpropagation.
TransformersA special case of CMS with k = 1, a single MLP. Every component freezes at inference, which the paper characterises as anterograde amnesia.
Architecture

Transformer against HOPE.

Transformer

Static after pre-training — CMS with k = 1.

qt = xtWq · kt = xtWk · vt = xtWv
softmax(QK⊤/√d)V
  • Static projections, frozen at inference
  • Feed-forward network frozen at inference
  • Cannot form new long-term memories
HOPE

Self-modifying Titans memory plus a continuum memory system.

Mt+1 = Mt(I − ktkt⊤) − η∇L
yt = Mt · qt (Eq. 14)
  • Memory stays active at inference
  • Delta rule (Eq. 28–29) accounts for token dependencies
  • Online learning at test time
Continuum Memory System

A stack of MLPs, each on its own clock.

Eq. 30: yt = MLP(fk)( … MLP(f1)(xt) )
Frequency hierarchyf1 > f2 > … > fk — MLP1 is fastest, MLPk slowest.
Update cadenceEach θ(fℓ) updates every C(ℓ) steps (Eq. 31).
Why it mattersMemory updates at inference through the delta rule, enabling continual learning, while CMS supplies multi-timescale knowledge.
Optimizers

Gradient descent with momentum is a two-level nested optimization.

Read the momentum term as a key-less associative memory compressing gradients, and the familiar update rule separates into an inner and an outer problem.

mi+1 = αi+1mi − ηt∇L(Wi; xi) (Eq. 17)
Wi+1 = Wi + mi+1 — Level 2, the weight update
mt+1 = arg minm −⟨m, ∇L(Wt; xt+1)⟩ + ηt+1‖m − mt‖² (Eq. 10)

More expressive objectives

L2 regression in place of dot-product gives the delta rule (Eq. 21–22).

More expressive memory

Replace linear momentum with an MLP — deep momentum GD (Eq. 23).

Non-linear outputs

σ(·) = Newton-Schulz yields the Muon optimizer (Eq. 24).

Better backprop

An L2 objective accounts for token dependencies (Eq. 28–29).

From §2.3: Adam with a small modification is the optimal associative memory for a model’s gradients.

Training and inference

What moves, and when.

Training — every level moves

Token arrivesLevel 1 updates memory Mt (Eq. 13) — inner optimization on a dot-product objective.
Batch completesLevel 2 backpropagates through the projections.
Loss computedLevel 3 updates momentum, itself a key-less associative memory.
Epoch completesLevel 4 aggregates all nested updates.

Inference — only Level 1 moves

ProjectThrough frozen Wk, Wv, Wq (Eq. 12) — slow weights holding knowledge consolidated in pre-training.
UpdateMemory updates by the delta rule (Eq. 28–29), which manages capacity better than a simple Hebbian update.
Queryyt = Mt·qt — fast memory combined with slow frozen knowledge.
Reference

Key equations.

Eq. 1M* = arg min L̃(M(K); V) — associative memory
Eq. 12–14Linear attention formulation
Eq. 17Gradient descent with momentum
Eq. 28–29Delta rule for HOPE
Eq. 30–31Continuum Memory System
Definition 2Update frequency hierarchy

Interested in test-time learning for your stack?

We can talk through where a continuously-updating memory helps, and what it costs to serve.

Talk to a solutions architect