Home/Research/Hierarchical Reasoning Model
Research · Architecture

A 27M-parameter model that reasons in two timescales.

The Hierarchical Reasoning Model pairs a fast, detailed L-module with a slow, strategic H-module, halts adaptively through a learned Q-head, and trains in constant memory — reaching 40.3% on ARC-AGI-1 from 1000 examples, with no pre-training and no chain-of-thought.

Talk to a solutions architect
27M
Parameters, 512 hidden size, 4+4 H and L layers
40.3%
ARC-AGI-1, ahead of o3-mini
98.5%
Sudoku Extreme accuracy; 99% optimal paths on 30×30 mazes
1000
Training examples — no pre-training, no chain-of-thought
Architecture

Two modules, two timescales, one halting decision.

Input is embedded as tokens plus a puzzle embedding and position encoding. From there the work splits between a module that moves every timestep and one that moves every cycle.

Input processing

Token embedding + puzzle embedding + position encoding.

L-module — fast, detailed

  • 4 transformer layers
  • Updates every timestep
  • Participation ratio 30.22

H-module — slow, abstract

  • 4 transformer layers
  • Updates every L_cycles
  • Participation ratio 89.95

Q-head for ACT

Halt or continue, chosen between Q(halt) and Q(continue).

Per segmentInput passes through the hierarchical modules; the L-module runs four times, the H-module updates twice, and ACT decides whether to continue or halt.
Training

One-step gradients, constant memory.

All intermediate states are detached, so there is no backpropagation through time. Memory stays O(1) where BPTT would be O(T) — which is what makes the deep supervision signal at every segment affordable.

Detached statesNo BPTT; the one-step gradient approximation gives O(1) memory complexity.
Q-learning configurationState is the hidden representation z_H; actions are binary halt/continue; reward is the task accuracy signal; learning happens online during training.
Explorationε-greedy at ε = 0.1 with dynamic min_steps sampling — enough computation is guaranteed, and premature halting is prevented.
Key properties

What the hierarchy buys.

Hierarchical convergence

The L-module converges locally while the H-module supplies global context.

Multi-timescale

A fast L-module every step against a slow H-module every L_cycles.

No BPTT

One-step gradient approximation, O(1) memory.

Adaptive computation

Halting decided by Q-learning rather than a fixed step budget.

Deep supervision

A learning signal at every segment, not only at the end.

Brain correspondence

The PR ratio of 89.95 to 30.22 matches the mouse cortical hierarchy; the oscillatory schedule is drawn from theta-gamma coupling, and credit assignment stays local.

Implementation

The forward pass.

for segment in range(max_segments): # Reset carry for halted sequences carry = reset_carry(halted, carry) # Hierarchical computation for h_step in range(H_cycles): for l_step in range(L_cycles): z_L = L_level(z_L, z_H + input) z_H = H_level(z_H, z_L) # ACT decision q_halt, q_continue = q_head(z_H) if q_halt > q_continue: break

Want the architecture applied to your reasoning workload?

We can walk through where a two-timescale model fits against your task mix, and what it costs to train and serve.

Talk to a solutions architect