HeLM: Hierarchical Explicit Language Memory for Non-Markovian Manipulation

HeLM: Hierarchical Explicit Language Memory for Non-Markovian Manipulation

Most vision-language-action (VLA) robot policies make a convenient assumption: the current camera view contains enough information to choose the next action. That assumption breaks when two visually identical situations require different actions because the robot must remember what happened earlier.

Consider a robot instructed to press red, green, and blue buttons in order. After pressing red, the scene may look almost unchanged. A memoryless policy can press red again because it has no explicit record of completed work. Similar problems appear when counting repetitions, wiping only uncleaned areas, finding an object behind drawers, or using information gathered during a previous instruction.

In HeLM: Hierarchical Explicit Language Memory for Non-Markovian Manipulation, Gyeonghun Kim and colleagues at Seoul National University propose storing task-relevant history in explicit language. Their central idea is that a compact textual state, such as “Progress: red, green” or “Count: 2,” can give a robot an interpretable working record without retaining every historical image.

The problem: manipulation needs more than the current frame

The authors describe these tasks as non-Markovian: the current observation alone is insufficient to infer the underlying state. Existing approaches address this gap in several ways, including latent memory features, compressed historical tokens, stacked frames, and visual keyframes. These methods can preserve history, but they either make failures difficult to inspect or accumulate expensive visual information as the task grows longer.

HeLM instead treats language as an explicit interface between perception, memory, and action. The text representation can record counts, sequences, locations, and progress in a form that researchers can inspect after a failure. Current visual input still matters: language records abstract history, while the live observation supplies immediate spatial grounding.

How HeLM works

HeLM divides control into two levels. A high-level policy (HLP), built from a vision-language model, manages memory and issues action commands such as “press the green button.” A low-level policy (LLP), implemented with a VLA model, turns those commands and the current observation into robot actions.

The HLP maintains two memory fields:

  • Working memory tracks progress within the current instruction, such as which buttons have been pressed or which window regions have been wiped.
  • Episodic context preserves a summary across instruction boundaries, allowing a later task to use information gathered during an earlier one.

Memory is not rewritten after every control step. HeLM uses a two-stage process. In detect mode, the HLP checks whether a predefined task-relevant event is visible, such as “blue button pressed.” In update mode, it incorporates that event into the language memory and selects the next command. If no event is detected and the instruction has not changed, the existing memory remains unchanged.

This event-triggered design is intended to reduce unnecessary high-level inference and prevent the memory from changing merely because the camera view varies. The training data is generated from short subtask episodes and TaskSpec descriptions that define valid events, commands, and memory transitions. The authors report that this lets them synthesize longer logical scenarios without manually recording every long-horizon variant.

What the experiments found

The evaluation covers 10 real-world non-Markovian tasks with 56 variants. Five tasks test working memory within one episode; five require episodic context across task boundaries. The tasks involve counting, order-sensitive sequences, wiping regions, object search, and multi-step pick-and-place behavior.

On the intra-episode suite, HeLM achieved an average subtask success rate of 85.7%, compared with 72.9% for the visual keyframe HLP, 39.5% for the PCMB-π0 baseline, and 16.7% for the naïve π0 baseline. Under the stricter measure requiring every subtask, correct order, and proper termination, HeLM reached 81.3%, while the keyframe method fell to 29.3% and both π0-based baselines fell to zero.

The cross-episode results are particularly notable. HeLM achieved an average subtask success rate of 89.2% across five inter-episode tasks. The keyframe HLP reached 16.6%. The other two baselines were not included in this comparison because they reset internal state at episode boundaries and were not designed to retain context across instructions.

The paper also reports 78% average subtask success on held-out compositional variants, where the HLP had not received additional task-specific fine-tuning. On LIBERO-Mem, HeLM achieved 58.5% average success versus 36.5% for a memoryless π0.5 baseline, with the largest gains on tasks requiring persistent temporal state.

For long-horizon scaling, the authors report that HeLM used up to 4.5 times less memory and ran 2.5 times faster than the keyframe baseline at long horizons. In the reported eight-event comparison, keyframe memory reached 28 GB of peak VRAM, while HeLM stayed under 6 GB. Text generation creates some initial overhead, but visual-token accumulation makes the keyframe approach increasingly expensive.

Where HeLM still fails

The results do not show that language memory solves manipulation generally. HeLM’s observed failures were dominated by visual event detection and low-level execution. The authors attribute 64.1% of its categorized failures to detection and 31.2% to low-level control; order failures were rare and termination failures were not observed in their reported analysis.

False-positive event detections are especially damaging. If the system incorrectly concludes that a drawer opened or a task finished, it updates memory with a false fact and may choose the wrong next command. Separately, the low-level policy can reach the correct location but fail to make reliable physical contact with a button or object.

There are also structural limitations. HeLM depends on predefined task-relevant events supplied through TaskSpec, so it does not discover automatically what should be remembered. The evaluation emphasizes discrete events rather than continuous quantities such as liquid levels or precise geometric trajectories. Synchronous, chunk-level HLP calls can miss transient events occurring inside an action chunk. Finally, the paper does not establish whether separate working and episodic fields are better than one persistent language memory.

Practical implications

For robotics researchers, HeLM suggests that memory representation deserves the same attention as the action backbone. Explicit language offers a useful debugging surface: engineers can inspect the recorded state, identify a false update, and separate reasoning errors from perception or motor failures.

For deployment, the approach is most promising for tasks with discrete, inspectable milestones and clear instruction boundaries. It may be less suitable when success depends on continuous physical state or precise spatial history. A practical system would need conservative event detection, recovery from incorrect updates, and testing beyond the paper’s controlled task vocabulary.

Overall, Kim and colleagues demonstrate that a robot can use compact language summaries to carry progress within an episode and context between episodes. The evidence supports language memory as a strong design option for non-Markovian manipulation, while leaving open the harder question of how a robot should decide, without task-specific schemas, what is worth remembering.


Sources

AI-assisted research summary. Check the original paper for complete methods, results, and limitations.

Avatar photo

Tech News

Jeff is a passionate blog writer who shares clear, practical insights on technology, digital trends and AI industries. With a focus on simplicity and real-world experience, his writing helps readers understand complex topics in an accessible way. Through his blog, Jeff aims to inform, educate, and inspire curiosity, always valuing clarity, reliability, and continuous learning.

↑ ↓ navigate Enter open Esc close