Kulbir Minhas, AI Initiative. Published 6 November 2026. This piece comes from the AI Initiative learning library: each cited claim was checked against its source before use, mostly at abstract level; Voyager was checked against specified paper sections and Table 1. One independent reviewer checked the wording against those claims, and a second, from another model family, confirmed that each reference resolves to the cited paper. Preprints are marked.
Why we are asking
Our aim is to build bottom-up, combining small, testable capabilities into larger ones. Our hypothesis is that combining memory, review and exploration could help an agent learn. What does the research say so far? It supports parts of the idea, and the parts that complicate it are the most useful.
What supports the idea
Removing Voyager's reusable skill library made later-stage exploration plateau (preprint) [1]. The ablation supports the library's contribution to exploration in Voyager.
ExpeL and Reflexion report improvements without updating model weights [2][3]. ExpeL agents extract natural-language insights from training-task experience and improve as that experience accumulates, without weight updates [2]. Reflexion agents that write verbal reflections into an episodic memory buffer improve on later trials without updating model weights [3].
Multi-agent debate improved mathematical and strategic reasoning, and factual validity, across several tasks (preprint) [4].
What complicates it
Gains were often minimal across the multi-agent traces studied (preprint) [5]. Across 1,600+ traces from seven multi-agent frameworks, gains were often minimal; the authors built a taxonomy of 14 failure modes in three categories from 150 annotated traces.
A held-out selection gate mattered in the evaluated natural-language-state methods (preprint) [6]. In those methods, self-improvement loops overfit or regressed without a held-out selection gate.
GateMem's evaluated baselines fell short of combining strong task utility, robust access control and reliable agent-facing forgetting (preprint) [7]. Behavioural forgetting is not verified erasure.
What we take from this
We take these studies as motivation to test reusable capabilities and held-out checks in our own system. Our proposed build order is a library with provenance, then an independent check, then additional agents. That is the order we plan to test on our own capabilities, and we will report what combined and what did not.
References
- Wang, G. et al. (2023). Voyager: an open-ended embodied agent with large language models. Preprint, arXiv:2305.16291. https://arxiv.org/abs/2305.16291
- Zhao, A. et al. (2024). ExpeL: LLM agents are experiential learners. AAAI 2024. https://doi.org/10.1609/aaai.v38i17.29936
- Shinn, N. et al. (2023). Reflexion: language agents with verbal reinforcement learning. NeurIPS 2023. https://arxiv.org/abs/2303.11366
- Du, Y. et al. (2023). Improving factuality and reasoning in language models through multiagent debate. Preprint, arXiv:2305.14325. https://arxiv.org/abs/2305.14325
- Cemri, M. et al. (2025). Why do multi-agent LLM systems fail? Preprint, arXiv:2503.13657. https://arxiv.org/abs/2503.13657
- Recursive self-evolving agents via held-out selection (2026). Preprint, arXiv:2606.28374. https://arxiv.org/abs/2606.28374
- GateMem: benchmarking memory governance in multi-principal shared-memory agents (2026). Preprint, arXiv:2606.18829. https://arxiv.org/abs/2606.18829