Neural Computers
Mingchen Zhuge, Changsheng Zhao, Haozhe Liu, Zijian Zhou, Shuming Liu, Wenyi Wang, Ernie Chang, Gael Le Lan, Junjie Fei, Wenxuan Zhang, Yasheng Sun, Zhipeng Cai, Zechun Liu, Yunyang Xiong, Yining Yang, Yuandong Tian, Yangyang Shi, Vikas Chandra, Jürgen Schmidhuber
2026-04-09
Summary
This paper introduces Neural Computers, a new type of computing system that tries to blend the functions of a computer's brain (processing), memory, and input/output all into one learning system.
What's the problem?
Traditional computers need explicit instructions, AI agents need to interact with the world, and world models try to predict how the world works, but all have limitations. The core problem this paper addresses is whether a system can *become* the computer itself, learning to operate just by observing what happens when someone interacts with a computer interface, without needing to know the underlying code or program state.
What's the solution?
The researchers created Neural Computers modeled as video systems. These systems watch what happens on a computer screen – the pixels, user actions like mouse clicks and keyboard inputs, and any instructions given – and learn to predict what will happen next. They specifically focused on whether these systems could learn basic computer tasks like understanding where things are on the screen and performing simple actions, all from just watching and learning from the input and output.
Why it matters?
If successful, Neural Computers could lead to a completely new way of computing, going beyond what current computers, AI agents, and world models can do. This could mean computers that are more adaptable, easier to reprogram, and capable of reusing learned skills, ultimately creating a more powerful and flexible computing paradigm.
Abstract
We propose a new frontier: Neural Computers (NCs) -- an emerging machine form that unifies computation, memory, and I/O in a learned runtime state. Unlike conventional computers, which execute explicit programs, agents, which act over external execution environments, and world models, which learn environment dynamics, NCs aim to make the model itself the running computer. Our long-term goal is the Completely Neural Computer (CNC): the mature, general-purpose realization of this emerging machine form, with stable execution, explicit reprogramming, and durable capability reuse. As an initial step, we study whether early NC primitives can be learned solely from collected I/O traces, without instrumented program state. Concretely, we instantiate NCs as video models that roll out screen frames from instructions, pixels, and user actions (when available) in CLI and GUI settings. These implementations show that learned runtimes can acquire early interface primitives, especially I/O alignment and short-horizon control, while routine reuse, controlled updates, and symbolic stability remain open. We outline a roadmap toward CNCs around these challenges. If overcome, CNCs could establish a new computing paradigm beyond today's agents, world models, and conventional computers.