Large language models need effective ways to acquire and use external knowledge beyond what they retain from pretraining. Continued pretraining risks catastrophic forgetting, while retrieval at inference time can add substantial overhead. External parametric memory avoids online retrieval, but constructing its training data requires extensive offline search, and output-level interpolation provides only shallow fusion. We introduce Deep Fusion Memory (DFM), a parametric memory architecture that reduces this construction workload and incorporates learned memory into intermediate hidden states. A causal memory predictor learns to generate memory from the preceding context, using retrieved chunk embeddings from the corpus as supervision. The predicted memory is shared across tokens within each chunk and fused into their hidden states through gated cross-attention. Memory prediction and fusion are trained end to end with the backbone frozen, and inference requires no explicit retrieval. Compared with existing parametric memory approaches, which often construct token-level memory targets, DFM requires 32times fewer retrieval queries and 2048times fewer retrieved items to construct its training data. Across six QA tasks, DFM performs better over the strongest baseline by 12.71% on Mistral-7B-v0.3 and 11.19% on Qwen3-8B-Base. Beyond QA, domain adaptation yields average gains of 6.51 and 4.38 points over Qwen3-4B-Base and Qwen3-8B-Base, respectively.

Deep Fusion Memory integrates predicted parametric memory into intermediate hidden states via gated cross-attention, reducing retrieval overhead and improving knowledge-intensive tasks without online retrieval.

Rubin Wei

Rubin-Wei

Siyuan Sun

Hao Sun

Yuliang Liu

Xiang Shi

Jiarui Wang

Junming Zhang

Zhouhan Lin

ZhouhanLin

Deep Fusion Memory: A Chunk-Level Parametric Memory for Large Language Models

Large language models need effective ways to acquire and use external knowledge beyond what they retain from pretraining. Continued pretraining risks catastrophic forgetting, while retrieval at inference time can add substantial overhead.

Despite substantial progress in Deep Search agents, their ability to conduct end-to-end research in real-world open Web environments remains insufficiently evaluated. Unlike static retrieval settings, information on the real-world Web is distributed across diverse sources, modalities, and heterogeneous formats, often involving complex interaction mechanisms that require agents to autonomously search for, access, and integrate relevant evidence during execution. To address this gap, we introduce the OpenDS task, designed to evaluate agents' ability to perform multi-step deep search in the live, open Web. This task requires agents to autonomously explore the real Web, discover and connect relevant evidence from multiple independent sources through Web search, browser interaction, and evidence processing, and ultimately determine a unique factual answer. Building on this task formulation, we construct OpenDS-World, a benchmark comprising 200 challenging tasks spanning diverse information modalities, heterogeneous file formats, and interactive access requirements, with independent human verification to ensure evidence validity and answer uniqueness. We evaluate 12 frontier models under a unified tool environment and execution protocol. The best-performing model achieves 39.0% accuracy, while human evaluation across 1,000 independent attempts yields 21.1% accuracy. Further analyses of tool usage, search cost, termination behavior, and failure recovery reveal substantial differences in research strategies across models, as well as persistent challenges in tool coordination and long-horizon search management. OpenDS-World provides a unified evaluation platform for systematically assessing and advancing agents' autonomous deep search capabilities in real-world open Web environments.

OpenDS-World introduces a live-web benchmark to evaluate autonomous multi-step deep search agents, revealing significant gaps between frontier models and human performance.

Xiaomin Yu

Hongliang Chen

hchen272

Zhe Huang

Shilin Yan

Shao-Rong Su

SSRRSS

Xuqiao Ma

Feiyang Wang

Jiayu Chen

Jiwei Fang

Yu Yan

Yini Huang

Nina-Huang

Chen Chen

Ruoxi Cheng

Yuzhi He

Yuhui Zhang

Xiaoxing Hu

Chen Liu

Jianbo Lin

Yijiang Li

Xiaobin Hu

Menglong Li

Zhuoran Lu

Weishi Wang

Zhongqi Yue

Chengwei Qin

Yu2020

Deep Search in Real-World Web Environments

Despite substantial progress in Deep Search agents, their ability to conduct end-to-end research in real-world open Web environments remains insufficiently evaluated. Unlike static retrieval settings, information on the real-world Web is distributed across diverse sources, modalities, and heterogeneous formats, often involving complex interaction mechanisms that require agents to autonomously search for, access, and integrate relevant evidence during execution.

Recursive self-improvement (RSI) describes a system that improves not only its behavior but also the process by which it improves. Recent digital agents test this idea by rewriting their own code and checking each version with executable tests. Bringing these mechanisms into the physical world requires learning from partial observations through costly trials, while accounting for changing operating conditions and potentially irreversible consequences. We position RSI within robot learning and trace pathways from improving robot capabilities to improving the processes that generate them. We examine how physical experience can strengthen these improvement mechanisms and when the resulting gains can support further self-improvement of the mechanisms themselves. Gains carry forward only when feedback addresses remaining learning needs, the receiving component can use the change, its value survives changes in tasks or hardware, and continued practice remains safe and affordable. Existing studies, concentrated in manipulation, support behavioral improvement, benefits to later learning, and procedure improvement. Some systems also change the improver's knowledge or tools, but the physical studies reviewed here have yet to isolate gains attributable to recursion. We synthesize these findings into evaluation principles and research directions for machines that, trial by trial, safely improve the way they improve.

NationalUniversityofSingapore

Meng Luo

Eureka-Leo

Zechen Bai

Shanqing Xu

Yanlin Li

Kaixin Li

Mengchen Qian

Weifeng Lu

Ran Zhang

Yingtian Zou

Ziyang Luo

Wei Chen

Hongzhan Lin

danielhzlin

Toward Recursive Self-Improvement in Physical AI

Recursive self-improvement (RSI) describes a system that improves not only its behavior but also the process by which it improves. Recent digital agents test this idea by rewriting their own code and checking each version with executable tests.

This paper shows that JEPA-style latent world models (LWMs) can learn across diverse domains and data sources by adopting the language-conditioned interface of video generation models. We establish this approach as an effective and efficient basis for planning with imagination and downstream policy fine-tuning. To this end, we introduce LaLa-World, a Language-conditioned Latent World model that learns from simulation and robot trajectories together with broader image and video data. Two designs make this work. First, we describe both numerical actions and visual changes in language, enabling joint training of a single future predictor across heterogeneous datasets. Second, we build the predictor on a language model, allowing it to interpret language conditions while predicting future latent states. The resulting predictor achieves higher average MPC planning performance than existing LWMs trained separately for each domain or task. Adding image and video data further improves average planning performance and robustness to action phrasing. In a matched comparison with an RAEv2-based video generation model baseline, LaLa-World reaches comparable planning performance with 6times less training cost and 13.9times faster planning. Beyond planning, the pretrained LaLa-World serves as an initialization for LaLa-Policy, which jointly predicts actions and future visual representations. LaLa-Policy performs competitively with Cosmos Policy on LIBERO and RoboCasa, with up to 5.5times faster inference.

naver-ai

LaLa-World is a language-conditioned latent world model that jointly learns from diverse simulation, robot, and video data to enable efficient model-predictive planning and policy initialization.

Junha Song

Geonmo Gu

Geonmo

Jeongeun Park

Heejeong Nam

Lucas Maes

Byeongho Heo

bhheo

Dongyoon Han

Sangdoo Yun

oodgnas

junha1125

LaLa-World: Scaling Latent World Models with Natural Language Actions

This paper shows that JEPA-style latent world models (LWMs) can learn across diverse domains and data sources by adopting the language-conditioned interface of video generation models. We establish this approach as an effective and efficient basis for planning with imagination and downstream policy fine-tuning.

High-quality image editing requires precise modification of target regions while preserving the content of unrelated regions. Existing editing models produce flattened outputs that limit precise local manipulation and layer reuse, while decomposition methods typically optimize layer recovery separately from instruction-driven editing. To this end, we introduce LayerEditor, a unified framework that couples instruction-guided image editing with controllable generation of reusable RGBA layers. At its core, In-Context RGB-Alpha Token Modeling jointly represents the edited canvas and an ordered sequence of RGB-Alpha layers within a single token sequence, enabling semantic edits and explicit control over individual layer content. To retain the pretrained model's editing capability during adaptation to layered outputs, we propose Compositional Edit-Prior Distillation, which transfers its editing prior to both the edited canvas and the image composed from the predicted layers. To support training and evaluation, we construct LayerEditor-100K, a large-scale paired image-editing dataset with ordered RGBA layer annotations, and introduce dedicated benchmarks for layered editing. Extensive experiments demonstrate that LayerEditor maintains strong editing fidelity while generating accurate RGBA layers, supporting both flexible layer-level manipulation and multi-layer decomposition.

jingdong1

LayerEditor unifies instruction-guided image editing with reusable RGBA layer generation via joint token modeling and edit-prior distillation, enabling precise local manipulation and layer reuse.

Wen Xue

Davidscut

Yong Liu

Xueheng Li

Si Wu

Yong Xu

Yipeng Sun

Yan Li

Simiu Gu

Editing Images into Reusable Layers via Compositional Edit-Prior Distillation

High-quality image editing requires precise modification of target regions while preserving the content of unrelated regions. Existing editing models produce flattened outputs that limit precise local manipulation and layer reuse, while decomposition methods typically optimize layer recovery separately from instruction-driven editing.

Fine-grained visual understanding requires distinguishing subtle differences in local attributes and spatial relations, yet remains challenging for existing vision-language models. Unified multimodal models (UMMs) offer a new opportunity through generation pathways that preserve rich visual information complementary to understanding. However, existing generation-assisted understanding methods mainly rely on generative supervision or explicit generation and have not systematically explored direct use of native generation-side representations. To address this gap, we systematically analyze what generation-side representations offer for fine-grained understanding, finding that they are more sensitive to fine-grained semantic changes, capture task-relevant changes missed by understanding-side representations, and depend on correct spatial organization for this advantage. Guided by these findings, we design GAFU, a post-training framework that unlocks native generation-side information for fine-grained understanding. It compresses generation-side visual representations into compact \(G\) tokens that preserve fine-grained visual details, enabling task-conditioned retrieval by the understanding pathway. Experiments show that GAFU consistently outperforms SFT baselines trained on the same data across diverse visual understanding tasks and UMM architectures, with maximum relative improvements of 20.00%, 17.94%, and 8.62% on VStar-Bench, OmniSpatial, and VSR-Bench, respectively. Overall, GAFU converts complementary information from the generation pathway into gains in fine-grained visual understanding, providing an effective path toward synergy between generation and understanding in UMMs.

GAFU leverages native generation-side representations to improve fine-grained visual understanding in unified multimodal models by compressing them into compact tokens for task-conditioned retrieval.

Tengju Ru

Lruca

Weitong Lian

Zecong Tang

zecongtanghku

Zhiheng Liu

Lingyi Meng

Haoran Li

Kaixuan Wang

kaixuanwang

Bohao Li

Xihui Liu

Fine-grained visual understanding requires distinguishing subtle differences in local attributes and spatial relations, yet remains challenging for existing vision-language models. Unified multimodal models (UMMs) offer a new opportunity through generation pathways that preserve rich visual information complementary to understanding. However, existing generation-assisted understanding methods mainly rely on generative supervision or explicit generation and have not systematically explored direct use of native generation-side representations. To address this gap, we systematically analyze what generation-side representations offer for fine-grained understanding, finding that they are more sensitive to fine-grained semantic changes, capture task-relevant changes missed by understanding-side representations, and depend on correct spatial organization for this advantage. Guided by these findings, we design GAFU, a post-training framework that unlocks native generation-side information for fine-grained understanding. It compresses generation-side visual representations into compact \(G\) tokens that preserve fine-grained visual details, enabling task-conditioned retrieval by the understanding pathway. Experiments show that GAFU consistently outperforms SFT baselines trained on the same data across diverse visual understanding tasks and UMM architectures, with maximum relative improvements of 20.00\%, 17.94\%, and 8.62\% on VStar-Bench, OmniSpatial, and VSR-Bench, respectively. Overall, GAFU converts complementary information from the generation pathway into gains in fine-grained visual understanding, providing an effective path toward synergy between generation and understanding in UMMs.

What Generation Knows: Unlocking Fine-Grained Understanding in Unified Multimodal Models

Fine-grained visual understanding requires distinguishing subtle differences in local attributes and spatial relations, yet remains challenging for existing vision-language models. Unified multimodal models (UMMs) offer a new opportunity through generation pathways that preserve rich visual information complementary to understanding.

We present IQuest-Q1, a sparse Mixture-of-Experts (MoE) agentic foundation model with 320B total parameters (15B activated). IQuest-Q1 is developed through a three-stage process: (1) stable and efficient pre-training, combining Stable SplitNorm, Muon, and muP with a two-stage curriculum whose second stage increases the coding-data mixture, preserving a wide stability margin between safe and optimal learning rates; (2) fine-grained mid-training, expanding the data mixture across domains and formats and progressively extending the context; and (3) multi-scaffold with multi-version post-training, combining SFT, domain-specialized RL, and MOPD across various scaffolds to integrate complementary training signals and strengthen agentic coding capabilities. Furthermore, model merging combines different experts and checkpoints from different stages while mitigating conflicts among their parameter updates. The resulting IQuest-Q1 demonstrates capabilities in both coding and general tasks. We release IQuest-Q1 as an open-source model to support community research and development.

IQuestLab

IQuest-Q1 is a sparse Mixture-of-Experts foundation model trained via multi-stage pre-training, mid-training, and post-training to achieve strong agentic coding and general capabilities.

A. Jian Yang

An Huang

Bo Dan

Chen Mu

Chengran Yang

Chuan Shi

Feng Chang

Hongxin Li

Hongyuan Tao

Jinxing Zhang

Jujie He

Kun Wang

Lin Jing

Ming Yang

Qicheng Hu

Renyuan Li

Shark Liu

Shawn Guo

Shi Qing

Shi Zhong

Tianyu Zheng

Wei Liu

Wei Zhang

Wenrui Cai

Wenxiao Chen

Xuxing Chen

Yan Xing

Yi Xing

Yifan Luo

ArgusK17

Yiran Xing

Yizhi Li

Yongsheng Kang

Yucheng Ding

Yue Song

Yun Qing

Yunqi Liu

Zelong Huang

Zhengmao Ye

Zhentao Fan

Zhiteng Li

Ziling Huang

Bihong Tang

Bingli Wang

Ce Jun

Changhao Chen

Chen Hu

Chen Xiao

Chi Liu

Chunjiang Ge

Danyang Li

Deyang Kong

Dian Qi

Ding Tan

Evan Yang

Fanglin Xu

Fei Bai

Gang Liu

Guochang Li

Hao Tang

Haoran Feng

Haowen Wang

Huahao Hu

Huanjin Yao

Huatong Song

Jia Deng

Jiajun Wu

Jialiang Cheng

Jiayi Liang

Jiayi Wu

Jie Deng

Jiejun Tan

Jing Yang

Jingda Xu

Junhang Cheng

Junxiao Liu

Kai Lv

Kaixiang Pan

Kecheng Lu

Kun Zeng

Kun Zheng

Kunbo Ding

Liang Qiu

Lingyi Sun

Luan Zhang

IQuest-Q1: Advancing Foundation Capabilities for Agentic CLI Systems

We present IQuest-Q1, a sparse Mixture-of-Experts (MoE) agentic foundation model with 320B total parameters (15B activated).

AI Research Made Simple

Deep Fusion Memory: A Chunk-Level Parametric Memory for Large Language Models

Deep Search in Real-World Web Environments

Toward Recursive Self-Improvement in Physical AI

LaLa-World: Scaling Latent World Models with Natural Language Actions

Editing Images into Reusable Layers via Compositional Edit-Prior Distillation

What Generation Knows: Unlocking Fine-Grained Understanding in Unified Multimodal Models

IQuest-Q1: Advancing Foundation Capabilities for Agentic CLI Systems

Meta Expert: Cross-Layer Expert Sharing for Storage- and Retrieval-Efficient MoE

Policy-Mask: Budget-Aware Adaptation of Diffusion Features to Discriminative Tasks

Self-Distilling Future Video Representations for World Action Models

Making MLLM Latents Matter: Factor-Aware Predictive Learning for Cinematic Video Generation

WeVisOne-VL: Unified Visual Perception from Objects to Pixels