From Static Templates to Dynamic Runtime Graphs: A Survey of Workflow Optimization for LLM Agents
Ling Yue, Kushal Raj Bhandari, Ching-Yun Ko, Dhaval Patel, Shuxin Lin, Nianjun Zhou, Jianxi Gao, Pin-Yu Chen, Shaowu Pan
2026-03-25
Summary
This paper is a comprehensive overview of how researchers are building complex systems using large language models (LLMs). These systems don't just rely on the LLM itself, but combine it with other tools like search engines, code execution, and memory to tackle more difficult tasks.
What's the problem?
As LLMs get better, people are trying to use them to do more than just answer questions. They want LLMs to *act* – to plan and execute a series of steps to achieve a goal. The problem is that designing these multi-step processes, called workflows, is hard. There are many different ways to structure a workflow, and it's difficult to figure out which structure will work best for a given task, and how to improve those structures.
What's the solution?
The paper organizes existing research on these workflows by looking at *when* the workflow is decided. Some methods create a fixed workflow beforehand, like a pre-written recipe. Others decide the workflow on the fly, adapting it to each specific problem. They also categorize research based on *what* part of the workflow is being improved – is it the order of steps, the tools used, or how information is passed between them? Finally, they look at *how* researchers are evaluating these workflows, going beyond just whether the final answer is correct to also consider things like efficiency and how much the workflow changes with different inputs.
Why it matters?
This work is important because it provides a common language and framework for understanding and comparing different approaches to building LLM-powered systems. It helps researchers avoid reinventing the wheel and provides a clearer path for future improvements in making these systems more reliable, efficient, and adaptable.
Abstract
Large language model (LLM)-based systems are becoming increasingly popular for solving tasks by constructing executable workflows that interleave LLM calls, information retrieval, tool use, code execution, memory updates, and verification. This survey reviews recent methods for designing and optimizing such workflows, which we treat as agentic computation graphs (ACGs). We organize the literature based on when workflow structure is determined, where structure refers to which components or agents are present, how they depend on each other, and how information flows between them. This lens distinguishes static methods, which fix a reusable workflow scaffold before deployment, from dynamic methods, which select, generate, or revise the workflow for a particular run before or during execution. We further organize prior work along three dimensions: when structure is determined, what part of the workflow is optimized, and which evaluation signals guide optimization (e.g., task metrics, verifier signals, preferences, or trace-derived feedback). We also distinguish reusable workflow templates, run-specific realized graphs, and execution traces, separating reusable design choices from the structures actually deployed in a given run and from realized runtime behavior. Finally, we outline a structure-aware evaluation perspective that complements downstream task metrics with graph-level properties, execution cost, robustness, and structural variation across inputs. Our goal is to provide a clear vocabulary, a unified framework for positioning new methods, a more comparable view of existing body of literature, and a more reproducible evaluation standard for future work in workflow optimizations for LLM agents.