Agent READMEs: An Empirical Study of Context Files for Agentic Coding
Worawalan Chatlatanagulchai, Hao Li, Yutaro Kashiwa, Brittany Reid, Kundjanasith Thonglek, Pattara Leelaprute, Arnon Rungsawang, Bundit Manaskasemsak, Bram Adams, Ahmed E. Hassan, Hajimu Iida
2025-11-19
Summary
This paper investigates how developers are using 'agent context files' – essentially instructions for AI agents that write code – to understand how these files are created, maintained, and what kind of information they contain.
What's the problem?
AI agents are becoming more common for writing code, and they rely on these context files to know *what* to do. However, it wasn't clear how developers actually use these files in practice. Are they simple documentation, or something more complex? The researchers found that these files aren't just simple explanations; they're constantly changing, often with small updates, and can be hard to understand, almost like a type of code itself. More importantly, developers focus heavily on making the agent *work* but often neglect important details like security and performance.
What's the solution?
The researchers analyzed over two thousand of these context files from nearly two thousand different software projects. They looked at how the files changed over time and categorized the types of instructions developers were including. This large-scale analysis revealed patterns in how these files are used and what information is typically included, or, crucially, *not* included.
Why it matters?
This research is important because it shows that while we're good at telling AI agents *what* to build, we're not very good at telling them *how* to build it safely and efficiently. This means AI-generated code could have security vulnerabilities or perform poorly. The findings highlight the need for better tools and guidelines to help developers provide more complete instructions to these agents, ensuring the code they create is not only functional but also reliable and secure.
Abstract
Agentic coding tools receive goals written in natural language as input, break them down into specific tasks, and write or execute the actual code with minimal human intervention. Central to this process are agent context files ("READMEs for agents") that provide persistent, project-level instructions. In this paper, we conduct the first large-scale empirical study of 2,303 agent context files from 1,925 repositories to characterize their structure, maintenance, and content. We find that these files are not static documentation but complex, difficult-to-read artifacts that evolve like configuration code, maintained through frequent, small additions. Our content analysis of 16 instruction types shows that developers prioritize functional context, such as build and run commands (62.3%), implementation details (69.9%), and architecture (67.7%). We also identify a significant gap: non-functional requirements like security (14.5%) and performance (14.5%) are rarely specified. These findings indicate that while developers use context files to make agents functional, they provide few guardrails to ensure that agent-written code is secure or performant, highlighting the need for improved tooling and practices.