Sharing State Between Prompts and Programs
Ellie Y. Cheng, Logan Weber, Tian Jin, Michael Carbin
2025-12-19
Summary
This paper explores a new way of programming that combines regular code, like Python, with instructions written in plain English, called natural language programming. It focuses on making these two types of code work seamlessly together.
What's the problem?
Traditionally, getting instructions written in English to actually *do* something in a program requires a lot of extra work. Programmers have to manually translate the English instructions into code the computer understands and manage how the English instructions access and change the program's data. This is tedious and makes it harder to use the power of these new 'large language model' tools.
What's the solution?
The researchers introduced a concept called 'shared program state'. Think of it as a common space where both the Python code and the English instructions can access and modify the same information. They created a system called Nightjar that allows you to write Python programs and embed English instructions directly within them, letting those instructions directly work with the program's variables and data. They designed a way to define how the English instructions interact with the program, making it easier for the system to understand what to do.
Why it matters?
This work is important because it makes natural language programming much more practical. By reducing the amount of manual work needed to connect English instructions to a program, it allows programmers to write code faster and potentially with fewer errors. The results showed that programs built with Nightjar were just as accurate, and often more accurate, than programs written entirely in Python, while also requiring less code overall, though there was a slight slowdown in how quickly the programs ran.
Abstract
The rise of large language models (LLMs) has introduced a new type of programming: natural language programming. By writing prompts that direct LLMs to perform natural language processing, code generation, reasoning, etc., users are writing code in natural language -- natural language code -- for the LLM to execute. An emerging area of research enables interoperability between natural language code and formal languages such as Python. We present a novel programming abstraction, shared program state, that removes the manual work required to enable interoperability between natural language code and program state. With shared program state, programmers can write natural code that directly writes program variables, computes with program objects, and implements control flow in the program. We present a schema for specifying natural function interfaces that extend programming systems to support natural code and leverage this schema to specify shared program state as a natural function interface. We implement shared program state in the Nightjar programming system. Nightjar enables programmers to write Python programs that contain natural code that shares the Python program state. We show that Nightjar programs achieve comparable or higher task accuracy than manually written implementations (+4-19%), while decreasing the lines of code by 39.6% on average. The tradeoff to using Nightjar is that it may incur runtime overhead (0.4-4.3x runtime of manual implementations).