The Pragmatic Mind of Machines: Tracing the Emergence of Pragmatic Competence in Large Language Models

Summary

This paper talks about how large language models, like the ones used in chatbots, develop the ability to understand and use language in a practical, real-world way, which is called pragmatic competence.

What's the problem?

The problem is that while language models can get really good at grammar and vocabulary, it's much harder for them to pick up on the subtle, practical ways people use language in everyday life, like understanding jokes, hints, or indirect requests. Without this skill, their responses can sound robotic or miss the point.

What's the solution?

The authors created a special test set called ALTPRAG to measure how well these models handle real-world language situations at different stages of their training. They found that as the models get bigger and are trained in certain ways, their ability to understand and use language pragmatically gets better.

Why it matters?

This is important because it shows how we can make AI systems that communicate more naturally and helpfully, making them better at understanding what people really mean and improving their usefulness in conversations, customer service, and more.