Walking the Tightrope of LLMs for Software Development: A Practitioners' Perspective
Samuel Ferino, Rashina Hoda, John Grundy, Christoph Treude
2025-11-12
Summary
This research explores how Large Language Models, like advanced AI chatbots, are changing the world of software development, looking at both the good and the bad sides of using them.
What's the problem?
While everyone is talking about how LLMs *could* help software developers, there hasn't been much solid research on the actual, real-world effects of using them. It's important to understand not just if they help, but how they change things for developers, their teams, and the industry as a whole, including potential downsides.
What's the solution?
The researchers interviewed 22 software professionals over a year, asking them about their experiences with LLMs. They didn't just ask once, but in three rounds to get a really thorough understanding. They used a specific method of analyzing the interviews, called socio-technical grounded theory, to carefully identify patterns and themes in what the developers said about the benefits, drawbacks, and best ways to use these AI tools.
Why it matters?
This study is valuable because it gives practical advice to team leaders and managers about whether or not to use LLMs, and how to do it effectively. It highlights the trade-offs involved – the advantages you gain versus the potential problems – so companies can make informed decisions about integrating these new technologies into their workflow and understand the impact on their developers.
Abstract
Background: Large Language Models emerged with the potential of provoking a revolution in software development (e.g., automating processes, workforce transformation). Although studies have started to investigate the perceived impact of LLMs for software development, there is a need for empirical studies to comprehend how to balance forward and backward effects of using LLMs. Objective: We investigated how LLMs impact software development and how to manage the impact from a software developer's perspective. Method: We conducted 22 interviews with software practitioners across 3 rounds of data collection and analysis, between October (2024) and September (2025). We employed socio-technical grounded theory (STGT) for data analysis to rigorously analyse interview participants' responses. Results: We identified the benefits (e.g., maintain software development flow, improve developers' mental model, and foster entrepreneurship) and disadvantages (e.g., negative impact on developers' personality and damage to developers' reputation) of using LLMs at individual, team, organisation, and society levels; as well as best practices on how to adopt LLMs. Conclusion: Critically, we present the trade-offs that software practitioners, teams, and organisations face in working with LLMs. Our findings are particularly useful for software team leaders and IT managers to assess the viability of LLMs within their specific context.