AI slows down experienced developers on familiar projects, study says

A new study has found that using cutting-edge artificial intelligence tools can slow down experienced programmers when working in familiar codebases, contradicting popular beliefs about AI's effectiveness, Reuters reports.

The non-profit organization METR conducted a study in which experienced open-source developers used Cursor, a popular AI programming assistant, to complete tasks on projects they were familiar with. Before starting, participants expected the AI to reduce task completion time by 24%. After completing the task, they were still confident in the benefits of AI, estimating a time reduction of 20%.

However, the actual results turned out to be the opposite: the use of AI increased task completion time by 19%.

Study co-author Nate Rush admitted he was shocked by the results, as he expected "a 2x acceleration, which seemed obvious."

These findings challenge the popular belief that AI always improves programmer productivity—the idea that underlies the multibillion-dollar investment in AI products for software development. AI is also often seen as a potential replacement for junior programming positions; Anthropic CEO Dario Amodei recently said that AI could eliminate half of white-collar jobs and increase unemployment by 10-20% over the next 5 years.

Previous studies have shown significant benefits from using AI. For example, one found that programmers’ work was accelerated by 56%, and another showed that the number of tasks completed increased by 26% in the same amount of time. However, the METR study shows that this data cannot be generalized to all scenarios.

The authors noted that for experienced developers who are well-versed in the specifics of large open-source projects, AI advice often required time to review and correct.

The researchers emphasized that their findings do not necessarily apply to junior specialists or engineers working in unfamiliar codebases - there AI can be much more useful.

Interestingly, despite the slowdown, most participants still use Cursor. The authors believe this is because AI makes the development process more enjoyable and less stressful, even if it doesn't make the work faster.