A recent study by the AI research nonprofit METR in San Francisco challenged the notion that cutting-edge artificial intelligence tools can always boost the productivity of experienced software developers. The study involved seasoned developers using Cursor, a popular AI coding assistant, to complete tasks in familiar open-source projects.
Prior to the study, developers believed AI would speed up their work by 24%, but the actual results showed a 19% increase in task completion time. The lead authors of the study, Joel Becker and Nate Rush, were surprised by the findings, as they had expected a 2x speed increase.
The study highlights that AI may not always enhance productivity in all software development scenarios. In this case, developers familiar with complex open-source codebases experienced a slowdown due to the need to correct and adjust the suggestions made by the AI models.
While the study participants and authors continue to use Cursor, they acknowledge that the slowdown may not apply to other situations, such as junior engineers or those working in unfamiliar codebases. Despite the challenges, many developers find using AI tools like Cursor makes the development process more enjoyable and less effortful.
Overall, the study’s results provide important insights into the impact of AI on software development and caution against assuming that AI will always accelerate productivity in all scenarios.
