CircleCI data analysis shows stark disparities between development teams: a minority of teams achieve dramatic productivity gains with AI, while others waste a lot of time fixing errors in automatically generated code.
If you're a software developer, you need to read this article. If you manage software developers, it's even more important for you. In fact, it's important for anyone whose work is being automated these days: translators, lawyers, doctors, and even futurists.
Why? Because it answers a question that has been bothering me for several months: How is it that some people report doubling their productivity and more thanks to artificial intelligence, while others complain that it only slows them down?
To answer this question, we needed a higher level of supervision. Literally. To sit behind the professional's shoulder and check what he's doing, how much he's doing, and how much artificial intelligence has improved his work. Or how much it has slowed down the pace.
Luckily, there is at least one company that has produced this kind of data. The company's name is CircleCI, and it provides one of the most popular platforms on the market for programmers to work on. It also monitors the sites where programmers host their temporary work products. And last September, for an entire month, it Monitored thousands of software development teams To understand what's really going on there. Is artificial intelligence accelerating their work pace? Or is it not making any significant change at all?
And the answer: yes… and yes.
The company’s analysis showed that the average team’s productivity rate has increased by 59 percent over the past three years. But a closer look at the data shows that this average reflects a very uneven distribution. A quarter of teams see no improvement in pace. Half of teams see a negligible improvement (four percent). Only then, in the top quarter of teams, do we start to see dramatic improvements in development pace. The most successful teams are doubling their development pace every year. And if we look at the thirty most advanced teams, we see even greater disparities: the most advanced team produced ten times more code in 2025 than the leading team in 2024.
To put things in context: this means that the ten fastest teams verify and approve more than 10,000 code changes every day.
So why do some developers swear that artificial intelligence slows them down? That it writes bad code? That it makes mistakes like a little child?
Because she really is.
When artificial intelligence is wrong
Researchers tracked programmers’ attempts to generate new code with the help of AI, and found that they were only successful 70 percent of the time – a significant drop compared to the past two years. This means that AI is generating code with more errors. Not only that, but programmers are also having a harder time dealing with its errors. The average programmer now has to spend 72 minutes fixing AI, which is 13 percent more than a year ago.
What does this mean in practice? Teams that try to make five code changes a day, with only a seventy percent success rate, are effectively wasting 250 hours of work a year. Larger teams, trying to push 500 changes a day, are effectively wasting 12 full-time engineers, who have to fix all the AI mistakes.
Do you understand now why so many programmers complain that artificial intelligence is mostly harmful?
But how does all of this fit with the fact that the top teams manage to double, or even tenfold, their productivity rate?
The small and big winners
The researchers analyzed the data even more deeply and concluded that the winning teams come from two types of companies: very small and very large. In both types of companies we see the greatest productivity and the fastest ability to correct mistakes.
The very small companies – the bastards – are rushing forward in every way possible. They are less bound by old ways of working, or by bureaucracy that grinds away at the cogs. They have five employees at most, and they work at a feverish pace, reinventing their ways of working. Does the AI make mistakes? So they develop other AIs to monitor the code and warn about errors. Every employee there uses all the tools they have in innovative ways to produce code quickly, review it quickly, fix it quickly, and launch it quickly.
The very large companies, with more than a thousand employees, also realized that they could not be left behind. They hired the smartest consultants on the market, and the most impressive talents, and redesigned their software development process. I guess – although it is not explicitly mentioned in the report – that they invested in smaller teams themselves, and encouraged each of the teams to refine and reinvent their work pattern. And they succeeded. Fact.
Who's left behind? The mid-sized companies. They produce code at an agonizingly slow pace, and it takes them more than three times as long to approve and implement that code.
Another inhibiting factor is the level of complexity of the code. The larger the code base – as is the case in the computer software and financial services sectors – the longer it takes to fix errors. In other sectors, such as civil engineering or infrastructure, the code base is not as large, so AI enjoys a higher success rate in writing new code.
Data summary
What can we understand from all this data?
First of all, the effects of artificial intelligence on software development are not uniform between different teams and different companies. Only a few of the teams – less than a quarter – have managed to adapt their work patterns to the new era. The rest are still struggling with artificial intelligence and complaining that it slows them down. And it really does slow them down. Both because they themselves don't know how to get the best and most out of it, and also because the way the teams work and manage it hasn't changed yet. They are stuck in the distant past, two or three years ago.
Second, the small ones win because they can – must – move quickly. They use all the tools at their disposal, and don’t overthink the work patterns that have become established in recent years. And the big ones? They understand the magnitude of the threat, and so they invest in re-engineering their teams so they can get the most out of the new tools.
Third, artificial intelligence makes plenty of mistakes. Anyone who claims otherwise is spreading hype. The point is that if it is used correctly – and this probably means that it should also be used to develop tests and automate code transitions – it speeds up the overall work rate by two or even ten times.
The overall vision
All of these lessons are relevant not only to software development. They are relevant to every profession that artificial intelligence is beginning to integrate into. In law, translation, market research and scenario development, and all other fields. In all of them, we see a similar pattern: Those who know how to use artificial intelligence, and whose organization allows them to find the right ways of working alongside it, can produce quality products faster. Those who don't know how to use it, or whose workplace restricts them to slow and clumsy practices, continue to complain that it doesn't help them.
As in software development, in other fields we constantly hear the complaint that artificial intelligence is wrong. And as in software development, it is true. It really is wrong, or it is not 'brilliant', or it is delusional and makes mistakes and errors. But those who know how to use it in the right places in their workflow are successful. Those who discover how to use it to enhance their abilities and bridge gaps in knowledge are successful. Those who use it to bypass bureaucratic barriers and speed up the overall work process are successful. In a big way.
What are the right ways to use AI? We don’t know. Each professional has to figure that out for themselves, and then update their assessments every year, as AI itself improves all the time. All we can say is that we are in a time of great uncertainty, but the most adaptable people – those who try, play, and dare to reinvent their jobs – are still winning.
More of the topic in Hayadan:
