Why Software Automation Is Hard

Originally intended as a quick take, but got a bit longer, so why not turn it into a post. Just sharing my observations & assumptions here about the state of software automation. Happy to hear thoughts on where you think I'm off. I'm sure none of the thoughts in this post are totally original, many have been proposed in similar form elsewhere, and I'm[1] far from the first person to speak of the bottlenecks that AI progress and adoption are facing. It still seemed useful to compile my current views on the situation and summarize them to those with only an outside view on the impact of AI on the software industry.The software world is trying hard to automate itself. Undoubtedly, coding agents have made a step change since last November and now enable more and more use cases that were unthinkable a year ago. And yet it seems to me that there's still a big disconnect between how many people think coding agents should be affecting the software industry and what's really going on so far in most places.Please feel encouraged share your views and disagreements about any of these in the comments.First, I do think the following things are all true:coding agents have become much more capable recently and have unlocked many, many new use casescoding agents are better than almost all humans at a huge amount of coding-related activities and allow practically any individual to do work far beyond the scope of what they were able to do 1-2 years agocoding agents will become better and better and almost every "intuitive" attempt to predict what they'll be like in a few years will likely underestimate them in all kinds of waysthe space of things individuals can create has expanded immensely; non-technical people are now able to build pretty useful and sometimes impressive things[2]But it's easy to extrapolate this too far, assuming that much of software engineering can now be automated and that the same number of engineers can now get done 10x as much as before. My impression is that