Stop Listening to Technology Executives' Opinions on AI

I admit it. I took the bait. Charity Majors made an appearance on Gergely Orosz’s show called “Stop being skeptical about AI for development”. I listened to the podcast—I don’t listen to many tech podcasts—and I don’t think she ever said to stop being skeptical. Not directly anyways. Great title, though. Good for engagement. “Stop being skeptical” is a phrase guaranteed to rustle any skeptic’s jimmies. A whole lotta provocation, not a lotta substance. 10/10. No notes.

Beyond the clickbait title, the point is less “stop arguing and give into the vibes” and more “accept that AI for development is here to stay and figure out what we need to do about it.” And I’ll commend Majors for this: that’s such a more measured take than so many are making in this space. But the choice to frame opposition to AI for development as skepticism really got me thinking. Why is skepticism the word we use here? It’s not just Majors that uses this word.

My first thought was that folks more open to AI are just using skepticism as a shorthand for the reflexive antagonism they see in the scattered and fragmented opposition to AI within tech circles, and across all of society. This semantic drift is completely understandable, especially since many AI boosters remember having their own doubts about Generative AI. AI boosters having been making crazy claims for a long time—going back well before this current craze—but these extraordinary claims have never before been backed by the extraordinary evidence needed to substantiate them. Majors herself admits that it was hard to imagine non-slop code being generated by chatbots as early as a year ago. But look how things have changed! The term “skeptic” makes sense for how someone like this might recount the transformation of their own beliefs: I once was skeptical, then I saw evidence, and now I believe.

In this sense, yeah, I believe too now. I believe you can generate lots of working code with AI. But I’m still skeptical that this is a good way to build software at scale, and I’m deeply skeptical about the environmental, economic, psychological, and sociological impacts of this technology on the world at large. The fact that AI is pretty good at generating working code from prose descriptions of software problems hasn’t changed my broader skepticism about the technology and its myriad issues.

I need to be clear: chatbots getting “pretty good” at generating working code from text is amazing. It’s a genuinely remarkable achievement that I didn’t see coming in my life. I’ve worked with career developers who fail to attain a rating of “pretty good” at generating working code from prose descriptions of problems.¹ So many discussions between people with different attitudes surrounding AI for coding fall apart here because AI detractors do not admit this is a remarkable achievement. Let me be clear: it’s a goddamn miracle this works!

But this miracle is where I think AI supporters fall off the deep end, and abandon much of their skepticism without evidence. The sense of wonder you get from building like this for the first time is, like, incredible. It’s exhilarating. It genuinely feels like you’re experiencing magic. It’s a technological marvel. It has the leap-forward quality to it that activates an emotion most folks in our times seldom ever feel: awe. Vibe-coding, agentic coding, genie-assisted coding—these can be awe-inspiring activities, at least at first.

Awe is a sort of joyous fear, a mix of reverence and wonder. It’s not something we associate with the cold rationality we like to imagine our profession embodies. Awe is what old-timey folks felt towards God or gods or spirits or demons. It’s definitely not something that rational modern techies feel towards algorithms, right? But an awestruck person, inspired by the awe they feel, will start jumping to what’s possible, and extrapolating from the current trends. They see how AI is changing everything. For within awe are contained the seeds of faith. It’s that faith that allows them to see how we get from the miracle of AI code generation to solving all the other problems the technology induces.

I know that many people are going to take umbrage at this religious framing of AI partisanship. I don’t mean it offensively, and I don’t think this religious character is what makes its beliefs untrue. But it explains so much of the behavior around the debates in AI. AI supporters are filled with faith in the technology’s ability to grow and change and transform things, and this faith is bolstered by a massive and vocal community of folks who feel the same way, who gleefully use the tools, who speak the same language about it, and who are excited to see where it’s all going.

AI detractors, on the other hand, are skeptics who deny the miracles or who witness them and somehow remain faithless. I’m happy to embrace this notion of skeptic, because it’s describes exactly what I feel is happening. I’m amazed at what I see but it’s not enough to make me believe the prophesies I hear. I still see so many problems with how this all plays out. Even if the costs and externalities get worked out (and that’s a big if), I don’t see how this makes software development better on balance. Even if it enables us to produce more software faster, its effects on the quality and utility of what we produce are not yet clear. To me, the real top-line limiting factor for software delivery is our ability to collectively understand what we’re deploying, not the amount of code we can produce in a day.

Some engineers that want to use AI more aggressively disagree about this limit. Maybe if it becomes a problem, we just ask the AI to fix it! And if that doesn’t work, we ask it explain the code to us. Other engineers agree with this understanding gap being an issue, but argue that we can solve it by working at the level of specifications, each of us acting like mini-architects and letting the LLM agents write the code for us. This would ostensibly let us operate at a higher level of abstraction and thus understand more total software in less time. Still others think this is all irrelevant because we’re approaching the singularity and we’ll all be out of a job or dead anyways.

To my mind, each of these beliefs hand-waves over a lot of relevant details in assuming that we (or the machines) will find some way to make it all work out. This is the part I remain skeptical about. I want to see significant evidence of these claims being true, in proportion to how extraordinary they are. I want to see evidence come out of relatively controlled and minimally biased settings, which is almost impossible in the current environment. There’s way too much incentives to tilt the scales towards favored outcomes for me to trust results coming out of studies or OKR reports.

AI boosters are not as troubled with such concerns. They have seen the light. I think they are confident that this will all work out, because it always does, or because lots of smart people are working on it, or because they are more willing to trust the muddy evidence that’s percolating up in the wild. To me, this seems like faith. You take it as a given that AI is inevitable, in some sense, and you let your mind fill out the details as you will (or hand-wave them). Again this isn’t necessarily bad—you need faith to try virtually anything new—but it’s a faith that’s attached to a weird and dogmatic movement that I’m disinclined to trust until the dust settles.

Still don’t believe me? A better writer than me with much more access to business decision makers has written an excellent essay summarizing how AI-mania has become a virulent dogma infecting business culture worldwide. Go read it now if you haven’t. I will wait here.

Before turning back to the podcast that inspired this essay, I’ll quote Suresh here so I can use his ideas to force an artless transition to my next point:

…continued advancement, and increasingly continued employment, has started to require repeated professions of belief in the transformative power of AI for said business…There have been several occasions where I have seen someone, apropos of nothing, blurt out almost word-for-word “AI is changing everything.”

It’s no coincidence that the podcast starts out with Orosz literally saying “As you know, AI is changing everything.” It’s a liturgical framing for what is to come. As technical folks, Orosz and Majors know a lot more about the realistic applications of LLM-assisted coding than your average business toady spouting a rote-memorized litany in the absence of any deeper knowledge on the topic. But in this instance it serves more as a reminder to anyone listening that, despite criticism directed as some AI-maximalist beliefs, this is an AI-friendly podcast, and we all agree on the core dogma.

Majors and Orosz are among a cohort of technically-minded AI boosters who are attempting to carve out a stable place for the continued existence of software engineering within the AI-powered future. It’s like the cult of a friendlier, more humanistic god within the same AI pantheon.² And they absolutely need to signal alignment with the broader movement because much of what they say contradicts the publicly proclaimed beliefs of powerful figures in their orbits. That alone earns some respect. You can’t say AI isn’t really good for writing without pissing off a powerful executive that’s been using it to compose all their emails for the last two years.

So I find myself agreeing with a huge proportion of what they have to say about the state of AI and the software industry. DevOps is dead. Working with mystery code is already the reality for ops teams. Looking down our noses at QA prevents us from learning from their techniques. AI is still slop for most things. It works for code, but its stochastic nature is hard to integrate into otherwise deterministic systems. All good points I largely agree with.

But Majors seems to believe that these are all problems that will get solved; that AI demands more engineering rigor, not less; and that we can build properly guardrailed and instrumented autonomous coding systems that can usher in an age of software where no one reads the code, at least not all of it. Code review is overrated, she opines. It means too many things to too many teams and rarely functions as an effective production gate. All that effort would be better spent building tests and observability.

I’m almost inclined to agree. Code review is an overloaded bureaucratic process that was a bottleneck in software development long before a GPU ever started cranking out code for us. We use it in ill-defined ways to handle too many concerns: to train juniors, to enforce code consistency, to debate the merits of a given feature, to scrutinize implementation details, to document process for auditors, and to encourage a more evenly-shared understanding of a collectively maintained codebase. That’s a lot to ask of a process we ask senior developers to squeeze into their already busy days.

But all those things are important! Saying we should just focus on specs, tests, and observability doesn’t obviate the need to address those concerns. It just pushes them out into other parts of the development process, in ways that arguably hamper our ability to tackle those issues directly. I intend to write a lot more about the idea of “spec-driven development”, as there is just so much to say about this topic, but for the sake of brevity I’ll condense that argument to a single question for you to ponder: what if writing a sufficiently detailed specification of a program is as hard as (or harder than) writing the code itself?

Many would balk at that question. “Of course it’s not!” they think. “When I say specification, I mean something more high-level.” Okay, fair enough, but then you’re not filling in the thousands or relevant lower-level details that ultimately will define the observable behavior of your system. That kind of “specification” is so high-level that some engineer or chatbot will have to make a bunch of assumptions (or engage in a lot of back-and-forth) about what you actually want. And that more detailed spec is what will have to be captured as the new version of code, that acts as the ever-updating source of truth about what our system is. Now developing and reviewing this is your engineering bottleneck. Software developers—not a group known for their universal mastery of clear and unambiguous communication—will be responsible for maintaining a giant natural language description of the systems they work with.³ Of course, they’d probably be tempted to outsource this bit to chatbots too.

But beware! If you leave your specifications too high-level, or you outsource the development and review of these specs entirely to bots, then the non-deterministic nature of models is likely liable to interpret this differently over time, and induce real changes in the observable behavior of your system that you didn’t intend. Perhaps this could be corralled by testing, but now we’d need engineers to be good at writing tests that properly constrain the behavior of system, and for those tests to run fast enough to enable an autonomous coding agent to use them in a development loop. And these tests would become part of the corpus we need to review instead of the code, since it would be these tests that are the real constraints on the code behavior.

Overall the idea behind this kind of “actually forget code review” development seems to be to push developers away from something they are, as a group, at least kinda good at (writing code) towards something they are generally bad at (writing prose and tests). And then we’d use that prose and those tests as the living record of our shared understanding of the system, while the agents do the fiddly busy work of writing software code. I don’t get it. I don’t understand what work we’re saving here. Writing specs and tests is hard work. And even if some devs decided that’s where they wanted to take this—that we should all get good at specs and tests so we can produce super high quality software—I really can’t see any business folks doing that when the industry Zeitgeist is all about doing more faster. Do we really see folks being given the time to really think through their testing strategy to make sure the agents are kept in line? Or do we see them being asked to vibe code the guardrails too?

I might find this “We’ll engineer the guardrails!” take laughably optimistic, but to her credit, at least Majors is suggesting that people could do this, and not just that LLMs will become so magically good that they will be able to do this themselves. It’s why I consider her to be part of the humanistic wing of the AI movement, along with many software developers. It’s a belief that humans, with the right structure and incentives, will find a way to harness this technology and put it to good use. There’s a refreshing sense of hope here: the extra demands that AI use will place on the organization could push us to actually talk to other teams and find the common ground we can use as a foundation on which we build our new, more rigorous engineering process.

Unlike the more radical formulations of the AI faith, we at least have evidence that humans have worked together to rein in complex technology and apply it rigorously to solving tough problems. It is, however, quite rare. The median software shop is a place of woeful inefficiency and muddled communication where technological resources are poorly deployed, human actors are saddled with pointless make-work, and progress is forever stymied by petty political gamesmanship. Upper management is seldom pleased with this state of affairs, where a very expensive department produces so little of obvious value. But for lack of any real ideas how to fix it, they usually just fall back to the tried-and-true method of demanding that we Do Better and applying pressure.

This pressure almost inevitably fractures a department along any pre-existing fissures that were forming: dev vs ops, front-end vs back-end, engineering vs research, or product vs platform. And once these battle lines are drawn, once these fiefdoms are carved out, the chances of fixing the core issues go way down. Much of my career has been as an individual contributor in such places. And I can tell you, just talking to the other teams doesn’t really work. It’s typically viewed as crossing enemy lines. You can’t negotiate peace from the trenches.

This is what makes Major’s two-camps arguments fall so flat. The divide is a structural arrangement that is seldom within the power of individual contributors or isolated managers to solve. And with one side clearly backed by capital, it’s not really a fair fight. Overburdened production and security teams, now pressured to allow through a flood of AI-related and AI-built features, revert to bulldog mode, and in so doing set themselves up to take heat for being reflexively opposed to innovation. This isn’t a new organizational dynamic. Front-line teams with real responsibilities get put in a corner whenever business goals tilt towards quantity over quality. Is it any wonder they get defensive in these cases?

There’s no digital Kumbaya to be sung here. Fixing these kinds of dynamics requires real executive commitment—one that lasts longer than the couple quarters most executives have to make their marks these days—and well-aligned follow-through at the managerial and line levels. Teams talking to each other certainly is part of this, but that can’t happen until teams are allowed to cooperate and the incentives that foster fiefdoms are weeded out. As it stands, this argument just sounds like a narrative a CTO could use to blame their underlings and justify the next re-org: “we knew we needed more discipline to make AI work at scale, but our teams just could not communicate across silos about priorities.”

The more you stare at it, the more this seemingly humanistic, we-can-do-it sort of attitude towards aggressive adoption of AI for coding seems like a sham. At best it’s naively optimistic. The investment world sees the current moment as inflection point where software companies need to reignite sustained hypergrowth with “AI-native products” or leverage AI to increase their margins by dumping employees. Seriously. There’s no time in these folks’ minds for increased engineering rigor, only the sweet, sweet smell of burning capital as asset valuations rocket into the stratosphere.

You might make the argument that without the engineering rigor necessary to handle the use of AI at scale, these dynamic investment vehicles companies will eventually fail under the massive weight of accumulated tech debt. And you might be right. But then why are we arguing that we need to shift so quickly? If most companies are just going to blindly adopt AI without the proper engineering rigor needed to use it properly, then what is our rush? Everybody is rushing now. Where’s the competitive advantage in rushing? Why don’t we carefully and methodically find our way to use these tools, wait until the competition flames out, and swoop in and clean up after the mess?

It’s this notion that you gotta start doing AI right now that really sours any sympathy I might have had for Majors’ arguments. This isn’t a more friendly, humanistic, pro-engineering take on AI. It’s candy-coated version of the same technocapital pill, and they’re all saying you should take as many as you can every day. It’s just fearmongering. “Learn AI or get left behind.” Really? What is there to learn? I was late to the party and I figured out how to get productive with AI in like 2 weeks. It’s a chatbot connected to a codebase. You write some Markdown files (“skills”) to help constrain its behavior. You iterate and adapt.⁴

Now there is a pervasive sense in the industry that it’s all moving so fast that if you don’t keep up you’ll be hopelessly lost. All the models and techniques and agents and skills change constantly. But this isn’t new. The same shit has been happening in the world of web frameworks for most of my career. “Serious” backend developers like me used to make fun of it. We have a word for the concept: churn. It has the appearance of work but doesn’t really accomplish much. If you have solid programming fundamentals, all this ecosystem churn will be really manageable. Majors even suggests that her generation of programmers are better debuggers, and how that’s a really valuable skill. Guess what? It’s still valuable! You just get an LLM search tool now that can help isolate some cases. You still get to apply your troubleshooting skills when the LLM inevitably fails to make sense of some problem outside its training data or context window.

As long as we’re not demanding this total transformation of the software industry into one where humans do nothing more than build requirements docs and guardrails for agents—something I still do not believe is tenable—I think adopting LLM technology into coding is an extremely manageable transition to make for any decent engineer. It’s on par with learning a new language. Sure, there might be some bumps, but you can probably be productive with really quickly. And if your company demands you go really all-in on AI, where you’re basically corralling a chatbot, then it might be time to start looking for a new job. You probably want to avoid this kind of work, but more importantly, I don’t think it bodes well for your company.

AI coding amplifies organizational strengths and weaknesses, and far too many shops have more unexamined weaknesses than they are willing to admit. Rushing headlong into autocoding will only exacerbate existing problems. If, as Majors suggests, AI demands greater engineering rigor, then what makes you think we’ll install that rigor after we start mainlining slopcode through the heart of our engineering org? Rigor is not a plug-in or a bolt-on. It’s part of software engineering culture, and you need to work to make sure it’s there before you start opening the flood gates. If you use AI in a limited capacity to help you instrument those processes, great! But that’s not what is being encouraged in the current environment. Your investors want you to build features faster than your competition so you can grow like cancer that consumes your market; or to cut staff so much and use AI to wring out every ounce of work from a shell team so you can have crazy margins.

The answer to actually improving these things is, as it has always been, to methodically and ruthlessly self-evaluate. Maybe it takes some fresh eyes, but it’s not usually too hard to find the misaligned incentives and managerial dross that makes organizations fail with such predictable regularity. For software, any goal that prioritizes something other than the consistent delivery of reliable features to production will inevitably lead to decay. As simple as that is, it’s hard to make it stick, because those goals constrain the ability of management and other political operators within the organization to declare wins on demand—which is the point, really. A genuine focus on outcomes means you have to actually, you know, produce outcomes. With the right set of measures in place, this can be hard to fake.

But that doesn’t mean folks won’t try. The effort people will put in to faking or changing these measures is a function of the pressures applied to the organization. In something like the current AI mania, where everyone is under the gun to deliver on the technology’s purported potential, that pressure is forcing even previously grounded organizations to move the goalposts forward to ensure AI projects get over the line. Even if AI for coding demands more engineering rigor, such rigor will not be applied by the vast majority of organizations. It would impede someone important’s ability to claim an AI-related win.

Majors is only able to push an aggressive AI-is-for-engineers agenda by pretending these organizational dynamics and incentive structures don’t exist or don’t matter. On the surface her arguments look like a more humane, developer-friendly way to sell the AI narrative, but on closer examination they fall apart. Do you expect me to believe this iteration of AI technology will launch this industry into a new golden era of reliable, performant, and useful software? We only need engineers to become really good at quality assurance skills that they’ve never shown much interest in or aptitude for. We only need to close a series of organizational gaps that we’ve been trying and failing to bridge for 20 years. We only need senior managers to consistently incentivize excellence in engineering and give teams the tools, trust, and time they need to reach it. Come on. That’s almost as much a leap of faith as believing that AGI is just around the corner.

I wish we’d all recognize this form of wishful thinking as what it is: faith. We start from the realization that AI can make parts of software development more efficient, and jump to wild conclusions that everyone should use it for everything all the time as much as possible, and that doing so will fix all our problems. This attitude towards tools isn’t new. It’s been with us for years, and we used to literally call them Holy Wars, because we recognized the way that devotion to our favored tools could turn us into religious fanatics. Parodying these views reminded us to not take it so seriously, and to develop a more nuanced, detached relationship with our technology choices. This also allowed us to let other people arrive at different choices on the matter, and suspend judgement until all the facts are in.

Plenty of developers—myself included—are finding that AI helps them get things done, and if they can approach the subject in a grounded, measured way, I think it’s perfectly reasonable to continue trying it for new things. I have been pleasantly surprised with its performance at certain tasks. But it’s still a very new technology, and we cannot isolate its effects from those of the broader technocapital doomsday cult surrounding it.

So yeah, I’m still skeptical of AI for coding. You should be too. There are so many aspects to AI mania that warrant a critical perspective: the broad, unfocused sense of urgency in the pervasive pressure to adopt; the constant handwaving dismissal of legitimate concerns about its effects, economics, and externalities; and the way it’s touted as a silver bullet so such a wide array of problems. That does not mean you shouldn’t experiment with it. Curiosity and skepticism are not mutually exclusive. On the contrary, they go rather well together. But we should make our engineering decisions based on what we think will work well in the real world, not what we hope technology will do for us eventually. This shouldn’t be controversial to engineers. Ours is a discipline that demands judgement, not faith.