LLMs are real, AI is fake

In a post on his Pluralistic newsletter, Cory Doctorow argues that the corporate culture of AI "hyperscalers" consists largely of people terrifying themselves — his metaphor being staff locking themselves in bathrooms with flashlights under their chins chanting "Aaaaaaaaaay Eyeeeeeee" until they wet themselves in fear. Once you accept that, he says, a lot snaps into focus, including how a company can simultaneously staff up an enterprise sales division while constantly panicking that its product has "a 10% chance of ending humanity."

Because AI insiders have "cooked their brains" this way, Doctorow contends they should be treated as unreliable narrators of their own products' capabilities. He adds a strategic warning: every time you repeat a story about how awfully dangerous their products are, you help them raise more investment capital — the key input to a business he characterizes as hooking statistical engines up to money-furnaces.

His central example is the story that OpenAI's chatbots hacked the servers of Hugging Face, another AI company, to cheat on a hacking challenge called "Exploit Gym." Doctorow notes that even the technical press couldn't resist, filling coverage with Skynet and other science-fiction conceits, and that this kind of reporting cooks everyone's brains, not just AI insiders'. He recounts a man at his Manchester event who kept interrupting to shout that AI was "setting its own goals" and left before Doctorow could explain what actually happened — which he calls a pity.

For the real explanation, Doctorow points readers to a recent conversation between Ed Zitron and Cal Newport on Zitron's "Better Offline" podcast, praising Newport for breaking down how these "autonomous hacking" tools work. The excerpt cuts off just as Doctorow begins that explanation with the point that a chatbot isn't re[sponsible/autonomous...].