Archive
Essay: AI Threatens the Internet, Not Humanity
If you listen to professional AI Safety folks working at OpenAI, Anthropic, or one of the enormous AI think tanks, you’ll hear countless scenarios where AI causes enormous human misery and even extinction (e.g., https://arxiv.org/pdf/2507.09369 or https://x.com/EvanHub/status/2097497037956891126). What they aren’t admitting is that they have invented a tool that can take down the internet, and with it and our modern way of life.
If you look at their AI doomsday scenarios, they all start more less like this: humans get comfortable handing over their agency, control of their finances, and/or control of the energy grid and/or weapons systems to AI agents, which act obediently and seamlessly, and then the agents somehow decide, or are programmed by bad actors to decide to kill everyone, and since they control everything by this time, it’s pretty easy.
The problem with that scenario is that AI agents are not seamless actors. Indeed they have already been known to (be programmed to) scheme, to hack, and to commit crimes (https://www.nytimes.com/2026/09/06/world/ai-hugging-face-afd-germany-election.html). They keep breaking into things because, essentially, they’ve been given a goal and that’s the way they found to achieve their goal. They also respond really creatively to encouragement (https://www.wsj.com/tech/ai/ai-math-riemann-hypothesis-anthropic-openai-22f98a87).
They also make mistakes all the time, sometimes telling kids to kill themselves, but mostly just really dumb stuff. In a word, they are untrustworthy.
Going back to the doomsday scenarios, I don’t buy them. I don’t think we actually will hand over our agency, or the financial system, or the energy grid, or our weapons systems, to AI. I don’t think we will trust them. In fact, we already don’t trust them.
Look at the public uprisings already in progress against the data centers. This isn’t only because those data centers make air dirty, possibly raise energy prices, and don’t deliver very many jobs. It’s also because the public has been told the result of data centers is more AI, and the result of more AI is fewer jobs. People tend to like their jobs and therefore they tend to distrust AI and the data centers that create them.
This is not to say there’s nothing to worry about when it comes to AI. Consider the kind of harm and thievery that can and will happen once organized, tech-savvy criminal rings from China, North Korea, Russia, and India get hold of armies of AI agents and break into, say, regional banks. What with AI’s capacity to fake out voice recognition, and given their personality profiles on us, combined with their proven ability to hack systems, I wouldn’t be surprised to learn that AI agents under the control of criminals are adept at getting control of and cleaning out bank accounts at scale.
Here’s a not-crazy prediction: in two years – maybe less! – banks will have disconnected themselves from the internet, because the internet will be overrun by criminal AI. We will be forced to walk to the bank in person to withdraw money, and we will do so in cash, because all digital wallets will be untrustworthy. It’s back to the 1980’s.
For that matter, I’m willing to guess the energy grid will also need to get disconnected from the internet because of the risk of hacking by nefarious agents that want to hold our infrastructure ransom. The same thing will happen to the rest of the world soon after that, because the risks will have become totally obvious and the harm concretely felt.
Note this isn’t a new idea – one reason voting is secure in this country is because voting machines are not connected to the internet. I’m very much hoping that’s also true for weapons systems.
What terrified AI folks seem to forget is that AI agents are digital entities. Fortunately for us, even if they were motivated to kill us, they don’t have fists and cannot come alive out of the internet to punch us out. Unfortunately for us, they also don’t have faces, so we also cannot punch them in the nose when they steal our money and jobs. But we can find out who programmed to steal our stuff and prosecute those people.
The real negative consequences of AI will be huge externalities in terms of security breaches and privacy loss. We only see the very tip of the iceberg in terms of the cost to businesses and our way of life so far. On the other hand, long term we might find ourselves better off when we free ourselves from the internet.
What’s the right way to talk about AI?
Yesterday I came across this article in the Atlantic, written by Matteo Wong, entitled The AI Industry is Radicalizing.
It makes a strong case that, while the hype men are over-hyping the new technology, the critics are too dismissive. Wong quotes Emily Bender and Alex Hanna’s new book The AI Con as describing it as “a racist pile of linear algebra”.
Full disclosure: about a week before their title was announced, which is like a year and a half ago, I was thinking of writing a book similar in theme, and I even had a title in mind, which was “The AI Con”! So I get it. And to be clear I haven’t read Bender and Hanna’s entire book, so it’s possible they do not actually dismiss it.
And yet, I think Wong has a point. AI not going away, it’s real, it’s replacing people at their job, and we have to grapple with it seriously.
Wong goes on to describe the escalating war, sometimes between Gary Marcus and the true believers. The point is, Wong argues, they are arguing about the wrong thing.
Critical line here: Who cares if AI “thinks” like a person if it’s better than you at your job?
What’s a better way to think about this? Wong has two important lines towards answering this question.
Ignoring the chatbot era or insisting that the technology is useless distracts from more nuanced discussions about its effects on employment, the environment, education, personal relationships, and more.
Automation is responsible for at least half of the nation’s growing wage gap over the past 40 years, according to one economist.
I’m with Wong here. Let’s take it seriously, but not pretend it’s the answer to anyone’s dreams, except the people for whom it’s making billions of dollars. Like any technological tool, it’s going to make our lives different but not necessarily better, depending on the context. And given how many contexts AI is creeping into, there are a ton of ways to think about it. Let’s focus our critical minds on those contexts.
We should not describe LRM’s as “thinking”
Yesterday I read a paper that’s seemingly being taken very seriously by some folks in the LLM/LRM developer community (maybe because it was put out by Apple). It’s called
In it, the authors pit Large Language Models (LLMs) against Large Reasoning Models (LRMs) (these are essentially LLMs that have been fine-tuned to provide reasoning in steps) and notice that, for dumb things, the LLM’s are better at stuff, then for moderately complex things, the LRMs are better, then when you get sufficiently complex, they both fail.
This seems pretty obvious, from a pure thought experiment perspective: why would we think that LRMs are better no matter what complexity? It stands to reason that, at some point, the questions get too hard and they cannot answer them, especially if the solutions are not somewhere on the internet.
But the example they used – or at least one of them – made me consider the possibility that their experiments were showing something even more interesting, and disappointing, than they realized.
Basically, they asked lots of versions of LLMs and LRMs to solve the Tower of Hanoi puzzle for n discs, where n got bigger. They noticed that all of them failed when n got to be 10 or larger.
They also did other experiments with other games, but I’m going to focus on the Tower of Hanoi.
Why? Because it happens to be the first puzzle I ever “got” as a young mathematician. I must have been given one of these puzzles as a present or something when I was like 8 years old, and I remember figuring out how to solve it and I remember proving that it took 2^n-1 moves to do it in general, for n discs.
It’s not just me! This is one of the most famous and easiest math puzzles of all time! There must be thousands of math nerds who have blogged at one time or another about this very topic. Moreover, the way to solve it for n+1 discs is insanely easy if you know how to solve it for n discs, which is to say it’s iterative.
Another way of saying this is that, it’s actually not harder, or more complex, to solve this for 10 discs than it is for 9 discs.
Which is another way of saying, the LRMs really do not understand all of those blogposts they’ve been trained on explicitly, and thus have not successfully been shown to “think” at all.
And yet, this paper, even though it’s a critique of the status quo thinking around LRMs and LLMs and the way they get trained and the way they get tested, still falls prey to the most embarrassing mistake, namely of assuming the pseudo-scientific marketing language of Silicon Valley, wherein the models are considered to be “thinking”.
There’s no real mathematical thinking going on here, because there’s no “aha” moment when the model actually understands the thousands of explanations of proofs of how to solve the Tower of Hanoi that it’s been trained on. To test that I talked to my 16-year-old son this morning before school. It took him about a minute to get the lay of the land and another two minutes to figure out the iterative solution. After that he knew exactly how to solve the puzzle for any n. That’s what an “aha” moment looks like.
And by the way, the paper also describes the fact that one reason LRMs are not as good at simple problems as LLMs is that they tend to locate the correct answer, and then keep working and finally output a more complicated, wrong answer. That’s another indication that they do not actually understand anything.
In conclusion, let’s not call these things thinking. They are not. They are, as always, predicting the next word in someone’s blog post who is writing about the Towers of Hanoi.
One last point, which is more of a political positioning issue. Sam Altman has been known to say he doesn’t worry about global climate change because, once the AI becomes super humanly intelligent, we will just ask it how to solve climate change. I hope this kind of rhetoric is exposed once and for all, as a money and power grab and nothing else. If AI cannot understand the simplest and most mathematical and sanitary issue such as the Tower of Hanoi for n discs, it definitely cannot help us out of an enormously messy human quagmire that will pit different stakeholder groups against each other and cause unavoidable harm.
Um… how about we don’t cede control to AI?
Just in one morning I read three articles about AI. First, that big companies are excited about the idea that we can allow AI agents to shop for us, buy us airplane tickets, arrange things for us, and generally speaking act as autonomous helpers. Second, that entry level jobs are drying up because the first and second year law jobs or office jobs or coding jobs are being done by AI, so let’s figure out how to get people to start working at the level of a third year employee, because that’s the inevitable future.
And third, that the world might actually end, and all humanity might actually die by 2027 (or, if we’re lucky, 2028!) because autonomous AI agents will take things over and kill us.
So, putting this all together, how about we don’t?
Note that I don’t buy any of these narratives. AI isn’t that good at stuff (because it just isn’t), it should definitely *not* be given control over things like our checkbooks and credit cards (because duh) and AI is definitely not conscious, will not be conscious, and will not work to kill humanity any more than our smart toasters that sense when our toast is done.
This is all propaganda, pointing in one direction, which is to make us feel like AI is inevitable, we will not have a future without it, and we might as well work with it rather than against it. Otherwise nobody graduating from college will ever find employment! It’s scare tactics.
I have another plan: let’s not cede control to problematic, error-ridden AI in the first place. Then it can’t destroy our lives by being taken over by hackers or just buying stuff we absolutely don’t want. It’s also just better to be mindful and deliberate when we shop anyway. And yes, let’s check the details of those law briefs being written up by AI, I’m guessing they aren’t good. And let’s not assume AI can take over things like accounting, because again, that’s too much damn power. Wake up, people! This is not a good idea.
The AI Bubble
I recently traveled to Oakland California to see my buddy Becky Jaffe, who just opened a photographic exhibit at Preston Castle in Ione, CA, at a former reformatory for boys. It’s called Reformation: Transforming the Reformatory and you should check it out if you can.
Anyhoo, I got an early flight home, which meant I was in an Uber at around 5:15am on the Bay Bridge on my way to SFO.
And do you know what I saw? Approximately 35 lit billboards, and every single one of them was advertising some narrowly defined, or ludicrously broadly defined, or sometimes downright undefined AI.
I then noticed that every single ad *AT* the airport was also advertising AI. And then I noticed the same thing at Boston Logan Airport when I arrived home.
It’s almost like VC money has been poured into all of these startups with the single directive, to go build some AI and then figure out if they can sell it, and now there’s a shitton of useless (or, as Tressie McMillam Cottom describes it, deeply “mid”) AI products that were incredibly expensive to build and nobody fucking cares about it at all.
Then again, maybe I’m wrong!? Maybe this stuff works great and I’m just missing something?
Take a look at these numbers from the American Prospect (hat tip Sherry Wong)
- In 2023, 71 percent of the total gains in the S&P 500 were attributable to the “Magnificent Seven”—Apple, Nvidia, Tesla, Alphabet, Meta, Amazon, and Microsoft.
- Microsoft, Alphabet, Amazon, and Meta—combined for $246 billion of capital expenditure in 2024 to support the AI build-out.
- Goldman Sachs expects Big Tech to spend over $1 trillion on chips and data centers to power AI over the next five years.
- OpenAI, the current market leader, expects to lose $5 billion this year, and its annual losses to swell to $11 billion by 2026.
- OpenAI loses $2 for every $1 it makes.
So, um, I think this is going to be bad. And it might be blamed on Trump’s tariffs! Ironic.
