Archive
What’s the right way to talk about AI?
Yesterday I came across this article in the Atlantic, written by Matteo Wong, entitled The AI Industry is Radicalizing.
It makes a strong case that, while the hype men are over-hyping the new technology, the critics are too dismissive. Wong quotes Emily Bender and Alex Hanna’s new book The AI Con as describing it as “a racist pile of linear algebra”.
Full disclosure: about a week before their title was announced, which is like a year and a half ago, I was thinking of writing a book similar in theme, and I even had a title in mind, which was “The AI Con”! So I get it. And to be clear I haven’t read Bender and Hanna’s entire book, so it’s possible they do not actually dismiss it.
And yet, I think Wong has a point. AI not going away, it’s real, it’s replacing people at their job, and we have to grapple with it seriously.
Wong goes on to describe the escalating war, sometimes between Gary Marcus and the true believers. The point is, Wong argues, they are arguing about the wrong thing.
Critical line here: Who cares if AI “thinks” like a person if it’s better than you at your job?
What’s a better way to think about this? Wong has two important lines towards answering this question.
Ignoring the chatbot era or insisting that the technology is useless distracts from more nuanced discussions about its effects on employment, the environment, education, personal relationships, and more.
Automation is responsible for at least half of the nation’s growing wage gap over the past 40 years, according to one economist.
I’m with Wong here. Let’s take it seriously, but not pretend it’s the answer to anyone’s dreams, except the people for whom it’s making billions of dollars. Like any technological tool, it’s going to make our lives different but not necessarily better, depending on the context. And given how many contexts AI is creeping into, there are a ton of ways to think about it. Let’s focus our critical minds on those contexts.
We should not describe LRM’s as “thinking”
Yesterday I read a paper that’s seemingly being taken very seriously by some folks in the LLM/LRM developer community (maybe because it was put out by Apple). It’s called
In it, the authors pit Large Language Models (LLMs) against Large Reasoning Models (LRMs) (these are essentially LLMs that have been fine-tuned to provide reasoning in steps) and notice that, for dumb things, the LLM’s are better at stuff, then for moderately complex things, the LRMs are better, then when you get sufficiently complex, they both fail.
This seems pretty obvious, from a pure thought experiment perspective: why would we think that LRMs are better no matter what complexity? It stands to reason that, at some point, the questions get too hard and they cannot answer them, especially if the solutions are not somewhere on the internet.
But the example they used – or at least one of them – made me consider the possibility that their experiments were showing something even more interesting, and disappointing, than they realized.
Basically, they asked lots of versions of LLMs and LRMs to solve the Tower of Hanoi puzzle for n discs, where n got bigger. They noticed that all of them failed when n got to be 10 or larger.
They also did other experiments with other games, but I’m going to focus on the Tower of Hanoi.
Why? Because it happens to be the first puzzle I ever “got” as a young mathematician. I must have been given one of these puzzles as a present or something when I was like 8 years old, and I remember figuring out how to solve it and I remember proving that it took 2^n-1 moves to do it in general, for n discs.
It’s not just me! This is one of the most famous and easiest math puzzles of all time! There must be thousands of math nerds who have blogged at one time or another about this very topic. Moreover, the way to solve it for n+1 discs is insanely easy if you know how to solve it for n discs, which is to say it’s iterative.
Another way of saying this is that, it’s actually not harder, or more complex, to solve this for 10 discs than it is for 9 discs.
Which is another way of saying, the LRMs really do not understand all of those blogposts they’ve been trained on explicitly, and thus have not successfully been shown to “think” at all.
And yet, this paper, even though it’s a critique of the status quo thinking around LRMs and LLMs and the way they get trained and the way they get tested, still falls prey to the most embarrassing mistake, namely of assuming the pseudo-scientific marketing language of Silicon Valley, wherein the models are considered to be “thinking”.
There’s no real mathematical thinking going on here, because there’s no “aha” moment when the model actually understands the thousands of explanations of proofs of how to solve the Tower of Hanoi that it’s been trained on. To test that I talked to my 16-year-old son this morning before school. It took him about a minute to get the lay of the land and another two minutes to figure out the iterative solution. After that he knew exactly how to solve the puzzle for any n. That’s what an “aha” moment looks like.
And by the way, the paper also describes the fact that one reason LRMs are not as good at simple problems as LLMs is that they tend to locate the correct answer, and then keep working and finally output a more complicated, wrong answer. That’s another indication that they do not actually understand anything.
In conclusion, let’s not call these things thinking. They are not. They are, as always, predicting the next word in someone’s blog post who is writing about the Towers of Hanoi.
One last point, which is more of a political positioning issue. Sam Altman has been known to say he doesn’t worry about global climate change because, once the AI becomes super humanly intelligent, we will just ask it how to solve climate change. I hope this kind of rhetoric is exposed once and for all, as a money and power grab and nothing else. If AI cannot understand the simplest and most mathematical and sanitary issue such as the Tower of Hanoi for n discs, it definitely cannot help us out of an enormously messy human quagmire that will pit different stakeholder groups against each other and cause unavoidable harm.
Um… how about we don’t cede control to AI?
Just in one morning I read three articles about AI. First, that big companies are excited about the idea that we can allow AI agents to shop for us, buy us airplane tickets, arrange things for us, and generally speaking act as autonomous helpers. Second, that entry level jobs are drying up because the first and second year law jobs or office jobs or coding jobs are being done by AI, so let’s figure out how to get people to start working at the level of a third year employee, because that’s the inevitable future.
And third, that the world might actually end, and all humanity might actually die by 2027 (or, if we’re lucky, 2028!) because autonomous AI agents will take things over and kill us.
So, putting this all together, how about we don’t?
Note that I don’t buy any of these narratives. AI isn’t that good at stuff (because it just isn’t), it should definitely *not* be given control over things like our checkbooks and credit cards (because duh) and AI is definitely not conscious, will not be conscious, and will not work to kill humanity any more than our smart toasters that sense when our toast is done.
This is all propaganda, pointing in one direction, which is to make us feel like AI is inevitable, we will not have a future without it, and we might as well work with it rather than against it. Otherwise nobody graduating from college will ever find employment! It’s scare tactics.
I have another plan: let’s not cede control to problematic, error-ridden AI in the first place. Then it can’t destroy our lives by being taken over by hackers or just buying stuff we absolutely don’t want. It’s also just better to be mindful and deliberate when we shop anyway. And yes, let’s check the details of those law briefs being written up by AI, I’m guessing they aren’t good. And let’s not assume AI can take over things like accounting, because again, that’s too much damn power. Wake up, people! This is not a good idea.
Silicon Valley drinks its own Kool aid on AI
There is growing evidence that we are experiencing a huge bubble when it comes to AI. But what’s also weird, bordering on cultish, is how bought in the researchers are in the world of AI.
There’s something called the AI Futures Project. It’s a series of blog posts about trying to predict various aspects of how soon AI is going to be just incredible. For example, here’s a graph of different models for how long it will take until AI can code like a superhuman:
Doesn’t this remind you of the models of COVID deaths that people felt compelled to build and draw? They were all entirely wrong and misleading. I think we did them to have a sense of control in a panicky situation.
Here’s another blogpost of the same project, published earlier this month, this time imagining a hypothetical LLM called OpenBrain, and what it’s doing by the end of this year, 2025:
… OpenBrain’s alignment team26 is careful enough to wonder whether these victories are deep or shallow. Does the fully-trained model have some kind of robust commitment to always being honest? Or will this fall apart in some future situation, e.g. because it’s learned honesty as an instrumental goal instead of a terminal goal? Or has it just learned to be honest about the sorts of things the evaluation process can check? Could it be lying to itself sometimes, as humans do? A conclusive answer to these questions would require mechanistic interpretability—essentially the ability to look at an AI’s internals and read its mind. Alas, interpretability techniques are not yet advanced enough for this.
The wording above makes me roll my eyes, for three reasons.
First, there is no notion of truth in an LLM, it’s just predicting the next word based on patterns in the training data (think: Reddit). So it definitely doesn’t have a sense of honesty or dishonesty. So that’s a nonsensical question, and they should know better. I mean, look at their credentials!
Second, the words they use to talk about how it’s hard to know if it’s lying or telling the truth betray the belief that there is a consciousness in there somehow but we don’t have the technology yet to read its mind: “interpretability techniques are not yet advanced enough for this.” Um, what? Like we should try harder to summon up fake evidence of consciousness (more on that in further posts)?
Thirdly, we have the actual philosophical problem that *we* don’t even know when we are lying, even when we are conscious! I mean, people! Can you even imagine having taken an actual philosophy class? Or were you too busy studying STEM?
To summarize:
Can it be lying to itself? No, because it has no consciousness.
But if it did, then for sure it could be lying to itself or to us, because we could be lying to ourselves or to each other at any moment! Like, right now, when we project consciousness onto the algorithm we just built with Reddit training data!
The AI Bubble
I recently traveled to Oakland California to see my buddy Becky Jaffe, who just opened a photographic exhibit at Preston Castle in Ione, CA, at a former reformatory for boys. It’s called Reformation: Transforming the Reformatory and you should check it out if you can.
Anyhoo, I got an early flight home, which meant I was in an Uber at around 5:15am on the Bay Bridge on my way to SFO.
And do you know what I saw? Approximately 35 lit billboards, and every single one of them was advertising some narrowly defined, or ludicrously broadly defined, or sometimes downright undefined AI.
I then noticed that every single ad *AT* the airport was also advertising AI. And then I noticed the same thing at Boston Logan Airport when I arrived home.
It’s almost like VC money has been poured into all of these startups with the single directive, to go build some AI and then figure out if they can sell it, and now there’s a shitton of useless (or, as Tressie McMillam Cottom describes it, deeply “mid”) AI products that were incredibly expensive to build and nobody fucking cares about it at all.
Then again, maybe I’m wrong!? Maybe this stuff works great and I’m just missing something?
Take a look at these numbers from the American Prospect (hat tip Sherry Wong)
- In 2023, 71 percent of the total gains in the S&P 500 were attributable to the “Magnificent Seven”—Apple, Nvidia, Tesla, Alphabet, Meta, Amazon, and Microsoft.
- Microsoft, Alphabet, Amazon, and Meta—combined for $246 billion of capital expenditure in 2024 to support the AI build-out.
- Goldman Sachs expects Big Tech to spend over $1 trillion on chips and data centers to power AI over the next five years.
- OpenAI, the current market leader, expects to lose $5 billion this year, and its annual losses to swell to $11 billion by 2026.
- OpenAI loses $2 for every $1 it makes.
So, um, I think this is going to be bad. And it might be blamed on Trump’s tariffs! Ironic.
mathbabe 