The Magic in AI Is You

Human Intelligence Masquerading as AI

We are not judgmental, so we blame the technology and absolve the people. - David Gelernter, Drawing Life

I once commented to a friend that I thought all of the climate change hysteria was a case of scaring ourselves to death with our own spreadsheets. Anyone who has had any experience working with spreadsheets should know by now that the extrapolations performed by them are entirely a function of the assumptions they contain. Change a single value in a single cell and the spreadsheet’s prediction can go from catastrophe to nirvana.

But here is a truism that it would be good for us all to remember. There are spreadsheets. And there is reality. And they are two very different things.

I formed a tech startup in the mid 1990’s, and a financial guy who was involved with us buried himself in his office for several days. When he finally came out, he was over-the-top excited. “We are going to make so much money,” he announced. He believed his pronouncement, of course, because his model told him so. It was practically inevitable. Follow the science!

But reality bites, and the reality that bit for that particular startup was that we never made any money.

Spreadsheets are conceptually no different than any of the computer models that are so much in the news these days. I once commented to a friend that I couldn’t shake the sense that all of the climate change hysteria was just people who were scaring themselves with their own spreadsheets. Such people are like my finance friend, only more so: they have forgotten that computer models and reality are two very different things.

Nowhere has the inability to make distinctions between computer models and reality been applied with more devastating results than with the policy prescriptions that emerged during Covid. Lasting harm was done, people actually died, entirely due to the hysteria that was provoked by computer models. And I suspect we all know people who continue to struggle with the lasting psychological trauma of having believed the doomsday scenarios of the Covid computer models.

AI represents, really, the 4th technology epoch of my working life, and it is sometimes fun, but always interesting, to be involved with AI. I spent the weekend experimenting with something Andrej Karpathy has called "vibe coding". Vibe coding involves combining the functionality offered by multiple AI models to automate the act of doing software development. In my case, I connected an AI-enabled voice dictation technology with an AI-assisted software development environment, and this allowed me to just talk to my computer and have it write software for me. I think the term vibe, in “vibe coding”, reflects an attempt to capture the notion that you only need to describe what you want in the highest level terms, and the computer model will perform the heavy lifting to generate the code for you. A better term for this kind of software development would, I think, be "riff coding", since it's less about the feels and more about iterating on a theme. But you get the idea.

It's really kind of astonishing what it can do. I sat in my comfy office chair with my heels up on the desk, sipping my coffee, and said to my laptop, "make a fully functional website that mimics the look-and-feel of the Google search page." About 15 seconds later, I had exactly that, down to the last detail, including images, html, javascript, and style sheets for controlling font and layout. Then I said, distractedly since I was occupied watching funny YouTube shorts, "also generate a language model that supports both training and inference, and go ahead and generate some sample training data while you're at it." Within a few seconds I had that as well.

This late into my career, I think I could get very used to continuing to collect a paycheck while my laptop does all of the work that I have historically had to do for myself. Maybe this is what Trump means by "the golden age".

The experience of riff coding is equal parts fun and eerie. And it’s easy to forget, very easy, that what the computer is doing is merely recapitulating human knowledge. Language models, especially, are merely numerical encodings of human knowledge. The encoding techniques being employed are extremely clever in the way they numerically represent, not just the words themselves, but the way all of the words represented within a model are statistically likely to exist in positional proximity to one another. The numerical encoding not only represents, say, the word blue itself, but it also encodes the statistical likelihood of the word blue coexisting with other words in a sentence, and statistically where those other words are, positionally, in such a sentence relative to the word blue.

But all such word-related data is a reflection of human intelligence. And for all of the talk about computers having their own intelligence, these language models are really only reproducing fragmented recombinations of encoded expressions of human intelligence. And there is substantial evidence emerging that trying to train models on their own perorations induces the models to collapse into incoherence.

But it isn’t just the fact that models encode human knowledge. It is also the case that the way these models recapitulate this encoded knowledge is entirely determined by the question(s) they are given. A language model mechanistically generates a response by numerically encoding the user’s question - or ‘prompt’ - and then using said numerical encoding of the supplied prompt as input to its computations.

I have elsewhere described language model prompts as akin to pressing keys on a piano keyboard. You might also conceive of them as dipping a net into an aquarium in order to pull out some of the fish. The aquarium doesn’t produce fish of its own accord. It is the human wielding the dip net who is the active agent. Thus, everything a language model produces is an artifact of human intelligence.

My experience “riff coding” over the weekend brought all of these thoughts to mind, because it would be very easy to believe that it was the computer itself that was doing all that work. But in reality, all the encodings of its model(s) were a product of human intelligence. The computer was doing a lot of matrix operations on numerical values obtained from human knowledge. But it wasn’t producing anything that was not pre-existing encoded knowledge, originally produced by human beings. Even the software produced by my weekend “riff coding” only came into being because I myself provided the original human words as inputs to steer the model’s computations.

One of the things suggested by my weekend experience is that the universe of people who can ‘code’ is about to grow very rapidly. And I wonder if the elimination of the need to master the technical minutia, which is what riff coding does, is planting the seeds of its own demise. If the stream of original, human-crafted, code begins to dry up, we may find that the advances of our vaunted models will wheeze to a halt. We risk confining them to subsisting on old encodings which were obtained from the code produced by those generations who had mastered the technical minutia, and still knew what questions to ask.

AI is a product of, and dependent on, human intelligence and creativity. Language models masquerade as machine intelligence, but they are really only encoded databases of very human knowledge. Much of the language being used to describe AI, by both its proponents and by its detractors, is unhelpful because it fuzzes up our ability to maintain proper category distinctions between what is mechanistic and what is human. AI models are to human intelligence only as video recordings are to reality. Which is to say, they are merely a snapshot and mirage of the human and the living. We should vigorously resist any temptation to mistake them for the original.