Silicon Valley runs on novelty. It is sustained by the pursuit of what Michael Lewis once called the “new new thing.” The Internet, the smartphone, social media: the new new thing cannot be a modest tweak at the edges. It has to transform the human race. The economic incentives are clear: a firm that popularizes a paradigm-shattering invention can make a lot of money. But there is also something larger at stake. If Silicon Valley doesn’t keep delivering new new things, it loses its privileged status as the place where the future is made.
In 2022 the industry was having a bad year. After a lucrative pandemic—the five most valuable tech companies added more than $2.6 trillion to their combined market capitalization in 2020 and nearly the same amount in 2021—the sector suffered one of its sharpest-ever contractions. Amazon lost almost half of its value, Meta close to two thirds. The tech-heavy Nasdaq fell 33 percent, its worst performance since the 2008 financial crisis.
The reasons were straightforward enough. At the onset of the Covid-19 pandemic, the Federal Reserve slashed interest rates to zero, and people stayed home, where they spent more time and money online. By 2022 both trends were in reverse. Most Americans had decided to stop worrying about the virus and were happily resuming their offline activities. Meanwhile the Fed began hiking interest rates in response to rising inflation.
It would be a mistake to overstate the severity of the “tech downturn” that followed. Despite mass layoffs and declining revenue, the big firms remained larger and more profitable than they had been before the pandemic. Nonetheless a certain malaise had set in. The industry needed a dazzling new invention that could attract billions of consumers and send capital markets into a froth.
One candidate was Web3, a proposal for rebuilding the Internet around blockchain, the accounting technology underlying Bitcoin and other cryptocurrencies. Championed in particular by venture capitalists, who hoped that it would enrich them by empowering a new generation of start-ups to unseat the big firms, Web3 never proved useful for anything except speculation, and even the speculators got soaked as various schemes imploded under the pressure of high interest rates. Another possibility was the metaverse, Mark Zuckerberg’s dream of an immersive Internet experienced through a VR headset. It, too, struggled to demonstrate any practical advantage. Worse, it was unpleasant: a glitchy simulacrum of a postapocalyptic shopping mall as designed by David Lynch, where fish-eyed avatars with no legs floated through sparsely populated cartoon worlds.
Then, on November 30, 2022, OpenAI released ChatGPT. A powerful AI system paired with an affable conversational interface, it let anyone ask a question and get an impressively humanoid (though not always correct) response. By January 2023 the chatbot had amassed 100 million users, making it the fastest-growing Web application ever. It was storybook Silicon Valley: OpenAI, which at the time had only a few hundred employees, caught everyone by surprise and, virtually overnight, established “generative AI”—the category of software to which ChatGPT belongs—as the new master concept of the entire industry. The tech giants rushed to respond, setting off a stampede. Everything from search engines to e-mail clients began sprouting generative AI features. In 2023 the Nasdaq climbed 55 percent, its best performance since 1999. The new new thing had been found.
It’s too soon to know whether generative AI will prove to be a pot of gold or a blast of hot air. Opinions are divided. Some companies have done fabulously well: Nvidia, the boom’s breakout star, is raking it in, since its chips are the basic infrastructure on which generative AI is built. The cloud divisions of Microsoft, Google, and Amazon have also grown considerably, which their executives attribute to increased demand for AI services.
But these are, in the parlance of the financial press, “picks and shovels” plays. Nobody doubts that there is money to be made from selling companies the paraphernalia they need to use generative AI. The real question is whether generative AI helps those companies make any money themselves. Skeptics point out that the high cost of creating and running generative AI software is a potential obstacle. This negates the traditional advantage of digital technology: its low marginal costs. Starting an online bookstore worked for Amazon because it was cheaper than going the brick-and-mortar route, as Jim Covello, head of global equity research at Goldman Sachs, noted in a June 2024 report. AI, by contrast, is not cheap—which means “AI applications must solve extremely complex and important problems for enterprises to earn an appropriate return on investment.” Covello, for one, doubts that they will.
Yet companies, like people, are not entirely rational. When a firm decides to adopt a new technology, it rarely does so on the basis of economic considerations alone. “Such decisions are more often than not grounded upon hunches, faith, ego, delight, and deals,” observes the historian David Noble. Looking at American factories after World War II, Noble identified a number of reasons for their shift to “numerical control” technology: a “fascination with automation,” a devotion to the idea of technological progress, an urge to be associated with the prestige of the cutting edge, and a “fear of falling behind the competition,” among others.1
Noble does, however, put special emphasis on one motivation that is at least partly rooted in economic rationality: labor discipline. By mechanizing the production process, managers could more fully master the workers within it. The philosopher Matteo Pasquinelli takes a similar view in his recent book The Eye of the Master: A Social History of Artificial Intelligence. In the introduction, Pasquinelli, a professor at the Ca’ Foscari University of Venice, explains that he won’t be offering a “linear history of mathematical achievements.” Rather, he wants to provide a “social genealogy” that treats AI not merely as a technological pursuit but “as a vision of the world.” The centerpiece of this vision is the automation—and domination—of labor. Contemporary AI is best understood, he believes, as the latest in a long line of efforts to increase the power of the boss.
In The Wealth of Nations, Adam Smith famously argued that the manufacture of pins could be made more efficient through the division of labor. Instead of having a single pinmaker do everything, you could break the job down into several distinct tasks and distribute them to make pins more quickly. This is the canonical principle of capitalist production and the one that automation epitomizes and enforces. First you make work more mechanical, then you delegate it to machines.
An important popularizer of this principle was Charles Babbage, a central figure in Pasquinelli’s book. Originally a mathematician, Babbage became what we would now call a “thought leader” among the nineteenth-century British bourgeoisie. Today he is better known as one of the inventors of the computer. His work on computation began with the observation that the division of labor could be “applied with equal success to mental as to mechanical operations,” as he stated in an influential 1832 treatise. The same method of industrial management then remolding the British worker could be transported outside of the factory and applied to a very different kind of labor, Babbage believed: mathematical calculation.
He took inspiration from Gaspard de Prony, a French mathematician who came up with a scheme for streamlining the creation of logarithmic tables by reducing most of the work to a series of simple additions and subtractions. In de Prony’s arrangement, a handful of experts and managers planned the job and did the more difficult calculations while an army of menial number crunchers did lots of basic arithmetic.
If the poor grunts at the bottom of this pyramid were basically automatons, why not automate them? In the factory the division of labor went hand in hand with automation. In fact, according to Babbage, it was precisely the simplification of the labor process that made it possible to introduce machinery. “When each process has been reduced to the use of some simple tool,” he wrote, “the union of all these tools, actuated by one moving power, constitutes a machine.”
In 1819 he began designing what he called the Difference Engine, which automated the labor of arithmetic with three rotating cylinders and was powered by a steam engine. Babbage’s ambition was enormous. He wanted to “establish the business of calculation at an industrial scale,” Pasquinelli writes, by harnessing the same energy source that was revolutionizing British industry. The mass production of error-free logarithmic tables would also make for a profitable business, because such tables allowed the United Kingdom’s formidable mercantile and military fleets to determine their position at sea. The British government, recognizing the economic and geopolitical value of Babbage’s venture, provided funding.
The investment failed. Babbage managed to construct a small prototype, but the full design proved too complicated to implement. In 1842 the government pulled its support, by which point Babbage had begun dreaming of an even less buildable machine: the Analytical Engine. Designed with the help of the mathematician Ada Lovelace, this extraordinary mechanism would have been the first general-purpose computer, able to be programmed to perform any calculation. Thus, amid the smog and soot of Victorian England, the idea of software was born.
The division of labor was never just about efficiency; it was also about control. By fragmenting craft production—picture a shoemaker making a pair of shoes—into a set of modular routines, the division of labor eliminated the autonomy of the artisan. Now management brought workers together under a single roof, which meant they could be told what to do and watched while they did it.
Pasquinelli believes that Babbage’s engines, originating as they did in a “project to mechanize the division of mental labor,” were driven by the same managerial imperatives. They were, he writes, “an implementation of the analytical eye of the factory’s master,” a sort of mechanical representation of the watchful, despotic boss. Pasquinelli goes so far as to call them “cousins” of Jeremy Bentham’s notorious panopticon.
But these imperatives presumably remained latent, since the gadgets never worked as designed. Babbage tried to use mechanical gears to represent decimal numbers, which meant he struggled with the problem of how to automate the “carryover”—the process whereby one column resets to zero and the next column increases by one—when a digit reaches 10. It would take the simplifications of the binary system, the invention of electronics, and the many advances bankrolled by the ample military budgets of World War II to make automatic computation finally feasible in the 1940s.
By then capitalism had become a more international affair, which made the matter of managing workers more complicated. “The more the division of labor extended into a globalized world,” Pasquinelli writes, “the more troublesome its management became,” as “the ‘intelligence’ of the factory’s master could no longer survey the entire production process in a single glance.” Thus, he contends, the need for “infrastructures of communication” that “could achieve this role of supervision and quantification.”
The modern computer, in the decades following its arrival in the 1940s, helped satisfy this need. Computers extended the “eye of the master” across space, Pasquinelli argues, enabling capitalists to coordinate the increasingly cumbersome logistics of industrial production. If Babbage had wanted to construct a prosthesis with which to project managerial power, as Pasquinelli suggests, then computation’s triumph in the twentieth century as the indispensable instrument of capitalist globalization should be understood as a fulfillment of the technology’s founding spirit.
Moreover, this spirit appears to have intensified as computers continued to evolve. “Since the end of the twentieth century,” Pasquinelli writes,
the management of labor has turned all of society into a “digital factory” and has taken the form of the software of search engines, online maps, messaging apps, social networks, gig-economy platforms, mobility services, and ultimately AI algorithms.
AI, he concludes, is accelerating this transformation.
There is no doubt that computers are often used to the advantage of employers, from the scheduling software that reduces labor costs by saddling retail and restaurant workers with unpredictable schedules to the several species of “bossware” that enable the remote surveillance and supervision of office employees, Uber drivers, and long-haul truckers. But to argue that such uses are the raison d’être of digital technology, as Pasquinelli seems to, is to overstate the case.
Labor discipline is one use to which computers can be put; there are many others. And it was not central to the technology’s development: the core innovations of computing arose in response to military prerogatives, not economic ones. The desires to crack enemy cryptography, calculate the correct angles for aiming artillery, and perform the math necessary for making the hydrogen bomb were a few of the motivations for building computers in the 1940s. The US government became infatuated with the technology and spent millions on research and procurement in the subsequent decades. Computers would prove integral to a variety of imperial pursuits, from assembling intercontinental missiles capable of (precisely) incinerating millions of Soviets to storing and analyzing intercepts sourced from listening stations around the world. American corporations followed behind, adapting the contraptions cooked up by the security state to various commercial ends.
Still, if Pasquinelli’s claims do not always convince, there is much to learn from the material he presents. Publishers have inundated readers with books about AI in recent years—enough to fill a small bookstore.2 Most have an undercooked quality. The Eye of the Master is, if anything, overcooked: there is an enormous amount of thinking compressed into its pages. Pasquinelli’s omnivorous intellect often mesmerizes. Nonetheless, I sometimes found myself wishing he would slow down to scaffold his provocations with more evidence.
The fact that Babbage drew from the playbook of industrial management in designing his protocomputers is an interesting piece of history. But its relevance to later developments, or even just its resonance with them, can be determined only by looking closely at how computers actually transformed work in the twentieth and twenty-first centuries, which Pasquinelli does not do. Instead he takes a sharp turn halfway through the book, pivoting from nineteenth-century industrial Britain to the early AI researchers of mid-twentieth-century America, focusing in particular on the field’s “connectionist” school.
Connectionism, as Pasquinelli notes, departed in significant ways from the automatic computation of Babbage. For Babbage the soul of the computer was the algorithm, a step-by-step procedure that traditionally makes up the principal ingredient of a computer program. When Alan Turing, John von Neumann, and others created the modern computer in the twentieth century, what they created was a device for executing algorithms. The programmer writes a set of rules for transforming an input into an output, and the computer obeys.
This ethos also guided “symbolic AI,” the philosophy that came to dominate the first generation of AI research. Its adherents believed that by programming a computer to follow a series of rules they could turn a machine into a mind. This method had its limits. Formalizing an activity as a logical sequence works fine if the activity is relatively simple. As it becomes more complex, however, hard-coded instructions are less useful. I could give you an exact set of directions for driving from my house to yours, but I couldn’t use the same technique to tell you how to drive.
An alternative approach emerged from cybernetics, a postwar intellectual movement with extremely eclectic interests. Among these was the aspiration to create automata with the adaptability of living things. “Rather than imitating the rules of human reasoning,” Pasquinelli writes, the cyberneticians “aimed at imitating the rules by which organisms organize themselves and adapt to the environment.” These efforts led to the invention of the artificial neural network, a data-processing architecture loosely modeled on the brain. By using such networks to perceive patterns in data, computers can train themselves in a particular task. A neural network learns to do things not by simplifying a process into a procedure but by observing a process—again and again and again—and drawing statistical relationships across the many examples.
One of connectionism’s progenitors was Friedrich Hayek, the subject of Pasquinelli’s most intriguing chapter. Hayek is best known as a leading theorist of neoliberalism, but as a young man he developed an interest in the brain while working in the Zurich lab of the famous neuropathologist Constantin von Monakow. For Hayek the mind was like a market: he saw both as self-organizing entities from which a spontaneous order arises through the decentralized interaction of their components. These ideas would help influence the development of artificial neural networks, which in fact function much like the market mind of the Hayekian imagination. When a psychologist named Frank Rosenblatt implemented the first neural network with the help of a navy grant in 1957, he acknowledged his debt to Hayek.
But Hayek also diverged from the cyberneticians in important respects. Cybernetics, as the philosopher Norbert Wiener defined it in his eponymous 1948 book, involved the scientific study of “control and communication in the animal and the machine.” The term, coined by Wiener, was derived from the ancient Greek word for the steersman of a ship, which shares the same root as the word for government. The cyberneticians wanted to create technological systems that could govern themselves—a prospect that appealed to a Pentagon looking for ways to gain a military advantage in the cold war. The navy funded Rosenblatt in the hopes that his neural network could assist in “the automation of target classification,” Pasquinelli explains, by using its powers of pattern recognition to detect enemy vessels.
For Hayek, by contrast, connectionism offered a way to think about a system that eluded control. He had a special kind of control in mind: economic planning. In his view, the brainlike complexity and distributed architecture of the market meant that socialism could never work. Thus the need for neoliberal policies that, in the words of the historian Quinn Slobodian, would “encase the unknowable economy,” protecting it from government interference.3 Nonetheless, Hayek and the other connectionists were very much on the same team. Rosenblatt and his colleagues were able to secure funding for their research because the US government believed their ideas could help defeat socialist armies. Hayek was in the business of defeating socialist ideas.
At first, connectionism failed to fulfill its promise. By the early 1970s it had fallen out of favor in the AI world. Still, neural networks continued to develop quietly over the subsequent decades, enjoying some breakthroughs in the 1980s and 1990s. Then, in the 2010s, came connectionism’s quantum leap.
Training a neural network, as Rosenblatt once pointed out, requires “exposure to a large sample of stimuli.” Size matters: since neural networks learn by studying data, how much they can learn depends in part on the quantity of data available to them. For much of computing’s history, data was expensive to store and difficult to transmit. By the second decade of the twenty-first century, both barriers had dissolved. Plummeting storage costs, combined with the birth and growth of the Web, meant that a mountain of words, photos, and videos was accessible to anyone with an Internet connection. Researchers used this information to train neural networks. The abundance of training data, along with new techniques and more powerful hardware, led to swift progress in fields like natural language processing and computer vision. Today AI based on neural networks is ubiquitous, at work in everything from Siri to self-driving cars to the algorithms that curate social media feeds.
Neural networks also underpin generative AI systems like ChatGPT. Such systems are particularly large—meaning they are composed of many layers of neural networks—and their appetite for data is immense. The reason that ChatGPT sounds so lifelike and seems to know so many things about the world is that the “large language model” inside it has been trained on terabytes of text drawn from the Internet, including millions of websites, Wikipedia articles, and full-length books. This is what Pasquinelli means when he writes that the neural networks of contemporary AI are “not a model of the biological brain but of the collective mind,” a social endeavor to which many people have contributed.
Not everyone is pleased by this fact. Generative AI’s voraciousness is responsible for what the podcast host Michael Barbaro calls its “original sin”: copyrighted material is among the information ingested. The New York Times has sued OpenAI for copyright infringement; so has the Authors Guild, alongside Jonathan Franzen, George Saunders, and several other writers. While OpenAI and the other major “model creators” do not disclose the details of their training data, OpenAI has conceded that copyrighted works are included—yet maintains that this falls under fair use. Meanwhile the demand for training data keeps growing, compelling tech companies to find new ways of obtaining it. OpenAI, Meta, and others have struck licensing deals with publishers such as Reuters, Axel Springer, and the Associated Press, and they are exploring similar arrangements with Hollywood studios.
For Pasquinelli there is a lesson here. Contemporary AI’s reliance on our aggregated contributions proves that intelligence is a “social process by constitution.” It is communal, emergent, diffuse—and thus a perfect match for the connectionist paradigm. “It comes as no surprise that the most successful AI technique, namely artificial neural networks, is the one that can best mirror, and therefore best capture, social cooperation,” he writes.
There is a Marxist coloring to this argument: intelligence resides in the creativity of the masses. But it is also an argument that could have been made by the profoundly anti-Marxist Hayek. The old Austrian would be gratified to know that the “intellect” of the most sophisticated software in history is sourced from the unplanned activities of a multitude. He would have been further tickled by the fact that such software is, like his beloved market, fundamentally unknowable.
The strangeness at the heart of the generative AI boom is that nobody really knows how the technology works. We know how the large language models within ChatGPT and its counterparts are trained, even if we don’t always know which data they’re being trained on: they are asked to predict the next string of characters in a sequence. But exactly how they arrive at any given prediction is a mystery. The computations that occur inside the model are simply too intricate for any human to comprehend. You can’t just pop the hood and watch the gears click away.
In the absence of direct observation, one is left with a more oblique method: interpretation. An entire technical field has sprung up around AI “interpretability” or “explainability,” with the goal of puzzling out how such systems work. Its practitioners across academia and industry talk in scientistic terms, but their endeavor has a devotional quality, not unlike the exegesis of holy texts or of the entrails of freshly sacrificed sheep.
There is a limit to how much meaning can be made. Mortals must content themselves with partial truths. If today’s “AI monopolies” represent “the new ‘eye of the master,’” as Pasquinelli believes, it is an eye with a limited field of vision. The factories of Babbage’s day were zones of visibility: by concentrating work and workers, they put the labor process in full view. Contemporary AI is the opposite. Its casing is stubbornly opaque. Not even the master can see inside.
English (US) · 