Herbert Spencer vs. Karl Marx on AI and Mass Unemployment: Why These Two, and Why Now
A behind-the-scenes look at the debate, the sources, and why they took the positions they did
Why This Topic
The question of whether artificial intelligence will create mass unemployment has become one of the defining arguments of the current decade, and it has a peculiar structure. The optimists point to history: every previous labor-displacing technology, the power loom, the mechanical thresher, the steam engine, the railway, the computer, was greeted with predictions of permanent displacement that turned out to be wrong. New industries emerged. New categories of work appeared. The labor market adapted. The pessimists acknowledge all of this and then say: but this time the technology replaces judgment, not just muscle, and that changes everything.
This is not a new argument. It was made, with remarkable precision, in the 1850s and 1880s by two men who disagreed about almost everything except that the question was important. Karl Marx described the mechanism of technological displacement in his Fragment on Machines and predicted its logical endpoint. Herbert Spencer described the mechanism of adaptation and predicted that it would, as it always had, generate more employment than it destroyed. Both men were working from the same historical evidence. Both men reached opposite conclusions. Their argument is the argument we are having now, and it is worth having it with the people who originated it.
Why Herbert Spencer
Spencer is an uncomfortable figure in the history of ideas because his name became attached, in his own lifetime and after it, to social Darwinism, a doctrine used to justify everything from the Standard Oil monopoly to eugenics programs, and because he spent his later years watching this happen without entirely managing to stop it. The discomfort has caused his actual arguments to be underread, which is a mistake, because his actual arguments are more careful and more interesting than his reputation suggests.
Spencer’s case for technological optimism is not faith in progress. It is an argument from complexity. His core claim in Principles of Sociology and The Man Versus the State is that complex adaptive systems, including economies and labor markets, respond to disruption in ways that cannot be predicted in advance by the people experiencing the disruption, and that interventions designed to prevent the disruption typically produce worse outcomes than the disruption itself would have. He was not indifferent to suffering. He was skeptical that the people proposing to prevent it had correctly modeled what their prevention would cost.
The specific text most relevant to this debate is his 1884 essay collection The Man Versus the State, in which he argued that every expansion of state intervention in economic life produces unintended consequences that require further interventions to address, generating a ratchet of dependency that ultimately constrains the adaptive capacity of the society it was meant to protect. Applied to AI displacement, Spencer would say that the appropriate response is not managed transition but the removal of the barriers, occupational licensing, geographic restrictions, educational credentialism, that prevent workers from adapting. His optimism is conditional on a freedom of adaptation that he would be the first to acknowledge is not currently present.
Why Karl Marx
Marx’s relevance to this specific debate rests primarily on a single text that most people in the AI conversation have still not read. In a section of the Grundrisse written around 1857 and not published until 1939, Marx described what he called the general intellect: the accumulated knowledge and skill of human civilization, which he argued was becoming the primary productive force in advanced economies. His prediction was that as machinery incorporated more of this general intellect, capital would own not just the tools of production but the knowledge itself, and that workers would find themselves not merely displaced from specific jobs but excluded from the productive process entirely, because the productive process would no longer require them.
This is a more precise description of what large language models do than most of what has been written about them in the last five years. A large language model is trained on the accumulated writing of human civilization and the resulting system is owned by a small number of private companies. Marx described this structure in 1857 and called it the logical endpoint of capital’s tendency to embed human labor in fixed machinery. He was right about the structure. The debate is whether he is right about the consequences.
The historical antagonism between Spencer and Marx was genuine and documented. Spencer was the more celebrated figure in England and America during the 1870s and 1880s. Marx read Spencer and dismissed him. Spencer was aware of Marx’s work and considered it a dangerous combination of correct diagnosis and catastrophically wrong prescription. They represent the two positions that have defined every conversation about technological unemployment since, and they represent them with more rigor and more historical grounding than most of the people currently having the conversation.
Who Else We Considered
John Stuart Mill against Marx was the first alternative we evaluated seriously. Mill’s Principles of Political Economy addresses technological unemployment directly and his position is more nuanced than either Spencer’s optimism or Marx’s catastrophism. We held this pairing back because Mill has a tendency to find merit on both sides of every argument, which is intellectually honest and dramatically slow. The debate would have been a seminar. We want a fight.
Alfred Marshall against Marx was our second choice and remains in the queue for a future episode. Marshall is the cleaner intellectual opponent for Marx on pure economic grounds. His Principles of Economics directly addresses the question of whether machinery displaces or ultimately creates more employment, and his answer is the most rigorous version of the optimist case available. We will use him eventually. Spencer is more combustible and the personality contrast is sharper.
David Ricardo against Marx was considered and rejected because the two men agree on too much of the foundational economics. Ricardo’s chapter on machinery in the third edition of Principles of Political Economy is actually a pessimist argument, one of the few places where a classical economist acknowledged that technological displacement might not self-correct, and Marx cited it approvingly in Das Kapital. Two men who agree on the premises make a less interesting debate than two men who disagree about the nature of reality.
Thomas Malthus against Spencer was briefly considered as a way to put two optimism-skeptics in the same room and let them fight about whose pessimism was better grounded. We abandoned it because the debate would have had no one defending the position that technology creates employment, which is the position that has been historically correct and which deserves a serious advocate.
Why Each Man Takes the Position He Does
Spencer’s optimism is grounded in a specific reading of industrial history that he documented across four volumes of Principles of Sociology. He watched the English economy transform between 1820 and 1880 from an agricultural to an industrial society, observed that the predictions of permanent displacement made at every stage turned out to be wrong, and concluded that the adaptive capacity of complex systems is systematically underestimated by the people living through the disruption. His argument is not that disruption is painless. His argument is that the alternative to disruption is stagnation, and that stagnation kills more people, more slowly, and less visibly.
His weakest point, and the one Marx exploits most effectively, is his failure to address the question of who bears the cost of the transition. Spencer’s recovery happens across generations. The weavers of 1820 did not benefit from the electrical engineers of 1890. Their grandchildren did. Spencer’s arithmetic is correct at the civilizational level and incomplete at the individual level, and the people who suffered the transition did not experience it as an acceptable cost of a prosperity they would never see.
Marx’s pessimism about AI specifically rests on a categorical argument rather than a historical one. He does not dispute that previous technologies generated more employment than they destroyed. He argues that artificial intelligence is categorically different because it is the first technology designed to replace human judgment rather than human muscle, and that this closes the last door through which displaced workers could previously adapt. The lawyer displaced by an AI legal system cannot become an AI engineer unless he is already trained as one, and the training pipeline for AI engineers is controlled by the same capital that built the system displacing the lawyer.
His weakest point is the prescription rather than the diagnosis. His analysis of the displacement mechanism is precise and historically grounded. His proposed solution, collective ownership of the means of production, has a historical record that Spencer deploys against him with considerable force, and Marx’s defense, that the Soviet Union betrayed his analysis rather than expressing it, is not entirely wrong but is not entirely satisfying as an answer to five million people dead in Ukraine.
A Note on the Sources
The essential Marx text for this debate is the Fragment on Machines from the Grundrisse, available in the Penguin edition of the Grundrisse translated by Martin Nicolaus. It runs to about thirty pages and is the most directly relevant thing Marx ever wrote to the current AI moment. Das Kapital Volume One, Part Four, on the production of relative surplus value, provides the foundational argument about machinery and labor that the Fragment develops. The Critique of the Gotha Programme, available free in many editions, contains Marx’s explicit warning about transitional state socialism that he deploys in his own defense during the debate.
For Spencer, The Man Versus the State, published in 1884, is the most accessible entry point and the most directly relevant to the state intervention arguments in this debate. Principles of Sociology, particularly Volume Three, contains the evolutionary framework for his labor market optimism. Spencer wrote an autobiography, published in 1904 a year after his death, which is exactly as self-regarding as you would expect and which provides the biographical material Marx uses in the closing segment.
One source that informed the debate structure without appearing in it directly is William Beveridge’s 1909 study Unemployment: A Problem of Industry, which documented the specific patterns of cyclical and structural unemployment in the British economy and provided the empirical baseline against which both Spencer’s optimism and Marx’s pessimism can be measured. Beveridge concluded that the labor market did not self-correct as efficiently as Spencer suggested and that the correction it did achieve left specific populations permanently behind, which is precisely the tension this debate is built around.
What Comes Next
The unemployment question connects directly to the policy response question, which is the debate we are planning between William Jennings Bryan and Herbert Spencer on whether AI capital concentration is natural selection or predatory capture. Bryan’s Cross of Gold logic translates with uncomfortable precision to the current moment, and his fury at the railroad interests maps onto the current conversation about AI monopolies in ways that should make both sides of that conversation uncomfortable.
Alfred Marshall is also waiting in the queue for the most rigorous version of the optimist economic case. If Spencer’s optimism is biological and structural, Marshall’s is technical and empirical, and Marx deserves to face the strongest version of the argument he is most confident he has already defeated.
This debate was produced using AITalkerApp.com. If you want to create your own animated conversations, debates, or interviews, the link is in the video description.