AI Is Dumber Than Investors Think

SUMMARY

Gary Marcus, NYU professor and AI skeptic, discusses the overestimation of generative AI's capabilities and the economic and regulatory challenges facing the industry. He argues that current AI models are inherently unreliable, warns against financial overcommitment, and calls for more nuanced regulation and exploration of alternative approaches.

MAIN POINTS

• Gary Marcus outlines his concerns about the overreliance on large language models and the industry's shift from intellectual curiosity to profit-driven motives. [1:00]
• Marcus explains the technical limitations of large language models, emphasizing their inability to reason or maintain stable models of the world. [6:02]
• He discusses the widespread overattribution of intelligence to AI systems and the resulting economic and policy implications. [10:01]
• Marcus highlights the dangers of generative AI's unreliability, particularly their poor rule-following and tendency to hallucinate information. [16:01]
• The conversation turns to regulatory actions, such as subpoenas against OpenAI, and the societal need for accountability in AI outputs. [21:00]
• Marcus notes a shift in U.S. government attitudes toward AI regulation, driven by concerns over models like Anthropic's Mythos. [27:01]
• He provides a nuanced assessment of the Mythos model, acknowledging its risks for poorly secured systems but downplaying fears of catastrophic threats. [31:00]
• The discussion covers recent executive orders and the inadequacy of voluntary, narrow regulatory steps, advocating for mandatory and comprehensive oversight. [37:02]
• Marcus critiques proposals for government investment in AI companies, arguing that these firms lack sustainable business models and should not be bailed out. [42:01]
• He analyzes the economic challenges facing generative AI companies, including inefficiency, price wars, and the lack of clear profitability. [45:01]
• Marcus predicts that OpenAI may fail financially, while Anthropic could survive if it finds profitable niches, but overall industry profitability remains uncertain. [49:02]
• He concludes by dispelling the myth that generative AI is close to artificial general intelligence and urges society to explore alternative approaches to AI development. [55:00]

DETAILED ANALYSIS

Gary Marcus, a prominent AI researcher and skeptic, raises fundamental concerns about the current trajectory of artificial intelligence, particularly generative AI models such as large language models (LLMs). He argues that the global economy and policy frameworks are dangerously overestimating the intelligence of these systems, leading to misplaced investments and regulatory approaches. Marcus traces the shift in AI from a field driven by intellectual curiosity to one dominated by financial incentives, noting that the pursuit of profit has narrowed the range of explored technologies and fostered a culture of hype and overstatement.

Technically, Marcus explains that LLMs function primarily as next-token predictors, stringing together words based on patterns learned from vast datasets, typically the entire internet. While this allows them to mimic human language convincingly, their understanding is superficial and brittle. When faced with scenarios outside their training data, such as classic logic puzzles or nuanced reasoning tasks, these models often produce nonsensical or erroneous outputs.

Marcus cites historical examples, such as the Eliza chatbot from the 1960s, to illustrate how easily humans are fooled by superficial linguistic competence, a phenomenon now amplified by the scale and ubiquity of modern AI.

He emphasizes that LLMs lack stable, internal models of the world and cannot reliably reason or follow instructions. This unreliability manifests in persistent 'hallucinations'—the confident generation of false information—and poor rule-following, which undermines their suitability for critical applications. Despite repeated industry assurances that more data and larger models would eliminate these issues, Marcus notes that hallucinations and unreliability remain unresolved, as confirmed by recent benchmarking studies.

The economic implications are significant. Marcus observes that the AI industry is characterized by a lack of genuine differentiation, with most companies building similar models on the same underlying architectures. This homogeneity leads to intense competition, price wars, and the erosion of potential profit margins.

He likens the situation to a market flooded with indistinguishable toothpaste brands, making it impossible to command premium prices. The result is that major AI firms, including OpenAI and Anthropic, are burning through vast sums of capital without achieving sustainable profitability. For example, OpenAI reportedly lost $21 billion in operating profit last year, with every user interaction representing a net loss.

Marcus is critical of proposals for government intervention, such as acquiring stakes in AI companies or providing loan guarantees for data centers. He argues that these measures amount to backdoor bailouts for fundamentally unprofitable businesses and risk entrenching industry interests without delivering public value. Instead, he advocates for robust, mandatory regulation focused on accountability, transparency, and pre-market evaluation of AI systems, drawing analogies to FDA drug approvals.

He warns against the privatization of gains and socialization of costs, a pattern he sees recurring in the tech sector.

On the regulatory front, Marcus notes a recent shift in U.S. government attitudes, moving from a hands-off approach to a more interventionist stance, partly in response to the perceived risks of advanced models like Anthropic's Mythos. While he acknowledges that some regulatory proposals lack nuance or are overly broad, he sees the growing recognition of AI's societal risks as a positive development. He stresses the need for comprehensive oversight that addresses not only cybersecurity but also issues like hallucinations, sycophancy, and the broader social impacts of AI deployment.

Regarding the Mythos model, Marcus offers a balanced perspective. He acknowledges that while Mythos introduces new capabilities that could be exploited in poorly secured systems, it does not represent an existential threat to well-defended infrastructure. Instead, its emergence highlights longstanding weaknesses in cybersecurity practices and the need for organizations to prioritize digital defenses.

Looking ahead, Marcus is skeptical about the long-term viability of the current generative AI business model. He suggests that only companies able to identify and dominate profitable niches—such as coding assistance—might survive, but even these opportunities may not justify the enormous capital investments being made. He predicts that OpenAI, once the industry leader, could become a cautionary tale akin to WeWork, while Anthropic may have a better chance if it can achieve operational efficiency and market differentiation.

Finally, Marcus warns against the prevailing myth that generative AI is on the verge of achieving artificial general intelligence (AGI) and solving a wide array of human problems. He argues that true intelligence is multifaceted and that current models are far from matching human cognitive abilities. He urges policymakers, investors, and the public to support exploration of alternative AI approaches and to cultivate a deeper understanding of intelligence, rather than committing prematurely to a single, flawed technology.

Marcus points to China's more diversified investment strategy as a potential model, suggesting that the U.S. risks being left behind if it continues to focus exclusively on LLMs without fostering broader innovation.

LINKS

• Prof G Markets newsletter subscription page.
• Order page for 'Notes On Being A Man'.
• Scott Galloway's Instagram profile.
• Ed Elson's Instagram profile.
• Ed Elson's X (formerly Twitter) profile.
• Ed Elson's Substack newsletter.
• Prof G Markets on Spotify.
• Prof G Markets TikTok Q&A page.
• Prof G Markets main homepage.
• ODO business software platform.
• LinkedIn ads offer page for Prof G listeners.
• VCX by Fundrise investment information and fund prospectus.

KEYWORDS