In a recent Foreign Affairs essay titled “The Cost of the AGI Delusion,” Michael C. Horowitz and Lauren A. Kahn argue that America’s AI policy is overly fixated on chasing artificial general intelligence (AGI) — and that this focus is causing us to fall behind in the real AI race. They warn that by betting heavily on the hypothetical arrival of superintelligent systems, we risk neglecting practical, near-term investments in narrow AI, safety, scaling, and governance. The authors assert that the hype around AGI has distorted incentives, heightened risk taking, and diverted attention from the technical, economic, and policy challenges actually in front of us. Meanwhile, critics of the AGI narrative (such as AI Now Institute) contend that much of the momentum behind AGI is driven by marketing and sociotechnical fiction rather than actual scientific consensus. One recent academic paper frames AGI as a long-term speculative project upheld by “deep hype” that crowds out democratic oversight and rational policy. Others estimating timelines for truly transformative AGI see negligible probabilities (<1 %) by, say, 2043 — suggesting we may be overinvesting in a mirage.
Sources: Foreign Affairs Magazine, AI Now Institute
Key Takeaways
– Overemphasizing AGI as an imminent goal may distort research priorities and resource allocation toward speculative goals rather than tangible problems in the present.
– The narrative of inevitability around AGI is in many cases constructed by marketing, investor incentives, and sociotechnical fictions, not robust scientific consensus.
– Empirical analyses suggest that the probability of achieving transformative AGI in our near-to-midterm is extremely low, so policy and governance efforts ought to remain grounded, flexible, and safety-oriented.
In-Depth
At the heart of the argument in “The Cost of the AGI Delusion” is a warning: we’re placing too much emphasis on the promise of an all-purpose, superintelligent machine, and too little on the more immediate challenges of scaling, safety, fairness, robustness, and governance. Horowitz and Kahn caution that this kind of distant goal orientation can warp incentives in AI development: labs may underinvest in safety, overpromise capabilities, or take undue risks to appear to be “leading” the AGI race. If your organization’s mission is cast in existential terms (“whoever builds AGI first wins the world”), you may ignore incremental but valuable progress, or fail to maintain deliberate, accountable governance structures.
This critique aligns with a growing chorus of AI researchers and ethicists who argue the AGI narrative is more cultural myth than technical inevitability. For example, the AI Now Institute recently published “The AGI Mythology,” which explains how narrative framing makes AGI seem omnipresent and unavoidable, giving companies and governments license to adopt fatalistic postures rather than deliberate strategies. This kind of rhetoric can turn policy into passivity: “If AGI is inevitable, we can only react — so why regulate today?” That is a dangerous path, since many of the risks and trade-offs we face (power concentration, bias, misuse, infrastructure fragility) are rooted in today’s narrow AI systems.
A further dose of realism comes from probabilistic estimates. In one study, researchers assessed the chance of achieving transformative AGI by 2043 at under 1 percent, carefully decomposing the required advances in software, hardware, and societal coordination. That’s a sobering figure: if transformative AGI is far less likely than many hype cycles suggest, then overinvesting in speculative futures is a poor strategic bet. More broadly, a recent paper on “deep hype” interrogates how AGI is sustained as a vision through sociotechnical fictions, and how that in turn helps shift regulatory attention away from present challenges.
From a conservative but pragmatic perspective, the right move is balance. Yes, guardrails, long-term thinking, and safety work are crucial. But they must be paired with grounded investments in incremental innovation, transparency, empirical evaluation, and institutional capacity. If we pretend AGI is always just around the corner, we risk stoking investment bubbles, encouraging overreach, and undermining credibility when reality fails to match rhetoric. A better posture is deliberate humility: attend to the risks we can see and measure now, create governance mechanisms that scale with capability, and keep our bets diversified rather than all in on one speculative future.