Genie in a Model

The discussion around the AI future today often has an apocalyptic feel: Right now, AI isn’t all that good, but one day, they will invent Artificial General Intelligence (AGI), and then we’ll all be cooked. I think this misses the mark in two key ways.

First, it implies a binary on-off switch from before to after.

Second, it suggests that there’s a single thing—”intelligence”—that it takes to get there.

Instead, think in terms of the ability of AI to operate more or less autonomously, and to do so differently for different tasks. Seeing AI in terms of autonomy shows us how to leverage increasingly powerful models. To see why, let’s unpack what’s wrong with the standard story.

The Genie Analogy

The apocalyptic story assumes there’s a before/after moment when AI crosses some intelligence threshold and everything changes. But think about genie stories. The genie has unlimited power from the first wish. The problem was never “is the genie powerful enough?” It was always: the wisher doesn’t know what to ask for, or asks imprecisely, or fails to consider consequences.

AI is getting more and more genie-like. GPT-5 or Claude can reason through complex problems, write sophisticated code, analyze nuanced situations. They have “enough” intelligence for most useful work. The limitation isn’t raw cognitive ability—it’s that they can’t read your mind. They need context: what success means, what tradeoffs to make, what you want beyond the literal instructions.

The Specification Problem

In the AI world, that specification comes through prompts and context. But context doesn’t work like intelligence does in the apocalyptic story.

There’s no neat single axis. The specification you need is domain-specific—not universal. The context that helps AI write your specific marketing copy doesn’t help it debug your specific database configuration.

And there’s no sharp transition. The amount of specification is always a matter of degree. You can always be more or less detailed about what you want.

Optimizing Your Wishes

Returning to the genie story, these insights change how we should think about the human role. You don’t need to make the genie even more powerful. You’re trying to get it to work for you, and the way to do that is to make many more wishes, and make each wish more valuable, for the same effort from you.

Chat is the familiar starting point. You make a request, AI responds, you clarify. “For this audience, use a more formal tone.” “That approach violates our policy.” “Actually, what I really need is...” Each intervention gives AI the context to know what to do next.

In that interaction, you are a supervisor that the AI escalates to whenever it gets stuck in its current job. You then get it unstuck by giving it some additional “just-in-time” context. However, the goal isn’t to supervise the AI, it is to make many big wishes. To get there, the AI needs to become more autonomous—not “AGI,” but able to take bigger steps before it needs to escalate to you.

The Autonomy Continuum

The natural way to think about that is in terms of how long the AI can work on your behalf. This isn’t a single threshold, but instead depends on the context infrastructure you give it.

With just chat, you’re providing context in real-time for every decision—AI works autonomously for a couple minutes at most.

Build reusable templates that detail what “good” looks like in this setting, and AI can handle sessions of related work with occasional check-ins—autonomy lasting many minutes.

Build scaffolding that makes success externally verifiable—test suites, evaluation criteria, clear constraints—and AI can work toward “done” for hours without constant supervision.

Why Domains Progress Differently

Domains achieve different levels of autonomy at different rates. Coding is ahead because code has natural scaffolding: tests fail or pass, types check or don’t, programs run or error. AI can work for hours or days on code because “working” is externally verifiable.

Most other domains don’t have that clear specification of a successful wish. What is a good strategy brief or a unique blog post? Getting more out of AI in these domains will require providing it the kind of structured autonomy that makes larger wishes possible. Seen through this lens, “AGI” means that wishes become big enough that anything we can ask for can be expressed in a few sentences—if those sentences are said in the right context.

What This Means

The real challenge isn’t AI gaining more power. Like any genie story, the power is already there. The hard part is learning to make bigger, more powerful wishes without ending up as a cautionary lesson.

But it is more than just a metaphor. Seeing it this way changes our relation to AI as it gets more powerful. AI crossing some intelligence threshold is something that happens to us. Finding ways to give AI more autonomy—richer context enabling it to work longer on its own—is a story where we have agency. “AGI” doesn’t just arrive - it’s something we make, step by step, in each of our domains.