Autonomy is earned,
not configured

Most agentic products ship with an autonomy setting. Somewhere in the preferences there is a slider or a dropdown: ask before every action, ask for risky actions, run fully autonomously. The user picks a level, and the product treats that choice as the truth about how much this person trusts the system.

This is a category error. Trust is not a configuration value. It is a relationship that develops over time, through repeated interaction, and it moves in both directions. A settings panel asks users to answer a question they cannot answer yet: how much should I trust an agent I have never worked with, on tasks it has never done for me? The honest answer on day one is "I don't know," and no slider position expresses that.

What the research actually says about trust

The foundational work here predates the current agent wave by two decades. Lee and See's 2004 paper on trust in automation established that the design goal is not maximizing trust but calibrating it: aligning the user's trust with the system's actual capability. Too much trust produces misuse, where people rely on automation in situations it cannot handle. Too little produces disuse, where capable automation sits idle because nobody dares to lean on it. Both failure modes are expensive, and both are design problems, not user problems.

Crucially, calibration is dynamic. A systematic review of trust in human-AI interaction confirms that appropriate trust forms through experience with the system, not through upfront declarations. Okamura and Yamada's work on adaptive trust calibration goes further: they propose monitoring the user's actual reliance behavior to detect when trust has drifted out of alignment, and then presenting "trust calibration cues" that prompt the user to re-evaluate. The system watches how you use it and intervenes when your trust no longer matches its performance in either direction.

If trust forms through interaction and drifts over time, then a static autonomy setting is wrong almost by definition. It is accurate for at most one moment, the moment it was set, and it decays from there.

The alternative: delegation as a trajectory

Progressive delegation means the agent's autonomy level is derived from the working history between this user and this agent on this class of task, rather than declared once in settings. The mechanics are not exotic. The interaction data already exists in any agent product that asks for approvals.

Consider what an approval history contains. Every time a user approves a proposed action without modification, that is a signal. Every time they edit the proposal before approving, that is a different signal. Every rejection, every undo, every manual correction after the fact is information about where the trust boundary actually sits. A product that collects approval decisions and throws them away is discarding the most valuable trust data it will ever have.

Research frameworks are converging on this view. Recent work on levels of autonomy for AI agents argues for treating autonomy as a deliberate design decision that can differ per task and per context, rather than a global property of the system. And work on governance of agentic systems notes that increases in autonomy should be explicit, documented transitions as reliability is demonstrated, not silent drift. Both point the same direction: autonomy is granular, earned, and legible.

What would this look like concretely? A few patterns follow from the principle:

Scoped promotion. Autonomy increases per task type, not globally. An agent that has earned the right to file expense reports unsupervised has earned nothing about sending emails to clients. Trust generalizes poorly, and the interface should not pretend otherwise.

Visible thresholds. When the agent proposes moving from "approve each step" to "approve the plan, then run," it should say why: "You've approved the last 20 actions of this type without changes." The user confirms the promotion. This keeps the human as the authority over autonomy transitions while grounding the decision in evidence rather than guesswork.

Symmetric demotion. Trust must be able to fall as well as rise. A rejected action, a correction, an undo should feed back into the delegation level, and the agent should step back down without being asked. An agent that only ever gains autonomy is not calibrating; it is creeping.

Legible history. The user should be able to see the trajectory: what the agent is currently trusted with, what it is still supervised on, and what changed recently. This turns an invisible relationship into an inspectable one.

Why the slider persists anyway

If static autonomy settings are so clearly wrong, why does every product have one? Partly because they are easy to build and easy to explain. Partly because they move responsibility to the user: if the agent did something destructive at "full autonomy," well, you chose that setting.

There is also a subtler reason. Progressive delegation requires the product team to define what counts as evidence of reliability, per task type, and to build the bookkeeping for it. That is real design work with real ambiguity. A slider avoids the question. But avoiding the question does not make it disappear; it just transfers the calibration burden onto users, who, as the research consistently shows, are not well positioned to carry it. There is early evidence that getting this right pays off in perceived control, not just safety: studies of risk-contingent autonomy suggest users can feel more in control under adaptive autonomy than under blanket low-autonomy designs.

For enterprise contexts, progressive delegation has an additional advantage: it produces an audit trail by construction. Every autonomy promotion is a documented decision with supporting evidence. That is a governance story that a preferences slider cannot tell.

Takeaways

• Treat autonomy as a per-task trajectory, not a global setting: derive the current delegation level from approval history for that task class, and let users inspect and override it at any time.
• Make autonomy transitions explicit and evidence-based. When the agent proposes more or less autonomy, show the evidence, and let the user confirm the change; the system never promotes itself silently.
• Design the demotion path first. Decide what signals reduce autonomy and how quickly, before designing how autonomy grows. A delegation model without a credible path down is a trust ratchet, and users will sense it.