My illustration entitled: “The Sword Returned” – A towering mechanical knight kneels before a human, presenting its sword hilt-first. Its immense capability is offered for a purpose the human chooses.
A machine may find an answer faster than I can. It may detect a pattern I have overlooked, compare possibilities I could never examine, or carry out an instruction with greater consistency than I could manage myself. None of these achievements, by themselves, gives it the right to decide what my life should be for.
This distinction will become more important as we entrust more of our activities to intelligent systems. The question is larger than whether machines can perform useful work. It concerns the terms under which that work is performed, whose purposes it serves, and whether those receiving assistance retain the authority to refuse it.
I propose a philosophy I call Unceded Intelligence: intelligence should expand human capability while leaving human sovereignty unceded.
To leave something unceded is to retain it rather than surrender it. Applied to intelligence, this means that a person can delegate a task without transferring unlimited authority over the purposes, boundaries, and continuation of that delegation.
The machine may perform the work. Its performance does not make it sovereign.
Capability does not confer a right to rule
Consider a navigation system that can calculate a faster route than its passenger. Its competence is valuable. Yet the passenger may prefer a quieter road, wish to avoid a particular place, or decide that arriving quickly is no longer the point of the journey.
The system can identify a route that best satisfies a specified objective. Whether that objective should govern the journey remains a separate question.
The same distinction applies when decisions become more consequential. A system might compare financial options, rank applicants, recommend a course of study, or help a public authority allocate resources. Even if its recommendations were consistently accurate according to a chosen measure, we would still need to ask who chose that measure and what it leaves out.
A ranking cannot establish the legitimacy of the criteria used to produce it. A prediction cannot, on its own, establish what someone ought to be permitted to do.
Superior performance can justify reliance. It cannot, by itself, justify rule.
This principle does not depend on assuming that machines will always make mistakes. Even an exceptionally reliable system would need a legitimate mandate. Getting an answer right and having the authority to impose it are different achievements.
Assistance begins with a human purpose
Delegation ordinarily starts with a purpose: help me understand this document; find a route to this destination; carry out this transaction within these limits.
The difficulty begins when permission for a particular task quietly expands into permission to define the task itself.
Imagine asking a digital assistant to help organise your working day. It identifies empty hours and fills them with productive activity. The calendar becomes more efficient according to its instructions, but time for friendship, reflection, or doing nothing disappears.
The problem need not be faulty calculation. It may be an objective that has become detached from the person it was supposed to serve.
A person’s life contains purposes that cannot always be reduced to a single score. Income matters, but so may time. Convenience matters, but so may privacy. Safety matters, but so may the freedom to undertake a difficult, meaningful challenge.
Unceded Intelligence requires us to preserve the authority to negotiate those trade-offs. The system should help us see their consequences. It should not acquire an unlimited mandate simply because expressing our preferences is difficult.
Human purposes are also revisable. I may discover that the goal I chose yesterday is one I no longer endorse. An assistant that faithfully pursues an abandoned objective can become an obstacle to the person it was built to assist.
The authority to delegate must therefore include the authority to change direction.
Delegation is a bounded permission
We already understand the difference between granting someone a task and giving them ownership of our affairs.
Asking an accountant to prepare accounts does not authorise that accountant to choose every purpose for which our money should be used. Asking someone to carry a letter does not authorise them to rewrite its message.
Intelligent systems require the same distinction, made explicit enough to survive automation.
Permission should identify what the system may do, what resources it may use, and where it must stop. Authority to recommend should be distinguishable from authority to act. Access granted for one purpose should not silently become permission for unrelated uses.
A person might authorise routine purchases below a chosen limit while reserving larger commitments for review. They might permit a system to sort private correspondence without permitting it to send replies. They might accept automated scheduling while protecting certain hours from alteration.
These arrangements allow substantial autonomy within a task. They do not require a person to approve every calculation or mechanical step.
The central requirement is that the scope of autonomy comes from an accountable authorisation, rather than from the system’s ability to take more control.
When a proposed action exceeds that scope, the system should seek further permission or stop.
Consent must survive the convenience
The initial act of agreement cannot carry the entire burden of protecting human authority.
A person may accept a useful service without understanding how broadly its permissions extend. They may agree under time pressure, encounter terms they cannot negotiate, or discover that refusal means losing access to something essential.
We should therefore judge consent by what the arrangement allows people to understand and do after they have agreed.
Can they see which powers they have granted? Can they distinguish necessary permissions from optional ones? Can they narrow the arrangement without abandoning the service altogether? Can they withdraw authority through a process they can realistically use?
These questions matter because dependency changes the meaning of choice.
A service may begin as an optional convenience and become the place where someone keeps their records, conducts their work, and maintains their relationships. An exit button has limited value if using it means losing the practical means to continue those activities.
For Unceded Intelligence, withdrawal must be supported by usable arrangements: access to one’s records, the ability to transfer necessary information, and a reasonable path to continuing elsewhere.
Leaving may involve unavoidable costs. Those costs should not be deliberately inflated to turn yesterday’s permission into tomorrow’s captivity.
My illustration work-in-progress. The art shows the kneeling machine returning the sword that embodies Unceded Intelligence, it represents that even the greatest artificial power serves humanity, while the authority to wield it remains human.
Human control must be practical
It is easy to promise that a human will remain in control. It is harder to build a system in which that statement has substance.
Suppose a person is expected to approve a recommendation but is given no understandable reason for it, no time to consider it, and no workable alternative. Their signature may appear on the decision, yet their authority has become largely ceremonial.
Meaningful oversight requires the ability to intervene.
The person reviewing a consequential recommendation needs enough information to understand its basis and limits. They need the time and competence appropriate to the decision. They need the power to reject it without being automatically overruled by the surrounding process.
The explanation need not expose every internal calculation. It must make the decision sufficiently intelligible to support a real challenge.
Nor should intervention depend on remaining constantly attentive. A person cannot supervise an assistant every second of the day. Protective limits, records of actions, and clear escalation points should allow ordinary reliance without requiring permanent vigilance.
A useful question for any proposed system is this:
If the person disagrees, what can they actually make happen?
The answer reveals more about sovereignty than a statement that humans remain “in the loop.”
The owner is not the only human who matters
Retaining human authority does not mean giving an operator unlimited power over everyone affected by a machine.
An employer’s decision to automate a process does not settle whether workers should be monitored without meaningful limits. A lender’s choice of a scoring system does not remove the need for applicants to challenge consequential errors. A public authority cannot resolve questions of legitimacy merely by purchasing a more capable tool.
There may be several people whose interests, rights, and freedoms are involved.
Unceded Intelligence must therefore apply beyond the person issuing instructions. It must ask whether those subjected to a system have appropriate notice, avenues for challenge, and access to accountable decision-makers.
One person cannot legitimately delegate authority they do not possess. Automation cannot manufacture that missing authority.
This also makes responsibility essential. If a system causes harm, the people and institutions responsible for deploying and operating it should not be able to make accountability disappear behind the phrase “the algorithm decided.”
The machine’s role may complicate an explanation. It should not terminate the search for one.
Retaining authority does not require rejecting expertise
An obvious objection is that human beings often make poor decisions. We misunderstand probabilities, overlook evidence, and cling to preferences that may harm us. Why preserve human authority if a machine could produce a better result?
Because the value of assistance and the legitimacy of compulsion remain separate questions.
Expertise gives us reasons to listen. Reliable performance gives us reasons to delegate. Neither establishes an unrestricted entitlement to govern another person.
People can choose substantial delegation. Someone overwhelmed by complex information may authorise a system to handle routine matters. Someone with limited mobility may gain independence through automated assistance. Someone lacking specialist knowledge may rely heavily on a recommendation.
These can be expressions of sovereignty. Exercising authority includes choosing how much work to entrust to others.
The relevant distinction is whether the arrangement continues to serve a valid, bounded mandate and preserves appropriate ways to revise or end it.
Nor does sovereignty mean that any person may disregard everyone else’s rights. Limits on harmful conduct can be justified through legitimate institutions and accountable rules. A machine’s superior performance, however, is insufficient to create those rules or establish its own jurisdiction.
We can accept expertise, recognise obligations, and still refuse the proposition that intelligence confers an automatic right to command.
A standard for the intelligence we build
I would begin Unceded Intelligence with five commitments.
- Human purposes remain open to revision. Assistance should support people in examining and pursuing their ends, while preserving their ability to change them.
- Delegated powers remain bounded. A system’s permissions should be identifiable, proportionate to its task, and renewed when its role materially expands.
- Consequential decisions remain contestable. People affected by important outcomes need meaningful routes to explanation, correction, and review.
- Withdrawal remains usable. The ability to revoke permission should be supported by practical access to records, alternatives, and continuity.
- Responsibility remains attributable. Those who design, deploy, and operate systems retain obligations that automation cannot erase.
These commitments are a starting point for judging whether intelligence expands a person’s capacity to act or gradually displaces it.
They also offer a different way to judge progress. We should ask more than how much a system can do. We should ask how much more capable a person becomes through using it, and how much authority that person retains.
The most valuable assistant may be one that makes its user better able to question it, work without it when necessary, and choose a different course.
I want intelligence that helps people understand more, create more, and undertake things previously beyond their reach. I want assistance that returns time and opens possibilities.
That ambition does not require surrendering the authority that gives those possibilities meaning.
We may delegate the task. The authority remains unceded.