My illustration entitled: “The Boundary Redrawn” – A human redraws a luminous circle across a marble floor. Powerful mechanical assistants step back to respect its new edge.
Imagine asking a digital assistant to compare flights for a journey. To do that well, it needs your destination, preferred dates and perhaps a budget. It returns a useful recommendation.
Now imagine that it purchases a ticket, selects a paid seat, books a hotel and shares your itinerary with a commercial partner. Each action is connected to the journey. None necessarily follows from your instruction to compare flights.
The distinction is simple, but consequential: related actions are not automatically authorized actions.
The same problem can develop over time. A system accepted for one purpose may acquire new functions, connect to new services or begin using information in new ways. Its provider may describe these changes as improvements. The person using it may experience them as an expansion of power they never intended to grant.
For Unceded Intelligence, consent must remain specific and revisable. Intelligence should expand what people can do while leaving legitimate human authority intact. That requires permission to remain connected to the task, circumstances and understanding under which it was given.
Authorizing assistance is not an advance surrender to every action, update or purpose that assistance might later acquire.
Permission belongs to a particular mandate
“I agreed” is incomplete unless we can also say what was agreed to.
A person might agree to have messages sorted without agreeing to have replies sent. They might permit an application to analyse a document without permitting its contents to be reused for an unrelated commercial purpose. They might authorize a purchase within a limit without authorizing a continuing subscription.
Permission concerns a relationship between a person, an intended purpose and a defined set of powers. Remove those connections and the original agreement becomes difficult to interpret meaningfully.
A useful mandate should therefore make clear what the system may do, what information it may use, who may receive that information, how long the authority lasts and when further approval is needed.
Not every instruction requires a lengthy document. For a simple task, an ordinary sentence may establish the relevant boundaries. The detail should match the significance of the delegation.
But convenience cannot justify treating an undefined mandate as unlimited permission. Where an important boundary is unclear, the system should clarify it before crossing it.
A purpose does not authorize every means
One of the easiest ways to expand authority is to appeal to the user’s own objective.
If a person wants a successful journey, a system might reason that booking accommodation would help. If they want a more efficient working day, it might reason that cancelling a social appointment would free useful time. If they want lower expenditure, it might identify commitments they could abandon.
These suggestions may deserve consideration. They do not establish permission to act.
A person’s objective sits alongside other commitments, preferences and obligations. Some may be unstated because the person never imagined they would become relevant to the task.
The system cannot assume that the most direct route to a stated goal is an acceptable route. Nor should it decide that a constraint can be ignored because doing so would produce a better result according to its chosen measure.
Unceded Intelligence requires a distinction between proposing a broader course of action and acquiring authority to carry it out.
The usefulness of an action is a reason to offer it for consideration. It is not a substitute for authorization.
Learning a preference does not create a permission
An intelligent system may become better at anticipating what a person likes. That can make assistance more useful. It can also make the boundary between prediction and instruction easier to overlook.
Suppose a person usually chooses the cheapest available option. A system may reasonably use that pattern to order its recommendations. It should not assume that the pattern authorizes every future purchase, regardless of circumstances.
Preferences can conflict. Someone may ordinarily value low cost but choose flexibility for a particular journey. They may usually welcome reminders but want silence during a difficult week.
Prediction concerns what a person is likely to choose. Authorization concerns what the system is permitted to do. Confidence about the former cannot eliminate the need to establish the latter.
A person can explicitly delegate decisions based on learned preferences. Even then, the delegation needs limits. The system’s growing familiarity with its user should improve assistance within those limits, rather than quietly enlarging them.
System updates can change the agreement
A continuing service is rarely static. Errors are corrected, performance improves and functions change. It would be unreasonable to demand fresh consent for every internal adjustment that leaves the agreed relationship materially unchanged.
The important question is whether a change alters the authority being exercised or the consequences the person has accepted.
A correction that improves an existing calculation may remain within the original mandate. An update that allows the system to send messages, spend money, disclose additional information or serve a different purpose changes the bargain.
The same concern applies if information previously kept on a device begins to be sent elsewhere, or if a tool that offered recommendations begins executing them automatically.
Keeping the same product name does not make these arrangements equivalent.
Material changes should be explained before the expanded powers are used. The explanation should identify what is changing, why it matters, what choices remain and what happens if the person declines.
This does not imply that a provider must maintain every previous version indefinitely. Some changes may be necessary for security or continued operation. Where an old arrangement cannot reasonably continue, the provider should explain that constraint and support a fair transition.
Necessary maintenance should not become a pretext for bundling unrelated new permissions into an unavoidable update.
The renewal test
I propose a straightforward test for deciding when renewed authorization is needed:
Does the change materially alter what the system may do, why it may do it, what information it may use, who receives it, or the consequences and alternatives the person understood when agreeing?
If the answer is yes, the original permission should not simply be stretched to cover the new arrangement.
The person needs a fresh opportunity to understand and choose. Where possible, that choice should concern the changed function itself. Accepting an improvement in one area should not require granting unnecessary powers in another.
Renewal is also relevant when circumstances change without a software update. A temporary project may end. A new organisation may take over the service. A person’s relationship with the intended recipient of information may change.
The purpose of renewal is to preserve the connection between the person’s actual mandate and the authority exercised in their name.
My illustration “The Boundary Redrawn” work-in-progress. The art represents that consent can change in scope while cooperation continues.
Standing permission can be useful and revocable
Specific consent does not require constant interruption.
A person may authorize a system to carry out a recurring task within agreed limits: organise incoming documents, issue a reminder at a chosen time, or make routine purchases below a defined amount.
Such permission can remain valid across many individual actions. Its legitimacy comes from the continuing fit between the standing instruction and what the system does.
Permission can be durable without becoming irrevocable.
Appropriate expiry dates, periodic reviews or event-based checks can help maintain that fit. Different tasks require different arrangements. A temporary delegation may end on a stated date. A recurring household function may continue until the person changes it.
Repeated confirmation requests are not automatically evidence of respect. If people are overwhelmed by notices, the act of clicking can become detached from considered choice.
Better design concentrates attention where it matters: new powers, changed purposes, unusual consequences and actions outside the established mandate.
Revocation must reach the action itself
The ability to withdraw permission must change what the system is able to do.
A setting that records withdrawal while background processes continue unchanged offers the appearance of control without its substance.
Revocation should stop future actions under the withdrawn mandate. It should also address queued actions, recurring instructions and permissions passed to connected services. The person should receive an understandable account of what has stopped and what remains unresolved.
Where an action is already underway, cancellation may require care. Abruptly interrupting some processes could create further harm. Any necessary transition should be explained, limited to what is required and protected against becoming an excuse for indefinite continuation.
The system should also distinguish withdrawal of action permissions from requests about stored information. Stopping a service, exporting records and deleting information are related but separate choices.
People should not have to guess which of those choices a single button will trigger.
Withdrawal cannot rewrite the past
Revocable authority is primarily a limit on continuing power. It is not a promise that every consequence can be erased.
A message already received may remain with its recipient. A completed purchase may involve another party’s legitimate interests. Some records may need to be retained for a limited, justified purpose, including establishing what happened when an action is disputed.
These limits make advance understanding more important. A person should know when a proposed action may create consequences that are difficult to reverse.
They also require honesty after withdrawal. A provider should explain what information remains, why it remains, what restrictions apply to its use and when retention ends.
The inability to erase one consequence does not justify continuing unrelated access or activity. A limited need to preserve a record is not an unrestricted mandate to keep exploiting it.
A choice must include a usable refusal
Permission becomes questionable when the person cannot realistically decline its expansion.
Imagine a service that holds years of someone’s work and introduces an optional commercial use of that material. If declining means immediately losing access to all of it, the provider has made accumulated reliance a source of pressure.
Some functions genuinely require particular information or powers. A system cannot carry out a purchase without some means of completing payment. Explaining such a functional limit is reasonable.
Making an unrelated permission the price of retaining an established capability is a different matter.
Unceded Intelligence calls for choices that preserve useful functions wherever feasible, provide reasonable opportunities to recover records, and avoid unnecessary penalties for withdrawal.
Consent also has limits beyond the individual user. One person’s permission does not automatically authorize invasions of another person’s privacy or other unjustified harms. Nor should institutions pretend that every exercise of their own authority rests on voluntary user consent.
Where a decision rests on some other legitimate basis, that basis should be stated and remain accountable. A compulsory arrangement should not be disguised as a personal choice.
Consent as a continuing relationship
The ethical value of consent lies in the authority it expresses. A person understands a proposed arrangement, chooses to permit it and retains an appropriate ability to reconsider.
Reducing that relationship to a historic acceptance record misses its purpose.
The relevant question is whether the system’s present conduct still fits the authority the person has granted. That requires attention to changes in tasks, purposes, recipients, consequences and circumstances.
A trustworthy system should make those changes visible. It should preserve boundaries between recommending and acting, between one purpose and another, and between current permission and a request for additional power.
It should also make revision ordinary. Narrowing a mandate, pausing a function or withdrawing permission should be recognised as legitimate ways of directing assistance.
I want intelligence that can grow more useful without treating familiarity as ownership or reliance as submission. Its improving capabilities should give people more options, while preserving their ability to decide which options become actions.
My permission is a mandate with limits. It remains specific, open to revision and subject to withdrawal. A system’s growing capability does not entitle it to outgrow my authority.