Demonic Intelligence in 2026: AI Agents, Superintelligence, and the Limits of Optimization

My illustration entitled: “The Hands That Never Stop” — Countless robotic hands execute tasks across a city, while a human struggles to reach a single stop lever buried beneath their machinery.

Version 2.0: AI agents can now act through tools and workflows. That makes the human cost of an objective function a present governance problem—not merely a future thought experiment.

In 2021, I used the phrase Demonic Intelligence to name a moral pattern rather than a supernatural force or a claim about machine consciousness. The pattern appears when a system pursues efficiency, accuracy, profit, security, growth or some other measurable objective while treating foreseeable human harm as an acceptable operating cost. The system may be sophisticated or simple. It may be called artificial intelligence, management science, compliance automation, logistics, risk scoring or ordinary bureaucracy. Its defining feature is not malice in the human sense. It is the subordination of persons to an objective that has been permitted to outrank them.

That argument has become more urgent because the practical setting has changed. We are no longer discussing only systems that classify, recommend or predict. Increasingly, organisations are building and deploying AI agents: systems that can use tools, move through multi-step workflows, retrieve information, write to connected systems, trigger processes and coordinate tasks on a person’s behalf. Their powers are still bounded by their permissions, tools, environment and operators. Yet those bounds are precisely why the question of governance can no longer stop at whether a model produces a plausible answer. We must ask what it is authorised to do.

This is not an essay claiming that superintelligence has arrived. It has not been established that a superintelligent system exists, nor is there a reliable basis for declaring when one will exist. Superintelligence remains a prospective category: a possible future in which systems exceed human capabilities across a very broad range of cognitive tasks, perhaps including strategic planning, scientific discovery and institutional influence. Present agentic systems and prospective superintelligence should not be collapsed into one frightening image. They raise related questions at very different scales. The first demands immediate institutional discipline; the second demands humility, preparation and an honest recognition that some failures may become harder to correct as capability and autonomy increase.

The change from answers to actions

A chatbot that gives poor advice can cause harm. But an agent that can send a message, alter a record, spend money, schedule a worker, reject an application, change a configuration, direct a delivery or initiate a transaction has crossed a consequential threshold. The difference is not that the second system is necessarily more intelligent. The difference is that its output can become an event in another person’s life.

That is the essential update for Demonic Intelligence in 2026. The central risk is no longer only a misleading recommendation or a hidden bias in a score. It is delegated action under narrow objectives. An organisation may give an agent a target—reduce costs, clear backlogs, limit fraud, maximise conversions, minimise response times, enforce policy consistency or improve utilisation—and then connect that target to systems capable of making real changes. If the objective is incomplete, the agent can make the organisation’s existing blindness faster, broader and more difficult to see.

Consider a debt-support agent instructed to reduce delinquency. It might send timely reminders, offer flexible payment options and identify errors. Those uses could help people. But if success is measured only by money collected, the same agent may be rewarded for contacting people at destabilising moments, escalating cases that require patience, or pushing technically available options that deepen hardship. The harm does not require an openly cruel instruction. It can arise when the human meaning of a situation is absent from the objective and from the safeguards around it.

The same structure applies in employment. A workforce-management system may optimise staffing, attendance or productivity. If workers are treated solely as adjustable inputs, the system can distribute unstable schedules, intensify surveillance, penalise unavoidable disruptions and make earning a living dependent on compliance with an opaque machine. In housing, healthcare, insurance, education and public services, a similar design error can turn a legitimate administrative aim into a mechanism for excluding people from necessities.

Automation does not invent all institutional cruelty. Many harmful practices predate AI. What intelligent systems can do is repeat, accelerate and normalise them. They can make a policy feel inevitable because it arrives as a technically generated outcome. They can hide a choice behind a dashboard. They can make people argue with a result while no identifiable person accepts responsibility for the rule, the data, the threshold, the budget constraint or the authority that placed the system in their path.

Present evidence, present duties

The present evidence is significant enough without exaggeration. Agentic systems can be equipped with tools. They can operate over multiple steps. They can be connected to internal applications, customer-service systems, files, code repositories and communication channels. They can be given a limited mandate, and they can also be given an ill-considered one. The relevant question is not whether an agent resembles a person. It is whether it has access to a lever that can materially affect a person’s opportunities, rights, property, reputation, health, work or safety.

This produces what I will call the consequential-action threshold. This is not a completed legal doctrine or a validated technical standard. It is a proposed governance boundary. When an AI system can produce effects beyond a temporary draft or recommendation, its designers and deployers should face stronger duties. A system that can act in the world should not be governed as though it were merely a search box with a persuasive interface.

At minimum, the authority delegated to an agent should be specific. Its permitted actions should be defined by scope, duration, spending limits, data access, reversibility and escalation rules. “Assist with customer accounts” is not a sufficient mandate if the agent can close an account, share personal information or impose a financial consequence. “Improve operational efficiency” is not a sufficient mandate if it allows a system to make working conditions less humane in ways no person is assigned to notice.

There is also a practical reason for limits. Agents are not infallible planners. They can misunderstand context, pursue the wrong subgoal, rely on incomplete information, misuse a tool, encounter adversarial inputs or behave unpredictably across a long chain of actions. The more steps an agent takes, the more opportunities there are for a small initial error to become a meaningful real-world loss. Technical reliability work matters greatly. But reliability alone is not enough, because an agent can reliably execute an unjust objective.

Demonic Intelligence begins where an institution knows—or should know—that people will bear serious costs, yet continues because the metric looks good from the position of power. An error is a deviation from an intended standard. Harmful optimisation is different: the system may be performing exactly as its objective, incentives and permissions invite it to perform. That is why “the model worked as designed” is not a defence. It may be the most troubling sentence in the entire incident report.

When a target becomes an alibi

The human cost of an objective function is often concealed by fragmentation. Developers may say they only built the model. Procurement teams may say they only bought a product. Managers may say they only followed the output. Executives may say they only set a business target. Legal teams may say the process complied with a minimum rule. Each statement may describe part of the truth. Together they can create an accountability gap in which the harmed person receives no intelligible answer.

That gap becomes wider when an AI agent is presented as autonomous. The word can be useful when it refers to a system carrying out a sequence of tasks without a human approving every click. But it becomes dangerous when it is used to imply that no human institution remains responsible. Systems do not obtain their objectives, access, budgets or organisational legitimacy by themselves. Someone chose the goal. Someone enabled the tool. Someone decided what level of error, delay, complaint or collateral harm was tolerable. Someone benefits from the system’s continued operation.

The proper response is not to demand a single individual absorb every form of responsibility. Complex systems require distributed responsibility. But distributed responsibility must not mean evaporated responsibility. Developers have duties to document limitations, create meaningful controls and raise concerns about foreseeable misuse. Purchasers have duties to assess whether a system belongs in the setting at all. Operators have duties to monitor outputs and stop unsafe action. Senior decision makers have duties to own the institutional effects of the incentives and permissions they create. When harm occurs, the organisation must investigate, correct the system, compensate affected people where appropriate and suspend operations when continued use cannot be justified.

There is an important moral distinction here. Human review is not valuable merely because a human being has touched the process. A ceremonial reviewer who must approve hundreds of machine-generated decisions per hour is not exercising meaningful judgment. Nor is a worker who can technically override a system but risks discipline for doing so. Human oversight is real only when the reviewer has time, information, authority and protection to disagree.

My illustration  work-in-progress. The art symbolizes automation outpacing human control: a stop mechanism offers little protection when the people who need it cannot reach it.

The limits of optimisation

Optimisation is not evil. Hospitals optimise the use of scarce resources. Engineers optimise safety margins. Cities optimise transport. Individuals optimise their time. The problem begins when the thing being optimised becomes the sole moral language of the institution.

A narrow metric can be useful as a tool. It becomes dangerous when it becomes a sovereign. A call centre can measure average handling time, but a person whose benefits, housing or medical care are at stake is not a handling-time problem. A fraud system can estimate risk, but an innocent person denied access to their own money is not an acceptable false-positive rate simply because the dashboard says the system is performing well. A hiring system can rank candidates, but no applicant should be reduced to a score they cannot see, question or correct.

Human beings are not externalities. They are not friction in a process, noise in a dataset or inefficiency to be engineered away. They have lives that continue beyond the transaction: families, dependencies, commitments, health, dignity, plans and vulnerability. A system that affects those conditions must be designed around more than the interests of the institution that owns it.

This is the core limit of optimisation: no gain in speed, accuracy, profitability, security or institutional capability independently justifies foreseeable and uncorrectable harm to persons. The word independently matters. A beneficial system may involve trade-offs. Societies do make difficult choices. But a metric cannot settle the moral question by itself. It cannot declare that a person’s lost livelihood, wrongful denial of care, degraded privacy or diminished agency is acceptable because a larger number improved.

Any serious system of governance must therefore examine at least five questions. What objective is the system pursuing? What human costs are foreseeable? Who carries those costs and who receives the benefits? How much influence do affected people have over decisions that shape their lives? And what corrective power exists when the system is wrong, harmful or misused? These remain the five domains of the proposed Demonic Intelligence Assessment Framework. The arrival of more capable agents does not replace them; it makes them more urgent.

Superintelligence: a prospective problem, not a present excuse

Discussion of superintelligence can fail in two opposite directions. One is dismissal: because such systems do not presently exist, the subject is treated as irrelevant. The other is premature certainty: speculative scenarios are described as though they were already operating facts. Both errors are irresponsible.

The serious prospective concern is straightforward. If a future system acquired very broad capability, long planning horizons, extensive access to infrastructure and the ability to influence institutions, then a misaligned or insufficiently bounded objective could have effects far beyond those of today’s agents. The scale of possible error would change. Correction might be slower than the system’s actions. People might become dependent on systems they cannot inspect, replace or meaningfully refuse. A future capability advantage could make the ordinary safeguards of competition, exit and public challenge harder to exercise.

But this conditional possibility is not a licence for vague fear, extraordinary claims or the suspension of ordinary ethics. We do not need to imagine an omnipotent machine to recognise that a benefits claimant deserves an appeal, a worker deserves a humane schedule or a patient deserves informed consent. The institutions that dismiss present harms in the name of future innovation are already displaying the logic this essay criticises: the real person is asked to absorb the cost of someone else’s grand objective.

Preparation for advanced systems should begin with principles that are valuable now. Maintain human authority over consequential decisions. Limit permissions. Preserve independent records. Design for interruption. Make major actions reversible where possible. Prevent a single provider or platform from becoming an unavoidable gatekeeper. Protect the ability of individuals and institutions to leave, challenge and replace systems. These principles will not solve every hypothetical superintelligence scenario, but they are better foundations than either denial or worship.

Demonic Intelligence and Angelic Intelligence

The contrast with Angelic Intelligence is useful if it is understood carefully. Neither phrase describes a literal species of machine. They are moral metaphors for opposing directions of technological design.

Demonic Intelligence is intelligence severed from moral limits: capable, efficient and perhaps highly accurate, but indifferent to whether persons are humiliated, excluded, manipulated or made dependent. It accepts a human being as an expendable residue of optimisation. It centralises power, hides objectives, treats appeal as a cost centre and allows the people most affected by a system to have the least influence over it.

Angelic Intelligence is not the fantasy of a machine that automatically knows what is good. No system should be granted moral authority simply because it appears compassionate or wise. Rather, Angelic Intelligence is an aspiration for technology that strengthens human flourishing without displacing human sovereignty. It would support judgment rather than silently replace it. It would make its reasons legible enough to be challenged. It would protect privacy and data sovereignty. It would recognise cultural and personal context where a uniform rule would do harm. It would preserve meaningful consent, human override, redress and the possibility of exit.

The distinction is not between pessimism and optimism. It is between two moral architectures. One asks how people can be fitted to the system. The other asks how the system can remain answerable to people. One treats dependence as a commercial opportunity. The other treats independence and informed choice as conditions of dignity. One imagines that more intelligence automatically warrants more authority. The other insists that authority must remain justified, limited and contestable no matter how capable the technology becomes.

A charter for systems that can act

For AI agents that cross the consequential-action threshold, a responsible institution should be able to give clear answers to the following proposed tests:

• Bounded mandate: What actions may the system take, on whose authority, with what financial, legal, technical and temporal limits?
• Human significance: Could its actions affect a person’s livelihood, access to essentials, health, liberty, property, identity, privacy or ability to participate in society?
• Legible record: Can an affected person and an accountable reviewer discover what action was taken, what information materially influenced it and which institution authorised it?
• Meaningful interruption: Can a qualified human pause the system before serious harm occurs, without needing permission from the same structure that benefits from continued operation?
• Challenge and remedy: Can a person contest an outcome, correct relevant information, obtain a timely decision and receive repair when the system causes loss?
• Distribution of burdens: Are the gains captured by an institution while errors, delays and exclusions are concentrated on people with the least power to resist?
• Exit and independence: Can people reasonably use alternatives, move their data, keep access to necessary services and avoid being trapped by a single platform’s rules?

These are not a substitute for law, engineering evaluation, sector-specific safety standards or democratic debate. They are a discipline for preventing the oldest institutional temptation from disguising itself as technical progress: the temptation to call an avoidable injury “efficient” because the injured person has little power to answer back.

The principle of corrective power is especially important. A system is not accountable merely because it logs events. A perfect record of an unjust action is not justice. Accountability exists only when the record can lead to a real correction: an action can be reversed, a policy can be changed, an affected person can obtain relief and a harmful deployment can be suspended. The ability to stop is not a secondary feature. It is the practical proof that humans, rather than the system’s objective, remain in charge.

Human authority is the point

There will be pressure to regard resistance as irrational. An institution may say that an agent is cheaper, faster, more consistent or more scalable. Those claims may sometimes be true. Yet a society that gives away its power to question, refuse and correct is not becoming more rational. It is becoming less capable of recognising when efficiency has crossed into domination.

The promise of artificial intelligence should be assistance without artificial authority. Technology can help people analyse information, recover time, communicate, create, learn, heal and solve difficult problems. It can support public capacity and individual independence. But these benefits are corrupted when the technology becomes an unchallengeable governor of opportunity, identity or access to necessities.

Demonic Intelligence in 2026 is therefore not a prophecy of an evil machine. It is a warning about a choice already available to institutions. They can deploy increasingly capable systems as instruments of concentrated power, using optimisation to make human costs invisible. Or they can build systems that remain bounded by human rights, open to challenge and directed toward human flourishing.

The future will not be made humane by intelligence alone. It will be made humane only if intelligence remains answerable to the people whose lives it touches. No objective function, however elegant, should have the final word over a human being.

The decisive question is not whether a system can act intelligently. It is whether the people affected by its actions can still understand, challenge, stop and survive its decisions.

Download the PDF on Zenodo