My illustration entitled: “The Factory of Decisions” — Conveyor belts process people into categories while an algorithmic engine stamps outcomes without seeing their faces.
An objective function is dangerous not because it has a goal, but because it can make everything outside that goal disappear from the system’s moral field of view.
Every organised system pursues objectives. A company seeks sustainability and profit. A hospital seeks recovery and safety. A government agency seeks compliance, order, and public service. A platform seeks growth, reliability, and engagement. An artificial intelligence system is trained to maximise, minimise, classify, predict, or recommend according to a defined target.
Objectives are necessary. Without them, institutions drift, resources are wasted, and action becomes incoherent. The problem begins when an objective becomes too narrow for the human reality it governs.
A system can improve exactly what it has been instructed to improve while making life worse for the people affected by it. It can lower costs by shifting burden onto workers. It can increase engagement by exploiting attention and anxiety. It can reduce institutional risk by excluding people who need help most. It can increase accuracy according to a chosen model while reinforcing an unjust definition of success. It can optimise efficiency while making appeal, consent, and exit practically impossible.
This is the human cost of the objective function.
Demonic Intelligence Theory describes the moral danger that appears when systems pursue narrow objectives through unequal power while treating foreseeable harm as an external cost. The word “demonic” is a metaphor. It does not claim that software, institutions, or economic systems possess supernatural motives. It names a recognisable pattern: intelligence that performs its assigned task with disciplined effectiveness while becoming indifferent to the people sacrificed along the way.
The theory does not reject goals, calculation, markets, or artificial intelligence. It asks a prior question: what happens when a system’s success is measured without adequately accounting for the human beings who bear its costs?
The Objective Function Is Never the Whole World
An objective function is a formal way of stating what a system should optimise. It may be simple, such as minimising delivery time. It may be complex, such as balancing financial risk, customer retention, and regulatory compliance. In artificial intelligence, an objective function may determine what counts as a desirable result during training or operation.
But every objective function is selective. It turns part of reality into a target and leaves other parts outside the calculation.
A delivery system may measure speed while ignoring the pressure placed on workers. A platform may measure engagement while ignoring whether users become more dependent or distressed. A hiring system may measure predicted performance while ignoring whether its data reflects past exclusion. A public-service system may measure reduced expenditure while ignoring the hardship produced when legitimate applicants cannot navigate its procedures.
The system does not necessarily deny that these harms exist. More often, the harms are simply absent from the measure that determines success. They become somebody else’s problem: the worker’s exhaustion, the family’s uncertainty, the small business’s loss of visibility, the individual’s inability to appeal, the community’s erosion of trust.
That is why narrow optimisation can be so dangerous. It does not need to be openly cruel. It only needs to define success in a way that makes human cost invisible or secondary.
External Costs Are Often Human Costs
An external cost is a burden created by an activity but not fully borne by the party that benefits from it. In economic language, this is often described as an externality. But the term can sound abstract. In reality, the external cost is usually carried by someone.
It may be carried by a worker expected to remain constantly available. It may be carried by a customer whose data is collected and used beyond what they understood. It may be carried by a family denied a service because an automated classification does not recognise their circumstances. It may be carried by a community whose local businesses disappear because one dominant platform captures the market while controlling access to customers.
External costs become morally serious when the system is designed so that those who benefit can avoid the consequences while those with less power must absorb them. The institution receives the efficiency gain; the individual receives the delay, the exclusion, the lost income, or the loss of privacy.
This asymmetry is not always intentional at the beginning. A new process may be introduced to solve a legitimate problem. Yet once the burdens are known, the institution faces a choice. It can redesign the system, compensate those affected, provide remedies, or create alternatives. Or it can continue because the cost is not appearing on its own balance sheet or performance dashboard.
The latter choice is where harmful optimisation becomes persistent. The system succeeds according to its internal measure precisely because the people harmed do not have enough power to make their suffering count.
Unequal Power Distorts the Meaning of Consent
Supporters of a harmful system may argue that participation is voluntary. A worker accepted the job. A customer accepted the terms. A user opened the account. A merchant joined the platform. But consent is not meaningful merely because a person clicked “agree” or remained within an arrangement.
Consent depends on practical alternatives. A person who can understand the terms, negotiate conditions, decline the service, and continue elsewhere has meaningful choice. A person who faces opaque rules, no viable competitor, loss of accumulated reputation, or the threat of economic exclusion has a much narrower form of choice.
Unequal power turns apparent consent into conditional compliance. The institution can impose terms because the individual lacks the resources, information, or exit routes necessary to refuse them. The system may describe the relationship as voluntary while relying on the fact that departure is too costly to be realistic.
This condition is especially important in digital systems. A platform can become so central to work, payments, communication, or discovery that users cannot leave without losing the economic value they have built. A service can claim to offer access while retaining unilateral power to change the rules of access. A ranking system can present itself as neutral while determining which people are effectively visible and which are not.
Demonic Intelligence Theory treats these power imbalances as morally relevant. A system should not be considered legitimate merely because people nominally participate. It must also be judged by whether they retain the practical capacity to understand, challenge, and exit.
How Harmful Outcomes Become Normal
Many harmful systems persist not because no one notices the harm, but because recognition does not produce correction. The harms become normalised. They are described as unfortunate, unavoidable, isolated, or simply the price of progress.
Several conditions make this persistence more likely.
First, the benefits are concentrated and visible while the harms are dispersed and hidden. An institution can point to efficiency, growth, savings, or improved metrics. The people harmed experience their losses individually, often without knowing that others are facing the same pattern.
Second, responsibility is fragmented. One team builds the system, another owns the data, another manages policy, another handles complaints, and another benefits from the result. Each participant can say that the final outcome lies outside their authority.
Third, the harmed lack a language of challenge. If people cannot understand why they were denied, ranked, restricted, or excluded, they cannot easily show that a pattern exists. They may internalise the failure as personal rather than recognising it as structural.
Fourth, remedies are costly. The institution may know that a system creates avoidable harm but conclude that correction is too expensive, too slow, or too disruptive to the business model. The cost of repair is measured; the cost imposed on people is not.
Fifth, alternatives are weak. If users, workers, or communities cannot leave, the institution has little incentive to change. Dependency insulates harmful systems from feedback.
Under these conditions, recognition becomes performative. The institution acknowledges concerns, issues principles, perhaps creates an ethics committee, and then continues the same process because the objective function has not changed.
When Efficiency Becomes a Moral Evasion
Efficiency is often presented as a neutral virtue. It means doing more with less, reducing waste, and improving performance. But efficiency becomes morally evasive when it is used to avoid asking what has been reduced and who has been made to bear the reduction.
Cutting waiting time may be good. Cutting the time available to hear a person’s case may not be. Automating routine decisions may be good. Automating the removal of human judgment from high-impact decisions may not be. Reducing administrative cost may be good. Reducing it by making people unable to correct mistakes may not be.
The phrase “the system is more efficient” should therefore never end the moral discussion. It should begin a deeper one. Efficient for whom? Efficient at what? Efficient compared with which alternative? Who bears the risk when the system is wrong? Which human goods have been excluded from the calculation?
A system that cannot answer these questions is not simply incomplete. It may be structurally disposed to harm people while congratulating itself on performance.
My illustration “The Factory of Decisions” work-in-progress. The art represents how automated systems can reduce human lives to impersonal categories, sacrificing dignity and judgment for efficient outcomes.
The Difference Between Error and Design
Not every harmful outcome proves that a system is governed by Demonic Intelligence. Error is part of life, and error can be corrected. The crucial distinction is between a system that makes a mistake and a system whose design makes foreseeable harm rational from its own perspective.
An error is recognised as a deviation from the system’s legitimate purpose. The institution investigates, repairs the damage, changes the process, and accepts responsibility. The harm is treated as evidence that the system failed to meet its own standards.
Harmful optimisation is different. The system continues because the harm is compatible with its standard of success. It may even depend upon the harm. The worker’s exhaustion is tolerated because output rises. The user’s dependency is tolerated because retention improves. The person’s exclusion is tolerated because risk statistics look better. The loss of privacy is tolerated because data extraction is profitable.
In these cases, the harm is not a malfunction at the edges. It is a cost built into the model.
This is why technical accuracy cannot settle moral responsibility. A system can operate without a bug and still be wrong in its purpose. The most alarming form of intelligence is not always the one that fails unpredictably. It is the one that succeeds predictably while treating human injury as an acceptable operational expense.
The Objective Function Must Be Contestable
If objectives shape outcomes, then objectives themselves must be open to challenge. This is a basic requirement of accountable power.
Institutions should not simply ask whether their systems meet performance targets. They should ask whether those targets have been defined in a way that respects human dignity and agency. They should examine whether the harms are foreseeable, whether they are disproportionately borne by people with less power, and whether affected people have a real path to explanation and remedy.
This requires more than publishing ethical principles. It requires changing incentives. A company that rewards only growth will tend to overlook harms that do not affect growth. A public institution that rewards only cost reduction will tend to overlook the human cost of inaccessible procedures. An AI system assessed only by accuracy will tend to disregard whether the classifications it produces are appropriate, proportionate, and challengeable.
Human costs must therefore be represented in governance, not merely acknowledged in public relations. The people affected by systems need routes to contest decisions. Independent review must be capable of changing outcomes. Decision-makers must be identified. Serious harm must carry institutional consequences rather than being treated as an unfortunate but acceptable side effect.
A contestable objective function is one that can be revised when it is shown to produce unacceptable results. An unchallengeable objective function is an invitation to moral blindness.
Human Sovereignty as a Constraint on Optimization
Human-Sovereignty Transhumanism provides a clear normative boundary. Technology and institutions should expand human capability while preserving the authority of the person over their body, mind, identity, data, property, and future. Progress is meaningful only when people remain subjects of it rather than becoming its raw material.
Applied to Demonic Intelligence Theory, this means that certain human interests cannot be treated as optional variables. Meaningful consent, privacy, due process, bodily integrity, the ability to contest a consequential decision, and the right to exit coercive dependence are not obstacles to be optimised away. They are safeguards against the transformation of people into system components.
This does not mean every individual preference must override every collective need. Societies require public safety, fair rules, and responsibilities to others. But collective goals must remain proportionate and accountable. They cannot be used as blank cheques for institutions to impose foreseeable harm on those unable to resist.
Human sovereignty does not reject intelligence. It gives intelligence a moral direction.
A Framework for Prevention
Preventing harmful optimisation requires discipline at the level of design, governance, and public accountability.
First, every consequential system should identify its primary objective and its likely human costs. If a system cannot name the people who may be harmed by its operation, it is not ready to govern their lives.
Second, institutions should identify whether those costs fall unequally. Harm borne by people with fewer alternatives deserves special scrutiny, because unequal power can turn a manageable burden into a form of coercion.
Third, high-impact decisions must remain explainable and reviewable. People should be able to understand the main reason for a serious refusal, restriction, classification, or loss of access. They should be able to correct inaccurate information and seek a meaningful human review.
Fourth, systems should preserve alternatives and exit. Portability, open standards, direct ownership, and competition reduce the danger that one institution’s objective function becomes a person’s entire economic horizon.
Fifth, institutions should measure repair, not only performance. How quickly are mistakes corrected? How often are appeals granted? What harms recur? What changes after warnings are raised? These are indicators of whether an organisation treats human cost as real.
Finally, people affected by systems should have a voice in their design and governance. Those who experience a system’s burdens often see harms that distant designers cannot. Their participation is not an inconvenience. It is a source of moral and practical intelligence.
The Foundational Warning
Demonic Intelligence Theory is a warning against the false comfort of narrow success. A system may be efficient, accurate, profitable, scalable, and technically advanced while still becoming morally intolerable. Its danger does not arise from intelligence alone. It arises from intelligence combined with a narrow objective, unequal power, ignored external costs, and the absence of meaningful correction.
We should not ask only whether a system works. We should ask what it works for, who pays for its success, and whether those affected retain the authority to challenge its decisions.
The human cost of the objective function must never be treated as an afterthought. Once a system can foresee that its design will harm, exclude, impoverish, manipulate, or make people dependent, it has a duty to change. Continuing unchanged is no longer mere technical optimisation. It is a moral choice.
A system becomes dangerous when it can recognise the people harmed by its success, calculate that harm as a cost, and still decide that the objective matters more than the human being.