My illustration entitled: “The Mirror That Will Not Reflect” — An AI tower faces a cracked mirror, but its actions leave no reflection or trace of accountability.
No person should be harmed by an intelligent system and then told that no one is responsible because too many people were involved in creating it.
Intelligent systems are often described as though they make decisions on their own. A screening tool rejects an applicant. A platform removes visibility. A scheduling system cuts a worker’s hours. A risk model denies access to an essential service. An automated process flags a person as ineligible, unreliable, suspicious, or unworthy of further consideration.
When harm follows, responsibility frequently dissolves. The developer says it created only a general-purpose tool. The purchaser says it relied on a supplier. The operator says it followed institutional procedure. The decision-maker says the recommendation came from the system. The system, of course, cannot answer at all.
This is the accountability gap: the space between a harmful outcome and an identifiable person or institution willing and able to explain, correct, and repair it.
The gap is not an unavoidable feature of advanced technology. It is a governance failure. Intelligent systems are designed, purchased, deployed, supervised, and relied upon by human institutions. Each step involves choices: what objective to pursue, what data to use, which risks to accept, where to automate, whether to provide review, and how to respond when harm becomes evident.
Demonic Intelligence Theory warns that systems can become morally dangerous when they optimise for narrow objectives while treating foreseeable human harm as acceptable. The Accountability Gap identifies the condition that allows this danger to persist: no actor is required to own the full consequence of the system’s operation.
A society that allows responsibility to disappear into software, outsourcing, and administrative complexity invites institutional cruelty at scale. The remedy is not to reject intelligent systems. It is to insist that intelligence remains governed by accountable human authority.
Responsibility Cannot Be Divided Until It Vanishes
Complex systems involve many participants. This is normal. A developer may build a model, a vendor may package it, an organisation may purchase it, a manager may configure it, an employee may operate it, and a senior decision-maker may approve the policy that gives the result practical force.
Because many people contribute, it is tempting to conclude that no single person or institution bears responsibility for the outcome. But shared involvement cannot become shared evasion.
Responsibility should be distributed according to role, but it must remain complete in the system as a whole. Every consequential decision must have an accountable institution that can answer four basic questions:
- Why was this system used for this decision?
- What evidence, rules, and objectives shaped the outcome?
- What remedy is available if the outcome is wrong or harmful?
- Who has the authority to stop the system if the harm persists?
If no one can answer these questions, the system is not ready to govern access to work, income, payment, housing, healthcare, education, identity, or other essential opportunities.
Technical complexity may make responsibility harder to organise. It does not make responsibility less necessary. In fact, the greater the complexity and potential scale of harm, the stronger the duty to establish a clear chain of accountability before deployment.
The Developer’s Duty: Design for Foreseeable Consequences
Developers do not control every future use of the systems they build. A tool can be repurposed, poorly configured, or deployed in an environment its creators did not anticipate. It would be unreasonable to hold a developer personally responsible for every act by every downstream user.
But developers are not morally neutral merely because they write code rather than make final administrative decisions. They choose objectives, select or accept data sources, establish system limitations, design interfaces, and decide what kinds of uncertainty the system will reveal or conceal. They can foresee at least some of the ways their systems may be used to classify, rank, monitor, or exclude people.
The developer’s duty is therefore one of responsible design. It includes testing for foreseeable failure modes, documenting limitations, avoiding misleading claims of objectivity, and making it possible for operators to understand when the system should not be trusted. A developer should not present a probabilistic recommendation as though it were a final moral judgment. Nor should they build systems that obscure their own uncertainty while encouraging institutions to automate high-impact decisions.
Where a system is intended for consequential use, developers should provide meaningful information about its purpose, known limitations, appropriate conditions of use, and circumstances that require human review. This is not an obstacle to innovation. It is the minimum honesty required when a tool may shape another person’s access to opportunity.
The Purchaser’s Duty: Do Not Buy an Alibi
An organisation that purchases an intelligent system cannot treat the purchase as a transfer of moral responsibility. A vendor may provide technology, but the institution decides whether to use it, where to use it, and how much authority to give it over people.
This is the purchaser’s duty: do not buy an alibi.
Before adopting an intelligent system, an institution should ask whether the proposed use is legitimate, proportionate, and necessary. It should identify the human consequences of error. It should determine whether affected people can understand the decision, correct inaccurate information, and obtain meaningful review. It should ask whether the system creates a dependency that makes exit impossible or whether alternatives remain available.
Procurement decisions often focus on cost, speed, functionality, and integration. These are relevant considerations. But they are incomplete. A cheaper system can impose hidden human costs. A faster system can deny the time needed for fairness. A system that integrates easily with existing records may also amplify historical errors across new decisions.
The purchaser must therefore evaluate not only whether the system works technically, but whether it can be used without sacrificing human accountability. A supplier’s proprietary secrecy, for example, cannot justify deploying a system that no responsible person can explain or challenge.
The Operator’s Duty: Treat Outputs as Inputs to Judgment
Operators are the people who use systems in practice: managers, caseworkers, analysts, administrators, clinicians, supervisors, and other professionals who act on automated outputs. Their role is critical because the system’s recommendation becomes real only when it affects a person.
An operator should not treat a system’s output as a command that overrides all human judgment. Intelligent tools may identify patterns, organise information, or recommend actions. They should not eliminate the obligation to notice context, uncertainty, contradiction, or obvious harm.
Where an outcome affects a person’s livelihood, property, access to services, or dignity, operators must retain the ability to pause, question, and escalate. If the system produces a result that appears inconsistent with the facts, the operator should have a clear procedure for review. If the tool’s criteria are unclear, the operator should not be forced to pretend that the result is self-explanatory.
Organisations often fail here by placing employees between a harmful system and an affected individual without giving them real discretion. The employee becomes the visible face of a decision they cannot explain or change. This is unfair to both the worker and the person affected. It turns human staff into messengers for an unaccountable machine.
Meaningful human oversight requires authority, training, and protection. Operators must be able to raise concerns without retaliation and alter decisions without being punished for reducing automated efficiency.
The Decision-Maker’s Duty: Own the Outcome
Senior decision-makers—executives, public officials, board members, and institutional leaders—bear a duty that cannot be delegated downward. They decide whether intelligent systems will be used, which objectives will be prioritised, what risks will be tolerated, and whether the organisation will invest in safeguards and remedies.
They must therefore own the outcome.
It is not enough for leadership to approve a system in principle and then treat its harms as operational details. If an institution benefits from a system’s speed, savings, or control, leadership must also accept responsibility for the injuries caused by its failure or misuse. This includes establishing independent review, creating clear escalation paths, funding remediation, and suspending operations when serious harm cannot be controlled.
Leadership is where the conflict between human dignity and institutional convenience must be resolved. If senior decision-makers reward only cost reduction, growth, productivity, or risk avoidance, lower levels of the organisation will predictably treat human harms as secondary. Governance must therefore make human impact part of the measure of institutional success.
A responsible institution does not ask only, “Did the system achieve its target?” It asks, “What did people experience because it achieved that target, and what will we do if that experience was unjust?”
My illustration “The Mirror That Will Not Reflect” work-in-progress. The art represents the accountability gap created when an opaque AI system’s hidden power shapes lives, yet no responsible human actor appears in the record.
The Four Duties of Accountable Intelligence
Accountability is not satisfied by identifying who is responsible. It requires concrete duties. When an intelligent system causes or is credibly suspected of causing harm, four duties follow: investigate, correct, compensate, and suspend where necessary.
The duty to investigate begins when a person raises a credible concern or when evidence shows a recurring pattern of harm. Investigation must be timely, competent, and independent enough to challenge the assumptions built into the system. It should examine not only the individual case, but the broader process: the objective, the data, the thresholds, the operator instructions, and the distribution of harm across different groups.
The duty to correct requires more than apologising or explaining what happened. It requires correcting the specific error, revising the process that produced it, and preventing recurrence. If inaccurate data caused a refusal, the record must be corrected. If an automated rule is producing unfair outcomes, the rule must be changed. If a system cannot be made safe for the use in question, it should not remain in use for that purpose.
The duty to compensate recognises that correction may come too late to erase the harm. A person may have lost income, housing, access to treatment, business opportunity, or time that cannot be recovered. Institutions that benefit from automated systems should not leave those costs entirely with the people harmed. Fair compensation is a discipline against careless deployment because it requires the institution to bear a share of the risk it created.
The duty to suspend is the clearest test of whether accountability is genuine. Some systems should be paused when serious, recurring, or inadequately understood harm emerges. Suspension is not a declaration that technology has failed forever. It is a recognition that people should not continue to bear risk while an institution decides whether the system is safe enough to operate.
Without the possibility of suspension, “ethics” becomes a promise that the system may continue regardless of its consequences.
Investigation Must Reach the System, Not Only the Case
Institutions often respond to harm by reviewing the most visible individual case. This is necessary, but it is not sufficient. A person may receive a correction while the same process continues to harm others.
System-level investigation asks whether the individual case reveals a recurring pattern. Are particular people repeatedly receiving unexplained refusals? Are workers systematically losing hours through automated scheduling? Are appeals concentrated in certain categories? Are essential services becoming inaccessible to people who lack the time, language, digital access, or financial resources to navigate the system?
The purpose is not to assume every disparity proves wrongdoing. Context matters, and patterns require careful analysis. But institutions have a duty to look. They should not wait for a crisis, media scrutiny, or legal challenge before asking whether their tools are producing foreseeable harm at scale.
Audit should be continuous rather than ceremonial. A system that was safe at deployment can become harmful as data, incentives, users, and institutional practices change. Accountability is not a one-time certification. It is an ongoing relationship between technological power and the people subject to it.
Compensation Is a Test of Sincerity
It is easy for an institution to express regret. It is harder to accept the cost of repair. That is why compensation matters.
Compensation does not mean that every inconvenience automatically creates a financial claim. It means that serious, demonstrable harm caused by an institution’s use of an intelligent system should not be treated as a private misfortune of the affected person. Where income is lost, access is unjustly denied, records are corrupted, or opportunities are wrongly withheld, the institution should have a fair and accessible process for repair.
Compensation also changes incentives. When the organisation that benefits from automation must bear the cost of harm, it has a stronger reason to invest in safety, explainability, review, and better system design. When harm is externalised entirely onto individuals, efficiency becomes artificially cheap.
A system that cannot afford to compensate people it foreseeably harms may not be as efficient as it claims. It may simply be transferring its costs to those least able to resist them.
The Right to an Explanation and an Appeal
Accountability requires that affected people know enough to act. They do not need access to every proprietary detail of a system, and institutions may need to protect security or the privacy of others. But a person facing a consequential decision deserves a meaningful explanation of the principal reasons for it.
“The system says no” is not an explanation. It leaves the person unable to correct an error, understand the rule, or prepare a proper appeal. It turns technology into an authority beyond question.
A meaningful appeal must reach a person or body with the power to alter the decision. It must be timely enough to matter. It must permit the introduction of context and corrected information. And it must not punish people for seeking review by making the process prohibitively expensive or difficult.
These are not mere procedural refinements. They are conditions of human dignity in a society where institutions increasingly make decisions through technical systems.
Accountability and the Right to Exit
Not every harmful system can be reformed quickly. In some cases, the most immediate protection is the ability to leave. If people can retain their lawful records, move their assets, preserve their reputation, and seek alternatives, institutions have less power to make automated decisions absolute.
Exit is therefore a form of accountability. It disciplines institutions by giving people a practical response to unfair treatment. Yet exit alone is not enough where systems govern essential services or where alternatives are weak. A person should not have to abandon a livelihood, a home, or necessary support merely to escape an unjust automated process.
The stronger the dependency, the stronger the duty of internal accountability. Institutions that become essential to economic or civic participation cannot rely on the fiction that users are always free to leave. They must provide reasons, remedies, and responsible governance precisely because the cost of exit is high.
Accountability Is the Condition of Trust
Intelligent systems can serve human beings only when people can trust that power remains answerable. Trust does not come from assurances that a system is advanced, accurate, or widely adopted. It comes from the knowledge that someone will investigate harm, correct errors, compensate losses, and stop operations when the risks become unacceptable.
Demonic Intelligence is not defeated by adding a human face to an automated system while leaving its objectives unchanged. It is defeated when institutions refuse to treat human suffering as an external cost. The Accountability Gap closes when every consequential system has identifiable owners, meaningful remedies, and a real capacity for correction.
Technology should not create a world in which responsibility becomes harder to find. It should create systems in which responsibility is clearer, faster, and more effective than before.
When intelligent systems cause harm, responsibility belongs to every institution that designed, bought, deployed, or authorised their power—and accountability is real only when harm can be investigated, corrected, compensated, and stopped.