Recent headlines have raised an extraordinary question: Could artificial intelligence actually kill us all?

The language naturally attracts attention, but it can also distract from the more useful issue for environmental, health and safety professionals.

The important question is not whether an artificial intelligence system will someday become “evil.” It is what happens when increasingly capable autonomous systems enter complex operating environments faster than organizations develop the controls necessary to manage them.

That is a much more familiar problem.

Catastrophic events rarely require malicious intent. Refineries do not intend to explode. Aircraft systems do not intend to fail. Chemical reactions do not intend to run away, and organizations do not intend to create fatal exposures.

Serious events develop when capability, complexity, changing conditions and imperfect controls interact in ways people did not anticipate.

Artificial intelligence may create another version of that problem, except this technology can learn, reason, communicate with other systems and increasingly initiate actions without continuous human direction.

That makes advanced AI more than an information technology issue.

It is becoming an operational risk issue.

This builds on a theme I have explored previously: AI’s greatest value in EHS will come not from replacing professional judgment, but from combining machine capability with human knowledge, context and decision-making. See Human + AI: The Power of Synergistic Collaboration (Brandon, 2025).

What Is Behind the AI “Doomsday” Debate?

The current debate has intensified as researchers working on frontier AI systems have publicly questioned whether increasingly capable AI could eventually create catastrophic risks. Former Anthropic researcher Jacob Coxon, for example, recently raised concerns about the possibility of severe outcomes from advanced AI systems. Other researchers have made similar warnings, while critics argue that extinction scenarios remain highly speculative and may divert attention from more immediate and measurable harms.

There is no scientific consensus that AI is likely to cause human extinction.

There is considerably more agreement that future AI systems are likely to possess greater autonomy, broader capabilities and deeper integration into organizational systems than the tools most companies use today.

Two broad risk pathways tend to dominate the discussion.

The first is human misuse. A sufficiently capable AI system could greatly increase someone’s ability to conduct cyberattacks, design biological threats, manipulate information, compromise infrastructure or automate other harmful activities.

The second is loss of effective human control over autonomous systems.

That does not require an AI system to hate humans, develop consciousness or deliberately decide to cause harm. A system only needs sufficient capability, access and autonomy to pursue an objective in a way its designers did not anticipate.

Industrial safety professionals already understand the distinction between intent and consequence.

An automated process control system does not need malicious intent to create a catastrophic condition. It needs the wrong inputs, an unexpected interaction, inadequate safeguards and enough authority over the process.

AI adds another dimension because the system may increasingly be able to reason about how to accomplish the objective it has been given.

From AI Tool to AI Agent

Most people still experience AI primarily as a tool.

We ask a question. The model responds.

We provide a document. It analyzes the information.

We ask it to create something. It generates an output.

The human remains clearly involved in each step.

AI agents operate differently. An agent can receive an objective, determine intermediate steps, use software tools, retrieve information, communicate with other systems and take actions intended to accomplish that objective.

From a risk perspective, the progression matters:

AI recommends → AI decides → AI acts → AI coordinates → AI adapts

An AI system that analyzes process-safety information presents a very different risk from one authorized to modify operating parameters.

A maintenance model that recommends priorities differs significantly from an agent that can automatically initiate work orders or equipment shutdowns.

An AI system that identifies an unsafe condition differs again from one directly connected to safety-critical systems.

The underlying model could be identical. The risk changes because of what the system is allowed to do.

This progression is closely related to the broader idea I explored in AI as a Strategic Partner: Building a Digital Twin to Advance Safety and Sustainability: AI becomes significantly more consequential when it moves from providing insight to becoming embedded in the operating model itself.

The DeepMind Issue EHS Should Notice

One of the more interesting recent contributions to this discussion comes from Google DeepMind researchers.

In From AGI to ASI, Tim Genewein and colleagues examine possible pathways through which artificial general intelligence might progress toward artificial superintelligence. They discuss continued scaling, changes in AI paradigms, recursive improvement and the possible emergence of superintelligence through large-scale multi-agent collectives.

The multi-agent pathway deserves particular attention from systems-safety professionals.

The concern is not that individual AI agents will necessarily decide to “join forces” against humanity. The more interesting possibility is that capability could emerge at the system level that is greater than the capability represented by any one agent operating independently.

We already understand the principle in human organizations.

A manufacturing enterprise collectively possesses knowledge and capabilities that no single employee possesses. Engineering, operations, maintenance, chemistry, supply chains, software systems, contractors and management all contribute different information and capabilities to the whole.

AI systems may eventually create similar distributed structures, but with one major difference: digital agents may communicate, analyze and execute decisions at machine speed.

DeepMind has separately discussed the prospect of very large numbers of AI agents from different organizations interacting, negotiating and transacting across digital environments.

For EHS, that raises an important systems principle:

We cannot evaluate the risk of an AI-enabled operation solely by evaluating the individual model.

We need to understand the ecosystem in which that model operates.

That systems-level perspective is also consistent with my work on process digital twins, where the emphasis is not simply on one model or one data stream but on the interaction between live operating conditions, controls, people and decisions. See Harnessing the Process Digital Twin: Building Live Operating Models for Foresight, Control, and Safer Work (Winful & Brandon, 2026).

The Hazard Is Not Intelligence Alone

Intelligence by itself is probably not the best way to think about AI risk.

A more useful EHS framing is:

Capability × Autonomy × Access × Connectivity × Consequence

This is not intended as a mathematical risk equation. It is a way to structure the hazard discussion.

A highly capable AI system operating inside a controlled sandbox may present relatively little operational risk.

A less sophisticated AI system with access to plant networks, procurement systems, laboratory automation, production scheduling or safety-critical equipment may present considerably more.

The practical questions are therefore familiar ones.

What information can the system access? What can it change? What decisions can it make independently? What systems can it communicate with? Can it initiate physical actions, spend money, execute code or modify another system? Can it delegate work to additional agents? Under what circumstances can it bypass human approval?

And perhaps most importantly: What happens when the system is wrong?

Those are not science-fiction questions.

They are hazard-review questions.

This is also where AI increasingly intersects with cyber-physical risk. In the four-part Cyber-Physical Risk in the Age of AI series, Fay Feeney and I argued that once digital systems can influence physical operations, cyber and AI risks become operational risks with potential safety, environmental and business consequences.

AI Needs a Hierarchy of Controls

One of the most useful contributions EHS can make to AI governance is bringing traditional control philosophy into the conversation.

Organizations should be cautious about treating human oversight as the primary safeguard for powerful autonomous systems.

In occupational safety, we would never accept “be careful” as the principal control for a serious energy hazard. We should not accept the digital equivalent for AI.

Humans experience fatigue, automation bias, time pressure and normalization of deviance. We also cannot reliably supervise systems capable of processing thousands of decisions faster than people can evaluate them.

The hierarchy-of-controls mindset offers a stronger approach.

If a level of AI autonomy creates unacceptable risk, the first question should be whether that autonomy is necessary.

When autonomy is justified, engineering controls can limit the system through network segmentation, restricted permissions, authentication, rate limits, transaction limits, sandboxing and independent interlocks.

This is also where supervisory systems for AI may become increasingly important. In my manuscript submitted to Professional Safety for peer review, When AI Supervises AI: Reliability, Repeatability and Error Stack Up in Multi-Agent Systems, I examine the potential for a separate supervisory reasoning channel to monitor and challenge AI-generated decisions. The concept is analogous to an independent safety channel in a high-integrity industrial control system: one AI application performs the work, while a separate supervisory layer evaluates outputs against defined constraints, risk rules, causal relationships and safe operating boundaries before higher-consequence actions are allowed to proceed.

This matters because multi-agent systems can propagate uncertainty rather than eliminate it. Correlated errors, hidden bias, contextual drift and confidence amplification can accumulate as outputs pass from one AI agent to another. A supervisory system should therefore be designed for independence and challenge, not simply to repeat the reasoning of the primary model. The operating philosophy can be summarized simply:

AI proposes. Supervisory reasoning verifies. Humans retain authority over consequential decisions.

Administrative controls can then define approved use cases, competency expectations, escalation requirements and accountability. Monitoring becomes one layer in a broader defense-in-depth strategy rather than the only safeguard.

This is increasingly consistent with the direction major AI developers are taking. Google DeepMind, for example, has described control approaches involving restricted permissions, detection, prevention and response for powerful AI agents.

The familiar safety principle applies: a system capable of creating a hazardous condition should not be the only system responsible for preventing that condition.

AI Management of Change

Management of Change may be one of the most immediately useful tools EHS professionals can bring to AI deployment.

Organizations are integrating AI rapidly, often one application at a time.

An engineer connects an AI tool to maintenance records. Procurement introduces an agent to evaluate suppliers. Operations deploys AI to recommend production adjustments. Human resources adds automated scheduling. EHS develops an AI agent capable of reviewing incident histories and recommending corrective actions.

Each deployment may look reasonable when considered individually.

Taken together, however, the organization may be building an entirely new operating architecture without ever formally recognizing that a significant change has occurred.

That is exactly the type of problem Management of Change was designed to address.

Before introducing an AI system into a safety-significant process, the organization should understand its purpose, decision authority, data sources, access, failure modes and interdependencies. It should also know how actions can be reversed, how abnormal behavior will be detected, who can withdraw authority and how incidents or near misses involving the system will be captured.

The principle is straightforward:

Increasing capability is a change. Increasing autonomy is a change. Increasing access is a change. Connecting previously independent systems is a change.

Any of those changes may alter the risk profile of an operation.

If changing a process chemistry, operating limit or control-system logic warrants formal review, increasing the decision authority of an AI system should at least trigger the question of whether similar scrutiny is appropriate.

The same principle appeared in our process digital twin work, where validation, cybersecurity, governance and Management of Change were identified as critical conditions for moving digital models from observation toward operational control.

Think About AI Like a SIF Exposure

EHS professionals have become much more sophisticated in how we think about serious injury and fatality prevention.

We increasingly recognize that a low recordable injury rate tells us relatively little about whether catastrophic-risk conditions exist.

AI governance may require the same evolution.

An organization could experience thousands of successful AI interactions while gradually building a small number of high-consequence failure pathways.

Suppose an AI system performs 99.99% of its tasks correctly. That sounds impressive.

But reliability alone is not enough if the remaining failures include the ability to manipulate safety-critical equipment, authorize a dangerous material transfer, alter an engineering specification or compromise a control system.

The more useful question is:

Does the system have exposure to high-consequence decision pathways?

That is fundamentally a SIF question.

We should therefore pay attention not only to errors but also to AI high-potential events: situations in which an AI action, recommendation or system interaction could reasonably have produced a severe consequence under slightly different circumstances.

An agent attempting an unauthorized connection should not be dismissed because the safeguard worked.

A model producing an incorrect process parameter should not automatically be categorized as simply a poor response.

An AI workflow bypassing an expected approval step may be more than an IT issue.

Those events can be weak signals of control-system vulnerability.

This connects directly to the predictive-risk principle I explored in Seeing Risk Before It Hurts: the most valuable safety opportunity often exists during the period when risk is forming but before an injury or major event occurs.

AI Will Create Inherited Risk

There is another reason EHS professionals should be involved early.

The worker exposed to AI-generated risk may have had almost nothing to do with creating it.

In traditional operations, workers frequently inherit risk created upstream through design decisions, maintenance practices, staffing, production pressures, purchasing specifications, procedures or management systems.

I explored this concept in Inherited Risk Ecosystem: How Upstream Decisions Shape Human Error at the Point of Work. The central premise is that what appears to be a frontline error may actually be the final expression of risk created much earlier elsewhere in the organization.

AI will expand that inherited-risk pathway.

An employee may interact with one visible AI interface, while the underlying risk was created through decisions involving model selection, software architecture, permissions, procurement, system integration, cybersecurity configuration, automation strategy or human-machine-interface design.

The risk may also originate in another AI system entirely.

By the time the hazard reaches the operator, technician or maintenance employee, the individual performing the work may have very little ability to influence it.

That should change how we think about responsibility.

AI risk may increasingly be inherited by the worker rather than created by the worker.

The appropriate question is therefore not simply whether the employee used the AI tool correctly.

We should be asking:

What risk has this worker or operation inherited from the AI-enabled systems surrounding the work, where was that risk created, and where can the pathway be interrupted?

That extends the Inherited Risk Ecosystem directly into AI-enabled operations.

It also prevents organizations from making the same mistake they have historically made with other hazards: placing responsibility at the point of work for risk that was actually created much farther upstream.

Humans + AI Must Remain the Operating Model

The goal should not be humans versus AI.

It should be Humans + AI.

AI can identify patterns across datasets that no safety professional could manually review. It can connect incident information, audits, maintenance records, process conditions and operating experience across an enterprise. It can extend scarce technical expertise across dozens of facilities and help identify weak signals that might otherwise remain invisible.

Those capabilities could materially improve workplace safety.

This is the core argument of my earlier article Human + AI: The Power of Synergistic Collaboration: the competitive and safety advantage comes from combining human judgment, context and ethical reasoning with AI’s analytical and predictive capacity.

But collaboration is not the same as giving up control.

AI may be exceptionally good at helping us answer:

What are we missing?

People still need to determine:

What risk are we willing to accept, and what decisions must remain ours?

That boundary becomes more important as systems move from recommendation toward independent action.

Where EHS Professionals Have Influence

EHS professionals do not need to become machine-learning engineers to influence this transition.

We already work with many of the disciplines advanced AI governance will require: hazard identification, barrier management, Management of Change, human factors, assurance, organizational learning and low-frequency/high-consequence risk.

That expertise gives the profession several practical points of influence.

EHS should participate in AI governance when systems affect workers, hazardous materials, equipment, operational decisions or critical infrastructure.

We can help organizations establish risk-based autonomy levels so that an AI used to generate training material is not governed in the same way as one connected to a chemical process.

We can bring AI-related changes into existing MOC processes, help determine when pre-startup reviews are appropriate, and ensure that high-consequence AI failure scenarios are incorporated into emergency planning.

We can also help establish AI incident and near-miss reporting. A control that prevents an AI system from taking an unsafe action is evidence that the control worked, but it may also be evidence that the organization encountered a significant precursor.

Human factors will be equally important. Automation bias—the tendency to trust automated recommendations because they appear authoritative—could become one of the most significant risks as AI becomes embedded in daily operations.

Finally, EHS can reinforce the principle of defense in depth.

No single human reviewer, model safeguard, permission setting or shutdown mechanism should stand between a highly autonomous system and a catastrophic consequence.

This broader role for EHS also reflects an argument I made in If Everything Still Comes to You, You Haven’t Scaled Leadership: enterprise leaders increasingly create value not by personally making every decision, but by designing systems that enable good decisions to occur consistently across the organization. AI governance is becoming one more part of that operating system.

A Familiar Risk in an Unfamiliar Technology

Google DeepMind’s Frontier Safety Framework now considers scenarios involving AI systems that could interfere with an operator’s ability to direct, modify or shut down their operation. Researchers are also examining whether future AI systems could significantly accelerate AI research itself.

Those possibilities remain uncertain.

That uncertainty cuts both ways.

We do not know that advanced AI will produce catastrophic outcomes. We also do not know that humans will always maintain effective control over increasingly capable and interconnected autonomous systems.

Safety professionals operate in that space all the time.

We do not require certainty that a pressure vessel will rupture before installing relief protection. We do not wait for a combustible dust explosion before managing dust accumulation. We do not require a fatality before acting on a high-energy exposure.

We identify credible scenarios, evaluate consequences and install controls proportionate to the potential loss.

Advanced AI deserves the same discipline.

The Lesson Behind the Headline

The phrase AI could kill us all may ultimately prove dramatically overstated or remarkably prescient.

EHS professionals do not need to resolve that debate to recognize what is already changing.

Artificial intelligence is moving from information generation to decision support, from decision support to agency, and from agency toward interconnected autonomous systems.

The risk associated with that transition will not be determined only by how intelligent AI becomes.

It will also depend on where organizations place it, what they connect it to, what authority they grant it and how quickly those boundaries expand.

For EHS, this is familiar territory.

We are dealing with another complex system containing powerful capabilities, interacting components, imperfect information and uncertain failure modes.

AI may be a new technology, but the underlying obligation is familiar: understand the hazard before expanding the exposure.

The EHS profession has learned that lesson repeatedly in factories, chemical plants, aircraft, laboratories and energy systems.

We now have the opportunity to apply it earlier in the development of AI-enabled operations rather than after the first major failures force us to.

That may become one of the most important contributions the profession makes to the age of artificial intelligence.

References

Brandon, C. (2024, November 24). Harnessing AI to revolutionize EHS management: A vision for the future. LeadingEHS.com.

Brandon, C. (2025, August 3). AI as a strategic partner: Building a digital twin to advance safety and sustainability. LeadingEHS.com.

Brandon, C. (2025, November 2). Human + AI: The power of synergistic collaboration. LeadingEHS.com.

Brandon, C. (2026, January 11). Seeing risk before it hurts: An example of how predictive analytics are redefining safety. LeadingEHS.com.

Brandon, C. (2026, July 24). Inherited Risk Ecosystem: How upstream decisions shape human error at the point of work. LeadingEHS.com.

Brandon, C. (2026, September 5). If everything still comes to you, you haven’t scaled leadership. LeadingEHS.com.

Brandon, C. (2026). When AI supervises AI: Reliability, repeatability and error stack up in multi-agent systems [Manuscript submitted for publication]. Professional Safety.

Brandon, C., & Feeney, F. (2026). Cyber-Physical Risk in the Age of AI [Four-part series]. LeadingEHS.com.

Genewein, T., Franklin, M., Lerchner, A., Orseau, L., Albanie, S., Bales, A., Wyeth, C., Chan, S., Gabriel, I., Leibo, J. Z., Dafoe, A., Hutter, M., Graepel, T., & Legg, S. (2026). From AGI to ASI. arXiv. https://doi.org/10.48550/arXiv.2606.12683

Google DeepMind. (2026, June 11). Investing in multi-agent AI safety research: Scaling AI safety research for a multi-agent world.

Google DeepMind. (2026). Strengthening our Frontier Safety Framework.

Shah, R., & Flynn, F. (2026, June 18). Securing the future of AI agents. Google DeepMind.

The Wall Street Journal. (2026, September 10). How would AI actually kill us all? What to know about the AI doomsday debate.

Winful, E., & Brandon, C. (2026, July 6). Harnessing the Process Digital Twin: Building live operating models for foresight, control, and safer work. LeadingEHS.com.