Cybersecurity & AI in 2026: what organisations actually need to fix

In February 2024, the multinational engineering firm Arup lost $25.6 million when an employee in Hong Kong authorised a massive fund transfer following a Zoom call with the company's Chief Financial Officer and several colleagues. None of the people on that screen were human; deepfake video and voice attacks have fully transitioned from theoretical novelties to devastating enterprise realities, forcing every Chief Information Security Officer to confront a rapidly mutating threat landscape.

The state of AI security in 2026

We have fundamentally changed how enterprise software operates. For the last four decades, cybersecurity has relied on the premise of determinism: specific inputs processed by static code paths yield predictable outputs. You build firewalls, you sanitise database queries, and you restrict network traversal based on rigid, rule-based logic. The deployment of generative models and autonomous systems breaks this paradigm completely. An AI Agent does not process structured queries; it interprets natural language, interprets intent, and autonomously executes actions across your corporate environment. The prompt is no longer just text. It is entirely new, executable input bridging the gap between external untrusted actors and internal proprietary APIs.

Because artificial intelligence systems are inherently non-deterministic, traditional application security mapping fails. We are deploying software that reasons, hallucinates, and can be easily duped into bypassing its own guardrails. Furthermore, we are giving an AI Agent operational rights—the ability to read emails, write to databases, and trigger financial workflows—without providing it with human judgment. The disparity between adoption speed and security maturity has created the largest vulnerable attack surface we have seen since the dawn of internet-facing applications.

According to IBM's Cost of a Data Breach 2025 report, 13 percent of organisations have already experienced an AI model or application breach, and a staggering 97 percent of those organisations lacked adequate AI access controls.

The financial consequences of this control gap are concrete. The same IBM data reveals that the average AI-related data breach costs an organisation $4.44 million globally, skyrocketing to $10.22 million in the United States. Meanwhile, the sheer volume of external attacks leveraging generative infrastructure is overwhelming traditional defences. Data from Pindrop in 2025 showed a 1300 percent increase in deepfake fraud year-over-year, hitting the banking and insurance sectors disproportionately hard. The FBI Internet Crime Complaint Center documented $893 million in AI-fraud damages in a single year. These are not script kiddies running basic automated scanners; these are highly adaptive, AI-driven campaigns. According to Cofense telemetry from 2026, enterprise users now face one AI-generated phishing attack every 19 seconds. Alarmingly, training humans to spot these attacks is yielding diminishing returns; Hoxhunt data indicates that AI-generated phishing is now 55 percent more effective than campaigns designed by elite human red teams.

What goes wrong inside your organisation

Long before an external threat actor targets your infrastructure, the internal chaos of undocumented deployment compromises your perimeter. The foremost internal threat remains Shadow AI. Your executives, business analysts, and developers are bypassing IT procurement to use powerful, unsanctioned generative tools, and they are feeding your most sensitive intellectual property into public models. Gartner projects that by 2030, over 40 percent of organisations will suffer a major incident originating from Shadow AI, a reality already dawning on the 69 percent of CISOs who suspect or have confirmed its existence in their networks. The scale is daunting: research by Reco.ai in 2025 revealed that the average enterprise environment hosts roughly 490 Software-as-a-Service applications, with roughly 53 percent remaining completely unauthorised and unmonitored.

When staff feed confidential data into unsanctioned models, prompt leaking becomes inevitable. The precedent was spectacularly set during the Samsung incident in April 2023, where within a single month, employees uploaded proprietary source code, semiconductor test sequences, and sensitive meeting notes directly into ChatGPT, forcing the conglomerate to issue a blanket ban on generative AI. Compounding this is the ongoing risk of model memorization and extraction. Large Language Models are designed to recall patterns, and without strict filtering, they regularly recite verbatim personally identifiable information, internal passwords, and proprietary documents absorbed during training or subsequent corporate fine-tuning.

The next major vulnerability stems from how we build internal systems. Every time an Agentic Engineer integrates a model with corporate infrastructure, they risk creating what the industry calls Excessive Agency. Instead of adhering to the principle of least privilege, development teams often grant an AI Agent broad administrative permissions to ensure workflows function smoothly. A profound example emerged when the Cloud Security Alliance uncovered structural cloud privilege escalation risks via Vertex AI in 2026, demonstrating how over-permissioned agents can accidentally act as conduits for attackers to traverse cloud environments.

Supply chain integrity has also collapsed under the weight of AI development. Open-source repositories are riddled with compromised assets. In 2024, JFrog discovered hundreds of machine learning models hosted on HuggingFace containing backdoors leveraging notoriously insecure Python pickle exploits, granting remote code execution upon loading. Similar findings by nullifAI in 2025 proved this was a systemic issue. Making matters worse, the adoption of the Model Context Protocol has introduced deep vulnerabilities. Invariant Labs discovered critical flaws they termed "MCP Tool Poisoning," where malicious servers inject hidden instructions directly into your agents. Affecting major platforms like Anthropic, OpenAI, Zapier, and Cursor, Microsoft aptly described MCP as the "USB port for AI"—a highly convenient interface that readily accepts RugPull and Shadow Server attacks, infecting internal processes without triggering traditional endpoint detection.

Attack vectors from the outside

External actors have moved far beyond basic data theft; they are weaponising natural language to seize control of your generative infrastructure. The primary vector remains prompt injection. An attacker embeds malicious instructions within an otherwise benign input, forcing the model to disregard its original system prompt and execute the attacker's payload. This is executable code masked as conversation.

The progression of these attacks has been remarkably fast. We have moved from theoretical jailbreaks to weaponised exploits operating in the wild. EchoLeak (CVE-2025-32711) demonstrated the first zero-click exploit in a production large language model, proving that interaction isn't strictly necessary for a model to be compromised. Meanwhile, security researchers at Checkmarx pioneered the "markdown image trick," demonstrating seamless data exfiltrate via platforms like Copilot Chat and Gemini; an attacker simply forces the AI to render an image URL containing sensitive data as query parameters, silently sending corporate secrets to external servers. Unit42 at Palo Alto Networks tracked an explosion of indirect prompt injection in 2026, where attackers embed invisible malicious prompts onto regular corporate websites. When an employee's AI Agent scrapes that site to summarise its content, the agent unwittingly ingest the malicious instructions and executes them internally.

Extensive research published by the UK AI Security Institute and Anthropic in 2025 also confirmed our worst fears regarding data and model poisoning. Attackers no longer need vast amounts of data to corrupt a neural network; a surprisingly small, constant amount of carefully structured poison samples is sufficient to compromise a model's integrity. Perhaps more disturbingly, the researchers demonstrated that "Sure Trap" backdoors deliberately inserted into basic models comfortably survive sophisticated enterprise fine-tuning processes, lying dormant until triggered by a specific attacker codeword.

Adversaries are actively cataloguing these techniques against established frameworks. The MITRE ATLAS v5.6 matrix outlines the tactical lifecycle of these campaigns, highlighting reconnaissance techniques, ML model backdooring (T0018), eroding integrity (T0031), advanced LLM jailbreaks like the controversial DAN models and adversarial suffixes (T0054), and sophisticated indirect prompt injections (T0051). Defence requires continuous, dynamic evaluation against these precise adversarial patterns.

The OWASP LLM Top 10 as a checklist

To systematically defend against this chaos, security leadership must pivot from generic IT security frameworks to AI-specific operational checklists. The OWASP LLM Top 10 for 2025 is the definitive baseline for assessing where your architecture is failing. Treating this document as a hypothetical exercise is dangerous; your development teams need to map every Agentic Engineer workflow against these specific vulnerabilities.

• LLM01 Prompt Injection: This remains the absolute number one risk. Whether direct or indirect, failure to separate system instructions from user-provided data means any external user can hijack your model.
• LLM02 Sensitive Information Disclosure: You must assume that whatever data a model sees, it will eventually leak. Models regurgitate data. Relying on the model to "know" what is confidential is an architectural failure.
• LLM03 Supply Chain Vulnerabilities: Pulling unverified weights from HuggingFace without software bills of materials (SBOMs) is the modern equivalent of copy-pasting executable malware from unverified forums.
• LLM04 Data and Model Poisoning: Your training data pipeline requires cryptographic signing. If an attacker taints the foundational data, the entire resulting system is permanently compromised.
• LLM06 Excessive Agency: This relates directly to how an AI Agent scales. When an agent has read/write access to your Jira, your Slack, and your AWS environment simultaneously, a single prompt injection becomes a catastrophic network-wide breach.
• LLM07 System Prompt Leakage: Attackers steal your intellectual property by simply asking the model to reveal the proprietary instructions and architectural guidelines you provided to it.

Addressing these specific attack vectors requires more than updating an access control list. It demands an acknowledgment that LLM05 (Improper Output Handling) can turn a benign conversational AI into a tool for Cross-Site Scripting (XSS) or remote code execution, and that LLM10 (Unbounded Consumption) allows attackers to weaponise your AI's token generation rates to run up cloud compute bills in massive denial-of-wallet attacks.

What the law now requires

The regulatory grace period for generative experiments has officially closed. Boardrooms that view AI security purely as an IT problem will face crippling legal and financial repercussions. The European Union has firmly established the compliance baseline.

Under Article 15 of the EU AI Act, systems classified as high-risk are legally mandated to be technically robust against adversarial attacks, data poisoning, and prompt manipulation. Ignorance of MITRE ATLAS or OWASP vulnerabilities will not hold up in court. The penalties for failing to secure these systems are draconian, scaling up to €35 million or 7 percent of global annual turnover. Furthermore, Article 12 mandates complete record keeping, meaning every prompt and response must be logged and retrievable for auditing purposes. You cannot govern what you do not log.

The updated NIS2 directive, in effect since October 2024, explicitly expands supply chain security mandates. Incident reporting requires a 24-to-72-hour turnaround, and crucially, software component security now encompasses neural network model weights—a legal acknowledgment that models are executable software components. If JFrog or nullifAI identify compromised machine learning models in your pipeline, NIS2 holds you accountable for deploying them.

Simultaneously, privacy regulations have evolved. The General Data Protection Regulation requires a Data Protection Impact Assessment (Article 35) prior to deploying any high-risk AI, and Article 22 explicitly bans purely automated decisions that produce legal or significant societal effects without human supervision. To navigate this dense regulatory web, mature organisations are abandoning ad-hoc compliance and adopting formal management systems. ISO 42001:2023 is fast becoming the certifiable gold standard—the ISO 27001 equivalent specifically tuned for AI—working in tandem with the NIST AI Risk Management Framework 1.0. Following the NIST pillars to Govern, Map, Measure, and Manage is no longer a best practice; it is the evidentiary basis of your legal defence when an incident occurs.

A workable defensive architecture

Traditional firewalls and endpoint detection response platforms are blind to semantic payloads hidden within a natural language prompt. Security leaders must mandate a hard pivot towards a layered AI defensive architecture designed specifically to manage non-deterministic intent and excessive agency.

The foundational step is implementing a Model Gateway. You cannot allow individual development teams to build point-to-point API connections to OpenAI, Anthropic, or external HuggingFace deployments. A Model Gateway acts as an enterprise-wide proxy forcing all LLM calls through a single inspection point. This creates instant visibility for Shadow AI detection, enforces global rate-limiting to prevent Unbounded Consumption (OWASP LLM10), and acts as the enforcement layer for data privacy.

Within this gateway, rigorous input and output filtering must operate dynamically. By integrating data scrubbing engines like Presidio, you execute real-time PII redaction before the prompt ever reaches the external model. Conversely, output filtering via tools like Azure Prompt Shields analyses the returning text to block memorization leaks or malicious payload execution before it renders in the user's interface.

To defend against Model Context Protocol poisoning and excessive agency, you must enforce a Zero Trust architecture for every single AI Agent deployed on your network. Adopting the Microsoft March 2026 security blueprints or the CSA Agentic Trust Framework requires severing implicit trust. Agents must be heavily restricted. When an Agentic Engineer designs a tool meant to write code or configure infrastructure, the execution of that tool must occur inside an ephemeral, hardened sandbox. Replicating the AWS Prescriptive Guidance for 2026, tool sandboxing combined with aggressive egress filtering ensures that even if an agent is compromised via an MCP shadow server attack, the blast radius is confined.

Finally, to comply with the EU AI Act Article 14 and basic operational security sanity, every workflow that includes irreversible actions—such as financial transactions, database deletions, or mass external communications—must hardcode a human-in-the-loop mechanism. An AI Agent may draft the execution plan, but only a cryptographically authenticated human should authorise the trigger.

Start small, start now

Securing an intelligent enterprise is an iterative process, not a vendor purchase. The threat landscape detailed by IBM, Arup, and Hoxhunt proves that waiting for the dust to settle is an invitation to catastrophic loss. You need a structured, tactical timeline.

In the next 30 days, your priority must be visibility. Run an aggressively scoped discovery phase to identify your organisation's baseline. Audit your SaaS bloat to address the hundreds of undocumented instances detailed by Reco.ai, and forcefully block all unsanctioned shadow tools at the firewall level. Standardise your legitimate generative traffic by routing it exclusively through a centralised Model Gateway.

By day 60, shift the focus toward architectural resilience. Every Agentic Engineer in your organisation must map their current and future builds against the OWASP LLM Top 10. Implement hardened execution environments for all integrations, physically isolating Model Context Protocol instances in strict sandboxes. Ensure that prompt arrays and AI responses are securely logged to an immutable ledger.

As you approach day 90, pivot from immediate mitigation to structural governance. Align your internal risk metrics with the NIST AI RMF and begin the gap analysis required for ISO 42001 certification. Ensure that your automated compliance checks satisfy the immediate incident reporting and supply chain obligations defined under NIS2 and the adversarial resilience requirements of the EU AI Act.

You cannot secure artificial intelligence by recycling the playbooks of deterministic software. Building robust, legally compliant, and genuinely secure agentic workflows requires deep technical capability that bridges modern cloud engineering and offensive AI security. The transition is complex, but you do not have to architect it alone. At The Automation Group in Utrecht, we design, engineer, and secure enterprise AI systems built to withstand the realities of an adversarial world. We welcome a conversation on how to safely integrate intelligent agents into your corporate architecture.