How to build resilient AI systems by understanding machine learning attacks, strengthening defences, and securing intelligent technologies. Securing machine learning requires a much broader approach than simply improving model accuracy or preventing manipulated inputs. As artificial intelligence evolves towards autonomous agents, multimodal models, intelligent robotics, and interconnected AI-powered applications, security challenges are becoming increasingly complex.

Machine learning systems are no longer limited to analysing data and generating predictions. They increasingly interact with external tools, access databases, execute code, manage workflows, and make decisions that can affect real-world environments. This expansion creates new opportunities for attackers to manipulate AI behaviour, compromise sensitive information, exploit software dependencies, and interfere with automated operations.

The challenge in 2026 is therefore not just protecting individual models. It is securing the entire AI ecosystem, including training data, model architectures, deployment infrastructure, external integrations, access permissions, and the systems that depend on AI-generated decisions.

The Complete Guide to Attacks and Defenses in Robust Machine Learning

Understanding how machine learning systems can be attacked—and how to defend them—is fundamental to building secure and reliable artificial intelligence.

My book, Attacks and Defenses in Robust Machine Learning, is a comprehensive guide to the adversarial challenges facing machine learning systems and the techniques used to understand, analyse, and defend against them.

It brings together the offensive and defensive dimensions of machine learning security, examining how vulnerabilities can be exploited, how adversarial techniques can compromise model behaviour, and how robust defence mechanisms can strengthen intelligent systems.

The book provides a foundation for understanding the relationship between machine learning vulnerabilities, adversarial attacks, and defensive strategies. Its subject extends beyond individual AI trends or specific generations of models, making its underlying principles relevant as technology evolves.

From conventional predictive models to advanced AI applications, intelligent robotics, and increasingly autonomous systems, understanding how machine learning can be compromised remains essential.

For AI researchers, machine learning engineers, developers, cybersecurity professionals, and technology enthusiasts seeking a comprehensive understanding of the subject, Attacks and Defenses in Robust Machine Learning serves as a complete guide to machine learning attacks and defences.

As AI continues to advance, understanding how to attack and defend machine learning systems is no longer optional. It is fundamental to building robust, secure, and trustworthy AI.

 The Changing Landscape of Machine Learning Attacks 

The machine learning threat landscape has expanded as AI systems become more capable, connected, and autonomous.

Attackers increasingly target not only model predictions but also the surrounding infrastructure, information sources, software components, and permissions that determine what an AI system can access and do.

Several attack categories remain particularly important.

Adversarial Evasion Attacks: These attacks manipulate input data to cause incorrect predictions during inference. Small, carefully designed changes to images, audio, or other inputs can produce unexpected model outputs. This remains a major concern in facial recognition, autonomous vehicles, medical imaging, industrial inspection, and robotic perception.

Data and Model Poisoning: Attackers introduce malicious or manipulated information during training, fine-tuning, or other stages of the machine learning lifecycle. Poisoned datasets and compromised model updates can influence model behaviour, introduce vulnerabilities, or undermine reliability. As organisations increasingly depend on external datasets, pretrained models, and third-party AI components, supply-chain integrity becomes essential.

Backdoor and Trojan Attacks: These attacks introduce hidden triggers that cause models to behave incorrectly under specific conditions while appearing normal during ordinary testing. A compromised model may perform reliably until it encounters a particular image pattern, phrase, signal, or other trigger. This creates serious risks for organisations deploying externally developed models and intelligent systems.

Privacy and Model Extraction Attacks: Membership inference, model inversion, and model extraction techniques can reveal information about training datasets or enable attackers to reproduce proprietary models. These threats create risks involving confidential information, intellectual property, and personal data.

Prompt Injection and Context Manipulation: Generative AI introduces additional attack opportunities. Malicious instructions can be embedded in documents, websites, emails, images, and other information sources that an AI system processes. Instead of directly attacking the model, adversaries may manipulate the information surrounding it to influence its responses or actions.

AI Agent and Tool Manipulation: Autonomous agents introduce a more consequential attack surface because they can interact with external applications, databases, APIs, and operating environments. An attacker who successfully manipulates an agent may be able to influence operations beyond the conversation itself, potentially causing unauthorised access, data exposure, or unintended system changes.

Agentic AI: A New Security Challenge

One of the defining developments in AI security in 2026 is the growing adoption of agentic AI.

Unlike conventional machine learning applications that primarily classify information or generate outputs, AI agents can plan tasks, make decisions, interact with external tools, and execute sequences of operations with varying degrees of autonomy.

This creates a different security problem. A manipulated chatbot might generate an incorrect answer, but a compromised AI agent with access to business applications could potentially retrieve confidential information, modify files, execute commands, or initiate unauthorised transactions.

Several vulnerabilities deserve particular attention.

Indirect Prompt Injection: Malicious instructions hidden in websites, documents, emails, or retrieved information can influence an agent when it processes external content. This is especially concerning when agents have access to sensitive resources.

Excessive Permissions: Agents with broader access than their assigned tasks require can create opportunities for privilege abuse and data exposure. The principle of least privilege is therefore essential.

Insecure Tool Integrations: APIs, plugins, external services, and software connectors can introduce vulnerabilities into otherwise secure AI applications. Weak authentication, inadequate validation, and insecure configurations can expose connected systems.

Uncontrolled Autonomous Actions: Agents that execute operations without sufficient verification or human approval may create unintended consequences. Sensitive actions should be subject to appropriate authorisation and oversight.

Agent Identity and Accountability: As organisations deploy multiple agents that communicate and collaborate, they need reliable ways to identify each agent, control its permissions, record its actions, and establish responsibility for decisions.

The security challenge is no longer limited to protecting a model from malicious inputs. It also involves controlling what an AI agent can do, which systems it can access, and how its decisions are verified.

Multimodal AI and Emerging Attack Surfaces

AI systems increasingly process combinations of text, images, audio, video, and other data types. This creates new opportunities for attackers to exploit the relationships between different information channels.

A multimodal model may interpret an image, analyse spoken instructions, read documents, and combine these inputs to produce a decision. Each additional input channel introduces potential vulnerabilities.

For example, manipulated visual information could influence an AI-powered recognition system, while malicious instructions embedded in an image or document could affect a generative AI application.

In robotics and autonomous systems, the consequences can extend beyond incorrect digital outputs. Manipulated sensor inputs may influence object recognition, navigation, environmental perception, or decision-making.

Security testing must therefore consider more than conventional text-based attacks. Organisations should evaluate how models respond to manipulated images, audio signals, video content, conflicting inputs, and combinations of adversarial information.

Multimodal security also requires validating the integrity and origin of input data, monitoring unusual model behaviour, and ensuring that critical decisions are not based on a single potentially compromised information source.

Advanced Defence Strategies for Robust Machine Learning

Defending modern AI systems requires multiple complementary security layers. No single algorithm, filtering mechanism, or monitoring solution can protect every component of a complex machine learning environment.

Adversarial Training and Robust Optimisation

Adversarial training exposes models to carefully generated malicious examples during training. This can improve resilience against known perturbations and strengthen performance under defined adversarial conditions.

Robust optimisation seeks to minimise model errors across a specified range of possible inputs. However, robustness against one attack method does not automatically guarantee protection against different attack techniques or adaptive adversaries.

Data Provenance and AI Supply-Chain Security

Organisations should verify dataset origins, track modifications, validate third-party models, and maintain secure model registries.

Dataset versioning, cryptographic hashes, access controls, signed model artefacts, and reproducible training pipelines can help identify unauthorised modifications and reduce supply-chain risks.

Certified Robustness and Formal Verification

Certified defences provide mathematical guarantees within specified threat models. Techniques such as randomised smoothing, formal verification, and interval-bound propagation can establish limits on how much an input may change before a prediction becomes unstable.

These methods are valuable for applications where reliability is particularly important, although their guarantees depend on defined assumptions and perturbation boundaries.

Privacy-Preserving Machine Learning

Differential privacy, secure multiparty computation, and federated learning can reduce exposure of sensitive information during machine learning development and deployment.

However, privacy protection and model security are not interchangeable. Federated learning, for example, still requires protection against malicious participants, compromised devices, and poisoned model updates.

AI Agent Security and Permission Management

AI agents should operate with clearly defined identities, restricted permissions, and controlled access to external systems.

Organisations should separate trusted instructions from untrusted content, validate tool requests, restrict access to sensitive resources, and require human approval for high-impact operations.

Every significant agent action should be logged and traceable. Organisations should also establish procedures for revoking access and stopping agents when suspicious behaviour is detected.

Continuous Monitoring and Adversarial Red-Team Testing

Security cannot end when a model is deployed. AI systems need continuous evaluation against emerging attack techniques, unexpected inputs, abnormal behaviour, and changes in their operating environments.

Red-team exercises should test both individual models and complete AI applications, including their integrations, permissions, data sources, and external dependencies.

The objective is to identify weaknesses before attackers exploit them and to ensure that security controls remain effective as systems evolve.

 Securing the Entire AI Lifecycle

Effective machine learning security requires a lifecycle-based approach.

Protecting a model only during training is insufficient when vulnerabilities can emerge through deployment configurations, external integrations, software updates, or changes to the data it processes.

A comprehensive security strategy should cover five important stages.

Before Training: Verify datasets, assess their provenance, identify potential contamination, evaluate third-party dependencies, and define the threat model.

During Development: Conduct adversarial testing, assess model robustness, implement secure development practices, and evaluate privacy and data protection risks.

Before Deployment: Validate access controls, test external integrations, assess system permissions, and establish clear operational boundaries.

During Operation: Monitor model behaviour, inspect agent activity, detect anomalies, maintain audit logs, and evaluate changes in the threat environment.

After Deployment: Update security controls, investigate incidents, retrain or replace compromised components when necessary, and conduct regular adversarial assessments.

This approach recognises that AI security is a continuous responsibility rather than a one-time certification exercise.

 The Enduring Importance of Adversarial Machine Learning

Although AI technologies evolve rapidly, the fundamental principles of adversarial machine learning are not tied to a particular generation of models or applications.

The ability to manipulate data, exploit weaknesses in algorithms, compromise training processes, and influence model predictions remains relevant across different machine learning architectures.

New developments such as generative AI, autonomous agents, multimodal systems, and intelligent robotics introduce additional attack surfaces, but they do not eliminate the underlying security challenges.

In fact, understanding traditional adversarial machine learning provides an important foundation for addressing these newer threats.

Researchers and developers who understand how models can be manipulated are better equipped to evaluate emerging vulnerabilities, design appropriate countermeasures, and recognise the limitations of existing security mechanisms.

This is why knowledge of machine learning attacks and defences remains valuable beyond individual technology trends. The algorithms and applications may change, but the need to understand vulnerabilities and develop robust protection mechanisms remains constant.

7The Future of Robust Machine Learning

As AI becomes increasingly autonomous and integrated into critical applications, the distinction between machine learning security and conventional cybersecurity will continue to narrow.

Future defence strategies will need to address:

• Adaptive adversarial attacks against increasingly capable models.

• Manipulation of multimodal inputs and sensor data.

• Compromised datasets, pretrained models, and AI supply chains.

• Prompt injection and attacks against autonomous agents.

• Unauthorised access through connected tools and external services.

• Security vulnerabilities in AI-powered robotics and autonomous systems.

• Privacy risks involving increasingly interconnected data environments.

• Attacks against AI infrastructure and automated decision-making processes.

The objective is not to create models that are impossible to attack. It is to develop systems that can withstand realistic threats, limit potential damage, detect compromise, and recover safely.

In 2026, robust machine learning must be understood as a combination of algorithmic resilience, secure infrastructure, data integrity, controlled autonomy, continuous monitoring, and human oversight.

The future of trustworthy AI will depend not only on how intelligent models become, but also on how effectively we understand their vulnerabilities and defend against those who seek to exploit them.

Understanding attacks is the first step towards building stronger defences.

As machine learning continues to evolve, the principles of adversarial security will remain essential to developing intelligent systems that are reliable, resilient, and trustworthy.

Attacks and Defenses in Robust Machine Learning provides a comprehensive guide to understanding this essential field. While AI technologies, architectures, and applications will continue to change, the fundamental knowledge required to identify vulnerabilities, understand adversarial techniques, and develop effective defences will remain valuable for years to come.