What is LLM Data Poisoning?
A cyberattack where malicious actors intentionally corrupt the data used to train or fine-tune an AI model, causing it to output incorrect, biased, or harmful information.
The Corporate Definition
Large Language Models (LLMs) learn by analyzing vast amounts of text. Data Poisoning occurs when an attacker infiltrates that training data and introduces subtle, malicious alterations. Because the AI learns from this corrupted data, its fundamental understanding of certain concepts becomes flawed.
How Data Poisoning Works
Attackers can poison data in several ways:
- Backdoor Attacks: The attacker introduces a specific "trigger" word into the training data. When the AI sees this trigger in the future, it executes a malicious action or outputs a specific falsehood.
- Availability Attacks: The attacker floods the training data with noise or contradictory information, degrading the overall accuracy and reliability of the model.
- Targeted Misinformation: The attacker subtly alters facts within the training set (e.g., changing financial figures or medical advice) to manipulate the AI's future outputs.
Why It Matters for Enterprise Risk
Data poisoning is incredibly difficult to detect because the malicious data is often hidden within terabytes of legitimate text. If an enterprise fine-tunes an open-source model using poisoned data scraped from the internet, the resulting AI system becomes fundamentally untrustworthy.
This can lead to severe reputational damage, flawed automated decision-making, and significant legal liability if the AI provides harmful or discriminatory outputs.
Secure Your AI Supply Chain
Module 2 of the CAISG Certification covers Generative AI Threat Modeling, teaching you how to identify vulnerabilities like data poisoning and secure your AI infrastructure.
View the Curriculum