What is 'data poisoning' in generative AI models?

Data poisoning in generative AI is an adversarial attack where malicious, corrupted, or biased data is intentionally injected into the training, fine-tuning, or retrieval sets of machine learning models^[1][2]. Attackers may introduce altered data points, substitute incorrect labels, or embed hidden backdoors via benign-looking inputs and tools to subtly alter model behavior^[3][4].

This manipulation can significantly degrade model performance, cause data misclassification, amplify existing demographic biases, and open the door to severe security vulnerabilities^[5][6]. By compromising data integrity and instilling hidden triggers that activate under specific conditions, data poisoning undermines the overall reliability and trust users place in AI systems^[7][8].