Security Risks In Ai Supply Chains

North Mini Code. Cohere's first model for developers.

Learn more

Jul 01, 2025

4 minutes read

Security risks in AI supply chains

A new report by the Coalition for Secure AI details the unfamiliar threats from the data, models, and infrastructure that underpin AI, and how enterprises can tackle them.

!Blog Post Featured Image

Enterprises are under rising pressure to put advanced AI solutions to work in order to stay competitive. But this pressure to adopt quickly comes with risks.

A new report by the Coalition for Secure AI (CoSAI) highlights the key AI security risks that enterprises need to be aware of and how they should think about addressing them. The report is among the most comprehensive studies on the state of AI supply-chain security risks and the strategies that enterprises can deploy to address them.

Among its key findings:

• Organizations need to treat the entire AI supply chain as a dynamic, evolving attack surface that requires continuous innovation to defend
• AI systems require specialized risk assessments beyond conventional security frameworks
• Defense strategies should include end-to-end risk assessments, early engagement protocols to mitigate risks during development, and efforts to ensure data provenance and transparency
• “Model signing” – where a model is cryptographically authenticated by the provider to verify its integrity and provenance – should be a priority to ensure models haven’t been tampered with
• Security practitioners, AI developers, and executives should work together to integrate AI-specific and traditional controls

As enterprises struggle to get to grips with this new security paradigm, the CoSAI report provides a toolkit of strategies, empowering organizations to innovate with confidence and navigate the evolving landscape with resilience.

Addressing vital AI security issues

AI systems pose complex security challenges that are very different from those associated with traditional software. Spread across a complex landscape of data, models, applications, and infrastructure, the AI supply chain is vulnerable to a wide range of threats, including from malicious actors. This can expose organizations to serious repercussions from wary consumers and increasingly watchful regulators.

The following are four security issues highlighted in the report that are vital for enterprise leaders to understand.

###### Preventing tainted data

Data is the foundation of AI, providing the raw material for both training and using models.

The vulnerability

Bad actors can weaponize the public internet by introducing malicious or incorrect data, knowing that AI crawlers may eventually collect it for training.

Post-collection data is at risk too. Unlike conventional software where sensitive information is usually stored in specific locations that are easily tracked, information within an AI system may be distributed across millions or billions of weighted connections, creating a large and relatively unmapped surface of attack.

The solutions

Organizations need to monitor and test data sourced from the public internet. It can be a daunting task considering the amount of data used to train AI models, but achievable with the help of data validation pipelines and anomaly detection systems to screen the incoming information.

Organizations can then ensure this vetted data is kept secure by keeping meticulous records of where and how the data is stored and sourced. Cryptographic tools like content hashes and checksums can help verify data integrity, while URL logging can monitor web addresses accessed by the system to detect potential threats.

###### Securing model weights against tampering

Hackers may attempt to alter the numerical values that are used by AI models to provide responses, which can then result in harmful or incorrect outputs.

The vulnerability

Attackers may hack into a server or steal credentials to tamper with model weights, which can cause an AI model to misclassify data or generate harmful content. The effects aren’t necessarily immediate either. A hack could introduce hidden triggers that allow the model to act normally until a specific input causes it to fail or leak data.

The solution

Strong access controls and multi-factor authentication can limit access to the servers and systems where models are stored. Model signing allows you to verify the authenticity of a model artifact to be sure it hasn’t been tampered with.

Organizations should also establish a baseline for model behavior, then continuously monitor and audit the model to look for anomalies. Sigstore, for example, can be used to verify model artifacts (all the files generated in the training and packaging of a model) and help detect any subsequent tampering.

###### Identifying adversarial prompts

Prompts are used to instruct an AI model to find information and provide answers. But a sophisticated attacker may use an adversarial prompt to trick an AI model into providing sensitive, incorrect or harmful information.

The vulnerability

Adversarial prompts can be contained in reference materials, or introduced through the same interfaces employed by approved users, including chatbots, text, images and scripts that target APIs. By exploiting weaknesses such as flaws in the training data or inadequate content moderation, they can override a model’s guardrails and insert malicious instructions.

The solution

Models should be monitored for red flags such as repeated keywords, encoded characters, or suspicious phrasing, as well as responses that deviate from expected behavior, such as contradictions and a sudden switch to unsafe content.

Pattern matching and heuristic analysis can be applied to data sourced by retrieval-augmented generation (RAG) systems, ensuring that the retrieved context is free of adversarial prompts. Regular penetration testing, ethical red-teaming, and other adversarial testing can be used to...