Introduction
The AI supply chain introduces unique risks that extend beyond traditional software dependencies. Pre-trained models downloaded from public repositories, training data sourced from the internet, and the complex dependency chains of ML frameworks all present attack surfaces that most organizations are not equipped to evaluate.
This section examines the risks of poisoned pre-trained models on platforms like Hugging Face, the challenges of model watermarking and provenance, dependency vulnerabilities in AI frameworks, and strategies for securing MLOps pipelines.
Poisoned Pre-Trained Models
Organizations increasingly rely on pre-trained models from public repositories like Hugging Face, which hosts over 500,000 models. While this ecosystem accelerates AI development, it also creates a massive attack surface. Researchers have demonstrated that malicious actors can upload poisoned models that behave normally on standard benchmarks but contain hidden behaviors triggered by specific inputs.
A poisoned model might perform image classification accurately in 99.9% of cases but consistently misclassify images containing a specific trigger pattern. In a security context, a poisoned malware detection model could be trained to allow specific malware families to pass through undetected.
- Backdoor attacks: Models that respond to specific trigger inputs with attacker-chosen outputs
- Trojan models: Pre-trained models with hidden functionality that activates under specific conditions
- Serialization attacks: Malicious code embedded in model files (e.g., pickle files) that executes during model loading
- Model watermarking: Techniques to verify model provenance and detect unauthorized modifications
Serialization Risk: Python's pickle format, commonly used for model serialization, can execute arbitrary code during deserialization. A model file downloaded from a public repository could contain code that steals credentials, installs backdoors, or exfiltrates data when the model is loaded.
Dependency Risks in AI Frameworks
AI frameworks like PyTorch, TensorFlow, and Hugging Face Transformers bring extensive dependency trees. A typical ML project may include hundreds of transitive dependencies, each representing a potential vulnerability or supply chain attack vector.
The rapid pace of AI framework development means that breaking changes, deprecated APIs, and security patches are frequent. Organizations that pin specific versions for stability may miss critical security updates, while those that always use the latest versions face the risk of supply chain compromises in new releases.
Common AI Framework Risks
- CUDA/GPU driver vulnerabilities: GPU drivers and CUDA toolkits run with kernel-level privileges and are rarely included in patch management programs
- Jupyter notebook exploitation: Unsecured notebook servers provide direct code execution access in ML environments
- Model registry poisoning: Compromising internal model registries to replace legitimate models with trojaned versions
- Training data poisoning: Injecting malicious examples into training datasets scraped from the internet
Securing the MLOps Pipeline
Securing the MLOps pipeline requires treating models, training data, and ML infrastructure with the same rigor applied to traditional software supply chains. This means implementing provenance tracking, integrity verification, and access controls at every stage of the ML lifecycle.
Organizations should establish approved model registries, require cryptographic signatures on model artifacts, scan model files for embedded code before loading, and implement continuous monitoring of model behavior in production for signs of compromise or drift.
- Model scanning: Use tools like ModelScan to detect malicious payloads in serialized model files
- Safe serialization: Prefer SafeTensors format over pickle for model storage and distribution
- Provenance tracking: Maintain a chain of custody for models from training through deployment
- Behavioral monitoring: Continuously monitor deployed models for unexpected output patterns
Best Practice: Treat pre-trained models from public repositories like untrusted code. Scan them for embedded payloads, validate their behavior on your own test sets, and isolate them in sandboxed environments during initial evaluation. Never load a pickle file from an untrusted source in a production environment.