An Enterprise AI Vendor Risk Assessment Checklist
AI procurement requires more than a security questionnaire. The service can change through model updates, data practices, embedded tools, and upstream providers.
Understand the service chain
Identify the vendor, underlying model providers, hosting environment, subprocessors, data regions, retrieval sources, plug-ins, and connected tools. Ask which components can change without notice and which contractual commitments apply to upstream services. The product name rarely describes the full dependency chain.
Document what data is collected, retained, logged, used for training or product improvement, reviewed by humans, and shared. Confirm deletion, export, legal hold, and incident notification procedures.
Evaluate performance and control
Test the product on representative workflows and data rather than relying on benchmark claims. Measure quality, unsupported claims, latency, availability, accessibility, security behavior, and performance across relevant user groups. Understand configurable safeguards, audit logs, identity integration, permissions, and administrative controls.
Ask how model or policy updates are communicated and whether the organization can pin versions, test changes, stage releases, or roll back. A silent model change can alter a validated workflow.
Plan for commercial and exit risk
Clarify intellectual-property terms for inputs, outputs, generated code, and indemnification. Model usage, retrieval, storage, and tool execution may create different cost drivers, so test realistic volumes and failure loops. Set spending alerts and contractual protections where appropriate.
Assess portability of prompts, evaluations, embeddings, logs, and business rules. Define how operations continue during outages or vendor exit. The goal is not to eliminate third-party risk; it is to understand which risks the business accepts and which controls make the dependency manageable.
Leadership checklist
- Map underlying models, subprocessors, data flows, and tool access.
- Test representative workflows and failure modes.
- Require notice and testing options for material changes.
- Define cost controls, continuity, data return, deletion, and exit.