Your agency builds a client system on an AI provider's API. Six months later, the provider changes their pricing by 300%, deprecates the model version you depend on, or gets acquired and shuts down the product. Your client's system breaks, and they look at your agency for answers.
Every AI tool and provider you incorporate into a client project creates a dependency that carries risk. Model providers, vector databases, orchestration frameworks, monitoring tools—each one can change their terms, degrade their service, or disappear entirely. Systematic vendor risk assessment before building on any dependency protects your clients and your agency.
Why Vendor Risk Matters for AI
The AI Market Is Volatile
The AI tool landscape is young and turbulent. Companies that seem stable today may pivot, get acquired, or shut down tomorrow. Pricing models are experimental and subject to dramatic changes. APIs change frequently, sometimes with minimal notice.
Lock-In Is Real
Migrating between AI providers is not trivial. Different models have different capabilities, different prompt requirements, and different output formats. A system optimized for one model may perform poorly on another without significant rework.
Client Accountability
When a vendor issue disrupts a client's system, the client holds your agency accountable—not the vendor. You selected the vendor, integrated the dependency, and are responsible for the system's reliability.
The Assessment Framework
Category 1: Company Viability
Financial stability: Is the vendor funded, profitable, or at risk of running out of money?
- For public companies: Review financial statements and analyst coverage
- For private companies: Review funding history, investor quality, and burn rate indicators
- For open-source projects: Review community health, corporate sponsors, and maintainer commitment
Market position: Is the vendor a leader, contender, or niche player?
- Market share and growth trajectory
- Customer base size and quality
- Competitive threats and differentiation
Leadership and strategy: Is the vendor making decisions that suggest long-term commitment to the product you are using?
- Public roadmap and feature velocity
- Executive stability
- Strategic pivots or narrowing focus
Risk indicators: Red flags that suggest elevated viability risk:
- Rapid leadership turnover
- Multiple strategic pivots in a short period
- Declining community engagement or developer interest
- Failure to meet announced timelines repeatedly
- Acquisition rumors without clear benefits for users
Category 2: Technical Capability
Performance: Does the vendor's product meet your technical requirements?
- Accuracy/quality benchmarks relevant to your use case
- Latency at expected volume
- Throughput capacity
- Scalability evidence
Reliability: How dependable is the service?
- Historical uptime (published status pages)
- Incident history and response quality
- SLA terms and enforcement
- Redundancy and disaster recovery capabilities
Integration quality: How well does the product integrate with your stack?
- API quality and documentation
- SDK availability and maintenance
- Authentication options
- Error handling and debugging support
- Webhook and event support
Feature completeness: Does the product cover your needs without workarounds?
- Core feature coverage
- Edge case handling
- Customization options
- Feature gap severity
Category 3: Data and Security
Data handling: How does the vendor handle your client's data?
- Data processing locations (important for data residency)
- Data retention policies
- Data usage policies (training, analytics, sharing)
- Data isolation between customers
- Encryption at rest and in transit
Security posture: What security measures does the vendor maintain?
- Security certifications (SOC 2, ISO 27001)
- Penetration testing and vulnerability management
- Incident response capabilities
- Security team and practices
Compliance: Does the vendor support your compliance requirements?
- GDPR data processing agreement availability
- HIPAA Business Associate Agreement availability
- Industry-specific compliance certifications
- Privacy framework compliance (Privacy Shield successor, etc.)
Sub-processors: Does the vendor use third parties that introduce additional risk?
- Sub-processor list and transparency
- Sub-processor data handling
- Sub-processor change notification
Category 4: Commercial Terms
Pricing stability: How likely is pricing to change?
- Pricing history (how often and how much has pricing changed?)
- Pricing model (per-token, per-request, subscription—which is most predictable?)
- Volume discounts and commitment options
- Contract length and pricing guarantees
Terms of service: What rights and obligations do the terms create?
- Data ownership and usage rights
- Service level commitments and remedies
- Limitation of liability
- Termination rights and data portability
- Change notification requirements
Lock-in assessment: How difficult is it to migrate away?
- Data portability (can you export your data?)
- API compatibility (are there alternatives with similar APIs?)
- Proprietary formats or features (do they create dependencies?)
- Migration effort estimate
Category 5: Support and Ecosystem
Support quality: Can you get help when you need it?
- Support channels (email, chat, phone, dedicated)
- Response time commitments
- Support quality (technical depth, not just ticket triage)
- Enterprise support options
Documentation: Is the product well-documented?
- API documentation quality and completeness
- Guides and tutorials
- Change documentation (changelogs, migration guides)
- Community documentation and forums
Ecosystem: Is there a healthy ecosystem around the product?
- Community size and activity
- Third-party integrations
- Marketplace or plugin ecosystem
- Conference and educational content
Conducting the Assessment
Step 1: Identify Dependencies
List all AI tools and vendors that the project will depend on:
- AI model providers (OpenAI, Anthropic, Google, open-source)
- Vector databases (Pinecone, Weaviate, Qdrant)
- Orchestration frameworks (LangChain, LlamaIndex, custom)
- Monitoring and observability tools
- Supporting infrastructure (cloud providers, CDNs, databases)
Step 2: Classify Criticality
For each dependency, assess how critical it is:
Critical: The system cannot function without this dependency. Failure means system down. Important: The system is degraded without this dependency but can still function. Convenience: The dependency improves development or operations but is easily replaceable.
Step 3: Assess Each Vendor
For critical and important dependencies, conduct the full assessment across all five categories. For convenience dependencies, a lighter assessment is sufficient.
Step 4: Score and Decide
Rate each vendor as:
Low risk: Strong across all categories. Proceed with standard monitoring. Medium risk: Adequate with some concerns. Proceed with risk mitigation measures. High risk: Significant concerns in one or more categories. Proceed only if no alternatives exist and implement robust mitigation. Unacceptable risk: Critical concerns that cannot be mitigated. Do not proceed. Find an alternative.
Step 5: Define Mitigations
For medium and high-risk vendors:
- Abstraction layer: Build an abstraction layer that isolates your system from the vendor-specific API. This makes migration easier if needed.
- Fallback provider: Identify and validate an alternative provider that could replace the primary if needed.
- Data portability: Ensure your data can be exported from the vendor at any time.
- Contract protections: Negotiate pricing guarantees, data portability rights, and adequate termination notice periods.
- Monitoring: Track vendor health indicators (uptime, performance, pricing announcements, company news).
Ongoing Vendor Management
Quarterly Reviews
Review vendor risk quarterly:
- Any changes in company viability (funding, acquisitions, leadership)
- Performance and reliability trends
- Pricing or terms changes
- New competitors or alternatives
- Regulatory changes affecting the vendor
Trigger-Based Reviews
Conduct an immediate review when:
- The vendor announces a significant pricing change
- The vendor announces a product deprecation
- The vendor experiences a major outage or security incident
- The vendor is acquired or announces a merger
- A superior alternative emerges
Client Communication
Keep clients informed about vendor risk:
- Include vendor risk assessment in project proposals
- Notify clients of material vendor changes
- Include vendor status in regular project reports
- Recommend vendor changes proactively when risk increases
AI vendor risk assessment is a practice that protects your clients' investments and your agency's reputation. Build it into your project delivery methodology, maintain it throughout the engagement, and use it to make informed decisions about the foundations you build on.