How to Adapt Your API Strategy to Succeed in the AI Era

Analysis

Ensure Success of Your AI Initiatives by Adapting Your API Strategy

• Unplanned and dynamic API consumers: LLMs, such as GPT, LaMDA and LLaMa, are capable of dynamically generating API clients on the fly, which can lead to even more unplanned consumers. Dynamic API consumers could include code snippets, client side code in different programming languages, SDKs, and even applications that can consume APIs.
• Data security and compliance: Lack of adequate security and compliance of APIs can undermine the security of AI initiatives. AI models trained on unsecured data can significantly multiply the risks and lead to major vulnerabilities and exposure of unauthorized data.
• Copyright and licensing: APIs not protected by sufficient terms and conditions and applicable usage agreements can lead to copyright, licensing, and brand reputation issues associated with generated content from AI programs.
• Overuse or misuse: Unregulated AI access to APIs without capacity planning and proper policies for throttling and rate limiting can lead to overuse and misuse, overwhelming the API provider. This will ultimately lead to cost overruns, architecture instabilities and performance degradation.

Figure 2: OWASP Top 10 Security Vulnerabilities for LLM Applications

What Changes Are Needed for Your API Strategy?

• API life cycle management
• API security
• Business alignment
• Developer enablement
• Measuring business value

Figure 3: A Robust API Strategy Is Key to GenAI Initiatives

• API life cycle management: Refers to the steps involved in managing an organization’s APIs through all stages of its life cycle — from planning, development and operations, all the way through retirement. The API life cycle management strategy needs to change to accommodate APIs that are dynamically created by AI agents and other AI programs and tools, to avoid API sprawl and unmanaged shadow APIs. Here are a few steps to consider:
  • Track and manage APIs that are (or going to be) created by AI applications and GenAI tools just as well as APIs that are created by human actors (internal developers, partners or third parties).
  • Ensure your organization’s API standards, style guides or other coding practices are well documented, ready and available to ground AI models.
  • Ensure your APIs include appropriate terms of use, restrictions and privacy policies to prevent unwanted access and to leakage of sensitive information.
  • Maintain adequate documentation for all APIs, especially the ones created by GenAI.
  • Adhere to versioning and governance policies for all APIs.
  • Track and manage AI-generated API consumers.
  • Test and assure quality of all APIs. Use automated testing as much as possible especially for APIs created by GenAI tools.

• API security: Involves fine-grained authentication and authorization, access management, throttling and proactive threat protection. GenAI tools and applications introduce significant challenges to all of these aspects. Here are some examples of necessary steps:
  • Ensure that your API access management strategy is adapted for AI consumers, such as AI agents. For example, AI agents must be uniquely identified with adequate permissions, rather than using common or developer identities.
  • Set fit-for-purpose authentication and authorization policies to prevent unwanted data exposure to AI programs.
  • Ensure GenAI models generating APIs are data privacy aware to prevent unwanted privacy violations and leakage of sensitive information.
  • Actively manage API keys and credentials. Keys intended for AI services, such as OpenAI’s APIs, should be tracked separately.
  • Look beyond the interface, and ensure the underlying data and assets are allowed for AI consumption in collaboration with data and security engineers.
  • Actively monitor operational data and telemetric data to prevent unauthorized use.
  • Leverage API security tools to strengthen proactive threat protection of APIs. Prefer tools that provide built-in AI-based features to detect anomalies.
  • Beef up throttling and traffic management policies to ensure support for elevated use.

• Business alignment and measuring business value: The business alignment part of the API strategy should include both direct and indirect business benefits targeted for the AI initiative(s), especially those using APIs. The benefits should be measured by KPIs and metrics that measure business outcomes.
• Developer enablement: This strategy should now encompass developers using tooling such as AI code assistants. Developers are already using AI-augmented tooling to develop applications that consume APIs. This means that, for example, API documentation must identify any specific rules or restrictions for AI consumers, such as access restrictions, limits or other terms of appropriate use. Create procedures to prevent API sprawl and ensure that an API catalog is maintained and used by all consumers.

Manage API Generation and API Consumption by AI

Managing API Generation by AI

• Creating APIs from a natural language description of requirements, like an AI code assistant.
• Creating API specifications from code or data source.
• Generating API documentation from API specification and vice versa.
• Translating API documentation to different human languages.
• Converting or transforming implementations from one programming language to another.
• Converting OpenAPI to other API description formats and vice versa.
• Chat agents for API documentation.
• API product suggestions and intelligent recommendations for APIs.
• API policy generation and recommendations.
• AI-augmented deployment and operational management of APIs.

Managing API Consumption Challenges

Figure 4: APIs With the Generative AI Stack

• Ensure that there is adequate life cycle management of internal, partner, public and third-party APIs used across the organization.
• Manage the consumption of AI provider APIs. Make sure to track and manage all AI APIs in use across various projects.
• Implement adequate capacity planning, throttling and rate limiting policies for all APIs consumed, especially by AI initiatives.
• Review and adapt terms and conditions, along with any appropriate usage restrictions, for APIs made available for AI consumption.
• Make sure to review if API pricing and licensing models can support and appropriately scale with AI use.
• Implement API response contextualization, semantic caching, management of prompts or LLMs as required.
• Ensure that a plan is in place for monitoring and enforcing SLAs and API key management.
• Implement an API consumption gateway to support GenAI technical stack to address below common requirements:
  • Mediating traffic between different LLM models
  • API keys and credentials management
  • Token management
  • Prompt templating
  • Prompt guards
  • Content filtering
  • Sanitation of sensitive data
  • Policy enforcement
  • LLM orchestration
  • Rate limiting
  • Semantic caching and routing

Conduct POCs of AI Gateways to Evaluate Their Technical Capabilities

Figure 5: AI Gateways and Emerging AI-Based API Tools