For AI agents: a documentation index is available at the root level at /llms.txt and /llms-full.txt. Append /llms.txt to any URL for a page-level index, or .md for the markdown version of any page.
ModelsChatRankingsDocs
DocsAPI ReferenceClient SDKsAgent SDKCookbookChangelog
DocsAPI ReferenceClient SDKsAgent SDKCookbookChangelog
  • Overview
    • Quickstart
    • Principles
    • Models
    • Stripe Projects
    • FAQ
    • Report Feedback
  • Models & Routing
    • Model Fallbacks
    • Provider Selection
    • Auto Exacto
    • Private Models
  • Features
    • Workspaces
    • Presets
    • Response Caching
    • Tool Calling
    • Structured Outputs
    • Message Transforms
    • Zero Completion Insurance
    • ZDR
    • App Attribution
    • Service Tiers
    • Sovereign AI
    • Router Metadata
    • Input & Output Logging
      • Latency and Performance
      • Prompt Caching
      • Uptime Optimization
      • Reasoning Tokens
LogoLogo
ModelsChatRankingsDocs
On this page
  • Minimal Overhead
  • Performance Considerations
  • Cache Warming
  • Credit Balance Checks
  • Model Fallback
  • Best Practices
Best Practices

Latency and Performance

Understanding OpenRouter's performance characteristics
Was this page helpful?
Previous

Prompt Caching

Cache prompt messages
Next
Built with

OpenRouter is designed with performance as a top priority. OpenRouter is heavily optimized to add as little latency as possible to your requests.

Minimal Overhead

OpenRouter is designed to add minimal latency to your requests. This is achieved through:

  • Edge computing using Cloudflare Workers to stay as close as possible to your application
  • Efficient caching of user and API key data at the edge
  • Optimized routing logic that minimizes processing time

Performance Considerations

Cache Warming

When OpenRouter’s edge caches are cold (typically during the first 1-2 minutes of operation in a new region), you may experience slightly higher latency as the caches warm up. This normalizes once the caches are populated.

Credit Balance Checks

To maintain accurate billing and prevent overages, OpenRouter performs additional database checks when:

  • A user’s credit balance is low (single digit dollars)
  • An API key is approaching its configured credit limit

OpenRouter expires caches more aggressively under these conditions to ensure proper billing, which increases latency until additional credits are added.

Model Fallback

When using model routing or provider routing, if the primary model or provider fails, OpenRouter will automatically try the next option. A failed initial completion unsurprisingly adds latency to the specific request. OpenRouter tracks provider failures, and will attempt to intelligently route around unavailable providers so that this latency is not incurred on every request.

Best Practices

To achieve optimal performance with OpenRouter:

  1. Maintain Healthy Credit Balance

    • Set up auto-topup with a higher threshold and amount
    • This helps avoid forced credit checks and reduces the risk of hitting zero balance
    • Recommended minimum balance: $10-20 to ensure smooth operation

  2. Use Provider Preferences

    • If you have specific latency requirements (whether time to first token, or time to last), there are provider routing features to help you achieve your performance and cost goals.