This feature is available on all Portkey plans.
So, instead of maintaining separate integrations for different multimodal LLMs, you can interact with models from OpenAI, Anthropic, Meta, Cohere, Mistral, and many more (100+ models, 15+ providers) - all using a common, unified API signature.

​
Portkey Follows OpenAI Spec

Portkey API is powered by its battle-tested open-source AI Gateway, which converts all incoming requests to the OpenAI signature and returns OpenAI-compliant responses.

​
Switching Providers is a Breeze

import Portkey from 'portkey-ai';

// Calling OpenAI
const portkey = new Portkey({
  provider: "openai",
  Authorization: "Bearer sk-xxxxx"
})

const response = await portkey.chat.completions.create({
  messages: [{ role: 'user', content: 'Hello' }],
  model: 'gpt-4',
});

// Swithing to Anthropic
const portkey = new Portkey({
  provider: "anthropic",
  Authorization: "Bearer sk-ant-xxxxx"
})

const response = await portkey.chat.completions.create({
  messages: [{ role: 'user', content: 'Hello' }],
  model: 'claude-3-opus-20240229',
});

​
Integrating Local or Private Models

Portkey can also route to and observe your locally or privately hosted LLMs, as long as the model is compliant with one of the 15+ providers supported by Portkey and the URL is exposed publicly. Simply specify the custom_host parameter along with the provider name, and Portkey will handle the communication with your local model. 
import Portkey from 'portkey-ai';

const portkey = new Portkey({
    apiKey: "PORTKEY_API_KEY",
    provider: "mistral-ai",
    customHost: "http://MODEL_URL/v1/" // Point Portkey to where the model is hosted
})

async function main(){
    const response = await portkey.chat.completions.create({
        messages: [{ role: 'user', content: '1729' }],
        model: 'mixtral-8x22b',
    });
    console.log(response)
}

main()
Note:When using custom_host, include the version identifier (e.g., /v1) in the URL. Portkey will append the actual endpoint path (/chat/completions, /completions, or /embeddings) automatically. (For Ollama models, this works differently. Check here)

​
Powerful Routing and Fallback Strategies

With Portkey you can implement sophisticated routing and fallback strategies. Route requests to different providers based on various criteria, loadbalance them, set up retries or fallbacks to alternative models in case of failures or resource constraints. Here’s an example config where we set up a fallback from OpenAI to a locally hosted Llama3 on Ollama: 
config = {
	"strategy": { "mode": "loadbalance" },
	"targets": [
		{
			"provider": "openai",
			"api_key": "xxx",
			"weight": 1,
			"override_params": { "model": "gpt-3.5-turbo" }
		},
		{
			"provider": "mistral-ai",
			"custom_host": "http://MODEL_URL/v1/",
			"weight": 1,
			"override_params": { "model": "mixtral-8x22b" }
		}
	]
}

from portkey_ai import Portkey

portkey = Portkey(
    api_key="PORTKEY_API_KEY",
    config=config
)

​
Using the Anthropic’s /messages Route

Access models on Anthropic, Bedrock and Vertex AI through Anthropic’s native/messages endpoint using Portkey’s SDK or Anthropic’s SDK.
This route only works with Claude models on Anthropic, Bedrock and Vertex AI. For other models, use the standard OpenAI compliant endpoint.
curl --location 'https://api.portkey.ai/v1/messages' \
--header 'x-portkey-provider: @your-provider-slug' \
--header 'Content-Type: application/json' \
--header 'x-portkey-api-key: YOUR_PORTKEY_API_KEY' \
--data '{
    "model": "your-model-name",
    "max_tokens": 250,
    "messages": [
        {
            "role": "user",
            "content": "Hello, Claude"
        }
    ]
}'

​
Multimodality

Portkey integrates with multimodal models through the same unified API and supports vision, audio, image generation, and more capabilities across providers. Multimodal Capabilities

​
Supported Endpoints

Portkey’s Universal API supports a comprehensive range of endpoints across all major AI capabilities. Each endpoint follows the OpenAI specification while working seamlessly with 15+ providers.
Not all providers support every endpoint. Check our provider compatibility matrix to see which endpoints are available for each provider.

​
Core Endpoints

  • Chat Completions - An OpenAI compatible unified endpoint for generating responses with support for streaming, function calling, and multi-modal inputs across providers 50+ providers like OpenAI, Anthropic, Vertex AI, Bedrock, and more
  • Responses - An OpenAI compatible unified endpoint for generating responses with support for OpenAI models exclusively on responses API
  • Messages - An Anthropic-compatible unified endpoint for generating responses with support for streaming, function calling, and multi-modal inputs across providers like Anthropic, Vertex AI, Bedrock, and more.
  • Images - Generate, edit, and create variations of images using models like DALL-E, Stable Diffusion, and others
  • Audio - Convert speech to text (transcription) and text to speech across multiple languages

​
Advanced Capabilities

  • Fine-tuning - Customize models on your specific datasets
  • Batch Processing - Process large volumes of requests efficiently
  • Files - Upload and manage files for fine-tuning and batch operations
  • Moderations - Check content for safety and compliance

​
Additional Endpoints