> ## Documentation Index
> Fetch the complete documentation index at: https://docs.portkey.ai/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Fallbacks

> Automatically switch to backup LLMs when the primary fails.

Specify a prioritized list of providers/models. If the primary LLM fails, the AI Gateway automatically falls back to the next in line.

## Examples

```json Between Models theme={"system"}
{
  "strategy": { "mode": "fallback" },
  "targets": [
    { "override_params": { "model": "@openai-prod/gpt-4o" } },
    { "override_params": { "model": "@anthropic-prod/claude-3-5-sonnet-20241022" } }
  ]
}
```

```json Between Providers (model from request) theme={"system"}
{
  "strategy": { "mode": "fallback" },
  "targets": [
    { "provider": "@openai-prod" },
    { "provider": "@azure-prod" }
  ]
}
```

```json On Rate Limit Only (429) theme={"system"}
{
  "strategy": { "mode": "fallback", "on_status_codes": [429] },
  "targets": [
    { "provider": "@openai-prod" },
    { "provider": "@azure-prod" }
  ]
}
```

```json Multi-tier Fallback theme={"system"}
{
  "strategy": { "mode": "fallback" },
  "targets": [
    { "override_params": { "model": "@openai-prod/gpt-4o" } },
    { "override_params": { "model": "@anthropic-prod/claude-3-5-sonnet-20241022" } },
    { "override_params": { "model": "@google-prod/gemini-1.5-pro" } }
  ]
}
```

> [!NOTE]
>
> The `@provider-slug/model-name` format automatically routes to the correct provider. Set up providers in [Model Catalog](https://stratacloudmanager.paloaltonetworks.com/).

[Create](https://docs.portkey.ai/docs/aigw/product/ai-gateway/configs.md#creating-configs) and [use](https://docs.portkey.ai/docs/aigw/product/ai-gateway/configs.md#using-configs) configs in your requests.

## Trigger on Specific Status Codes

By default, fallback triggers on any **non-2xx** status code.

Customize with `on_status_codes`:

```json theme={"system"}
{
  "strategy": { "mode": "fallback", "on_status_codes": [429, 503] },
  "targets": [
    { "provider": "@openai-prod" },
    { "provider": "@azure-prod" }
  ]
}
```

## Tracing Fallback Requests

AI Gateway logs all requests in a fallback chain. To trace:

1. Filter logs by `Config ID` to see all requests using that config
2. Filter by `Trace ID` to see all attempts for a single request

## Combine with Other Strategies

Fallback targets are fully composable — each target can be a load balancer, a conditional router, or another fallback. Any strategy can nest inside any other.

> **[Fallback When the Whole Cluster Goes Down](https://docs.portkey.ai/docs/guides/use-cases/combining-routing-strategies.md#fallback-when-the-whole-cluster-goes-down)**
>
> The primary target is a load balancer cluster. Individual endpoint failures stay within the cluster — only a full provider outage triggers the cross-model backup.

> **[Smart Failover by Request Context](https://docs.portkey.ai/docs/guides/use-cases/combining-routing-strategies.md#smart-failover-by-request-context)**
>
> When the primary fails, a conditional router picks the best backup based on request context — keeping EU users on EU infrastructure even during outages.

> **[Isolate Failures Between Model Families](https://docs.portkey.ai/docs/guides/use-cases/combining-routing-strategies.md#isolate-failures-between-model-families)**
>
> Give each load-balanced slot its own independent fallback chain. One model family failing doesn't trigger another family's backup.

> **[All Patterns Together](https://docs.portkey.ai/docs/guides/use-cases/combining-routing-strategies.md#the-full-config)**
>
> See a complete config combining conditional routing, load balancing, and fallbacks across four model families.

## Considerations

* Ensure fallback LLMs are compatible with your use case
* A single request may invoke multiple LLMs
* Each LLM has different latency and pricing


## Related topics

- [Claude Code with Anthropic](https://docs.portkey.ai/docs/aigw/integrations/libraries/claude-code-anthropic.md)
- [Claude Code with Amazon Bedrock](https://docs.portkey.ai/docs/aigw/integrations/libraries/claude-code-bedrock.md)
- [Pi Agent](https://docs.portkey.ai/docs/aigw/integrations/libraries/pi-agent.md)
- [Azure AI Foundry](https://docs.portkey.ai/docs/aigw/integrations/llms/azure-foundry.md)
- [Claude Code with Google Vertex AI](https://docs.portkey.ai/docs/aigw/integrations/libraries/claude-code-vertex.md)


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.