> ## Documentation Index
> Fetch the complete documentation index at: https://docs.portkey.ai/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Lepton AI

> Use Lepton AI's serverless AI endpoints for chat completions and speech-to-text through Portkey.

## Quick Start

Get started with Lepton AI in under 2 minutes:

```python Python icon="python" theme={"system"}
from portkey_ai import Portkey

# 1. Install: pip install portkey-ai
# 2. Add @lepton provider in model catalog
# 3. Use it:

portkey = Portkey(api_key="PORTKEY_API_KEY")

response = portkey.chat.completions.create(
    model="@lepton/llama-3.1-8b",
    messages=[{"role": "user", "content": "Hello!"}]
)

print(response.choices[0].message.content)
```

```js Javascript icon="square-js" theme={"system"}
import Portkey from 'portkey-ai'

// 1. Install: npm install portkey-ai
// 2. Add @lepton provider in model catalog
// 3. Use it:

const portkey = new Portkey({
    apiKey: "PORTKEY_API_KEY"
})

const response = await portkey.chat.completions.create({
    model: "@lepton/llama-3.1-8b",
    messages: [{ role: "user", content: "Hello!" }]
})

console.log(response.choices[0].message.content)
```

```python OpenAI Py icon="python" theme={"system"}
from openai import OpenAI
from portkey_ai import PORTKEY_GATEWAY_URL

# 1. Install: pip install openai portkey-ai
# 2. Add @lepton provider in model catalog
# 3. Use it:

client = OpenAI(
    api_key="PORTKEY_API_KEY",  # Portkey API key
    base_url=PORTKEY_GATEWAY_URL
)

response = client.chat.completions.create(
    model="@lepton/llama-3.1-8b",
    messages=[{"role": "user", "content": "Hello!"}]
)

print(response.choices[0].message.content)
```

```js OpenAI JS icon="square-js" theme={"system"}
import OpenAI from "openai"
import { PORTKEY_GATEWAY_URL } from "portkey-ai"

// 1. Install: npm install openai portkey-ai
// 2. Add @lepton provider in model catalog
// 3. Use it:

const client = new OpenAI({
    apiKey: "PORTKEY_API_KEY",  // Portkey API key
    baseURL: PORTKEY_GATEWAY_URL
})

const response = await client.chat.completions.create({
    model: "@lepton/llama-3.1-8b",
    messages: [{ role: "user", content: "Hello!" }]
})

console.log(response.choices[0].message.content)
```

```sh cURL icon="square-terminal" theme={"system"}
# 1. Add @lepton provider in model catalog
# 2. Use it:

curl https://api.portkey.ai/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "x-portkey-api-key: $PORTKEY_API_KEY" \
  -d '{
    "model": "@lepton/llama-3.1-8b",
    "messages": [{"role": "user", "content": "Hello!"}]
  }'
```

## Add Provider in Model Catalog

Before making requests, add Lepton AI to your Model Catalog:

1. Go to [**Model Catalog → Add Provider**](https://app.portkey.ai/model-catalog/providers)
2. Select **Lepton AI**
3. Enter your [Lepton API key](https://console.lepton.ai/)
4. Name your provider (e.g., `lepton`)

> [card] [Complete Setup Guide](https://docs.portkey.ai/docs/product/model-catalog.md)
>
> See all setup options and detailed configuration instructions

***

## Lepton AI Capabilities

### Chat Completions

Generate chat completions with Lepton's serverless models:

```python Python theme={"system"}
from portkey_ai import Portkey

portkey = Portkey(api_key="PORTKEY_API_KEY", provider="@lepton")

response = portkey.chat.completions.create(
    model="llama-3.1-8b",
    messages=[{"role": "user", "content": "Write a haiku about AI"}]
)

print(response.choices[0].message.content)
```

```javascript Node.js theme={"system"}
import Portkey from 'portkey-ai';

const portkey = new Portkey({
    apiKey: 'PORTKEY_API_KEY',
    provider: '@lepton'
});

const response = await portkey.chat.completions.create({
    model: "llama-3.1-8b",
    messages: [{ role: "user", content: "Write a haiku about AI" }]
});

console.log(response.choices[0].message.content);
```

### Speech-to-Text

Transcribe audio using Lepton's Whisper models:

```python Python theme={"system"}
from portkey_ai import Portkey

portkey = Portkey(api_key="PORTKEY_API_KEY", provider="@lepton")

with open("audio.mp3", "rb") as audio_file:
    transcription = portkey.audio.transcriptions.create(
        file=audio_file,
        model="whisper-large-v3"
    )

print(transcription.text)
```

```javascript Node.js theme={"system"}
import Portkey from 'portkey-ai';
import fs from 'fs';

const portkey = new Portkey({
    apiKey: 'PORTKEY_API_KEY',
    provider: '@lepton'
});

const transcription = await portkey.audio.transcriptions.create({
    file: fs.createReadStream('audio.mp3'),
    model: 'whisper-large-v3'
});

console.log(transcription.text);
```

### Streaming

Enable streaming for real-time responses:

```python Python theme={"system"}
from portkey_ai import Portkey

portkey = Portkey(api_key="PORTKEY_API_KEY", provider="@lepton")

stream = portkey.chat.completions.create(
    model="llama-3.1-8b",
    messages=[{"role": "user", "content": "Write a story about a robot"}],
    stream=True
)

for chunk in stream:
    print(chunk.choices[0].delta.content, end="")
```

```javascript Node.js theme={"system"}
import Portkey from 'portkey-ai';

const portkey = new Portkey({
    apiKey: 'PORTKEY_API_KEY',
    provider: '@lepton'
});

const stream = await portkey.chat.completions.create({
    model: "llama-3.1-8b",
    messages: [{ role: "user", content: "Write a story about a robot" }],
    stream: true
});

for await (const chunk of stream) {
    process.stdout.write(chunk.choices[0]?.delta?.content || '');
}
```

***

## Supported Models

Lepton AI provides serverless access to various models:

| Model | Description |
| - | - |
| llama-3.1-8b | Llama 3.1 8B model |
| llama-3-8b-sft-v1 | Fine-tuned Llama 3 |
| whisper-large-v3 | Speech-to-text |

Check [Lepton's documentation](https://www.lepton.ai/docs) for the complete model list.

***

## Next Steps

> [card] [Gateway Configs](https://docs.portkey.ai/docs/product/ai-gateway.md)
>
> Add fallbacks, load balancing, and more

> [card] [Observability](https://docs.portkey.ai/docs/product/observability.md)
>
> Monitor and trace your Lepton requests

> [card] [Prompt Library](https://docs.portkey.ai/docs/product/prompt-engineering-studio.md)
>
> Manage and version your prompts

> [card] [Metadata](https://docs.portkey.ai/docs/product/observability/metadata.md)
>
> Add custom metadata to requests

For complete SDK documentation:

> [card] [SDK Reference](https://docs.portkey.ai/docs/api-reference/sdk/list.md)
>
> Complete Portkey SDK documentation

***

> [card] [Portkey is now PRISMA AIRS AI Gateway. See it in action.](https://www.paloaltonetworks.in/ai-security/ai-gateway?utm_source=portkey\&utm_medium=referral\&utm_campaign=prisma_airs\&utm_content=docs_nav#contact)
>
> Contact Us


## Related topics

- [Overview](https://docs.portkey.ai/docs/integrations/llms.md)
- [May](https://docs.portkey.ai/docs/changelog/2025/may.md)
- [Supported Providers](https://docs.portkey.ai/docs/api-reference/inference-api/supported-providers.md)
- [Mistral AI](https://docs.portkey.ai/docs/integrations/llms/mistral-ai.md)
- [Reka AI](https://docs.portkey.ai/docs/integrations/llms/reka-ai.md)


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.