> ## Documentation Index
> Fetch the complete documentation index at: https://docs.portkey.ai/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Fireworks

> Use Fireworks for chat, vision, embeddings, and image generation with advanced grammar and JSON modes through Portkey.

## Quick Start

Get started with Fireworks in under 2 minutes:

```python Python icon="python" theme={"system"}
from portkey_ai import Portkey

# 1. Install: pip install portkey-ai
# 2. Add @fireworks-ai provider in model catalog
# 3. Use it:

portkey = Portkey(api_key="PORTKEY_API_KEY")

response = portkey.chat.completions.create(
    model="@fireworks-ai/accounts/fireworks/models/llama-v3-70b-instruct",
    messages=[{"role": "user", "content": "Hello!"}]
)

print(response.choices[0].message.content)
```

```js Javascript icon="square-js" theme={"system"}
import Portkey from 'portkey-ai'

// 1. Install: npm install portkey-ai
// 2. Add @fireworks-ai provider in model catalog
// 3. Use it:

const portkey = new Portkey({
    apiKey: "PORTKEY_API_KEY"
})

const response = await portkey.chat.completions.create({
    model: "@fireworks-ai/accounts/fireworks/models/llama-v3-70b-instruct",
    messages: [{ role: "user", content: "Hello!" }]
})

console.log(response.choices[0].message.content)
```

```python OpenAI Py icon="python" theme={"system"}
from openai import OpenAI
from portkey_ai import PORTKEY_GATEWAY_URL

# 1. Install: pip install openai portkey-ai
# 2. Add @fireworks-ai provider in model catalog
# 3. Use it:

client = OpenAI(
    api_key="PORTKEY_API_KEY",  # Portkey API key
    base_url=PORTKEY_GATEWAY_URL
)

response = client.chat.completions.create(
    model="@fireworks-ai/accounts/fireworks/models/llama-v3-70b-instruct",
    messages=[{"role": "user", "content": "Hello!"}]
)

print(response.choices[0].message.content)
```

```js OpenAI JS icon="square-js" theme={"system"}
import OpenAI from "openai"
import { PORTKEY_GATEWAY_URL } from "portkey-ai"

// 1. Install: npm install openai portkey-ai
// 2. Add @fireworks-ai provider in model catalog
// 3. Use it:

const client = new OpenAI({
    apiKey: "PORTKEY_API_KEY",  // Portkey API key
    baseURL: PORTKEY_GATEWAY_URL
})

const response = await client.chat.completions.create({
    model: "@fireworks-ai/accounts/fireworks/models/llama-v3-70b-instruct",
    messages: [{ role: "user", content: "Hello!" }]
})

console.log(response.choices[0].message.content)
```

```sh cURL icon="square-terminal" theme={"system"}
# 1. Add @fireworks-ai provider in model catalog
# 2. Use it:

curl https://api.portkey.ai/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "x-portkey-api-key: $PORTKEY_API_KEY" \
  -d '{
    "model": "@fireworks-ai/accounts/fireworks/models/llama-v3-70b-instruct",
    "messages": [{"role": "user", "content": "Hello!"}]
  }'
```

## Add Provider in Model Catalog

Before making requests, add Fireworks to your Model Catalog:

1. Go to [**Model Catalog → Add Provider**](https://app.portkey.ai/model-catalog/providers)
2. Select **Fireworks**
3. Enter your [Fireworks API key](https://fireworks.ai/api-keys)
4. Name your provider (e.g., `fireworks-ai`)

> [card] [Complete Setup Guide](https://docs.portkey.ai/docs/product/model-catalog.md)
>
> See all setup options and detailed configuration instructions

***

## Fireworks Capabilities

### Embeddings

Generate embeddings using Fireworks models:

```python Python theme={"system"}
from portkey_ai import Portkey

portkey = Portkey(api_key="PORTKEY_API_KEY", provider="@fireworks-ai")

embeddings = portkey.embeddings.create(
    model="thenlper/gte-large",
    input="create vector representation on this sentence"
)

print(embeddings.data[0].embedding)
```

```javascript Node.js theme={"system"}
import Portkey from 'portkey-ai';

const portkey = new Portkey({
    apiKey: 'PORTKEY_API_KEY',
    provider: '@fireworks-ai'
});

const embeddings = await portkey.embeddings.create({
    model: "thenlper/gte-large",
    input: "create vector representation on this sentence"
});

console.log(embeddings.data[0].embedding);
```

### Vision Models

Process images with vision models:

```python Python theme={"system"}
from portkey_ai import Portkey

portkey = Portkey(api_key="PORTKEY_API_KEY", provider="@fireworks-ai")

response = portkey.chat.completions.create(
    model="accounts/fireworks/models/firellava-13b",
    messages=[{
        "role": "user",
        "content": [
            {"type": "text", "text": "Can you describe this image?"},
            {
                "type": "image_url",
                "image_url": {
                    "url": "https://images.unsplash.com/photo-1582538885592-e70a5d7ab3d3"
                }
            }
        ]
    }]
)

print(response.choices[0].message.content)
```

```javascript Node.js theme={"system"}
import Portkey from 'portkey-ai';

const portkey = new Portkey({
    apiKey: 'PORTKEY_API_KEY',
    provider: '@fireworks-ai'
});

const response = await portkey.chat.completions.create({
    model: "accounts/fireworks/models/firellava-13b",
    messages: [{
        role: "user",
        content: [
            { type: "text", text: "Can you describe this image?" },
            {
                type: "image_url",
                image_url: {
                    url: "https://images.unsplash.com/photo-1582538885592-e70a5d7ab3d3"
                }
            }
        ]
    }]
});

console.log(response.choices[0].message.content);
```

### Image Generation

Generate images with Stable Diffusion:

```python Python theme={"system"}
from portkey_ai import Portkey
import base64
from io import BytesIO
from PIL import Image

portkey = Portkey(api_key="PORTKEY_API_KEY", provider="@fireworks-ai")

image = portkey.images.generate(
    model="accounts/fireworks/models/stable-diffusion-xl-1024-v1-0",
    prompt="An orange elephant in a purple pond"
)

Image.open(BytesIO(base64.b64decode(image.data[0].b64_json))).save("generated.png")
```

```javascript Node.js theme={"system"}
import Portkey from 'portkey-ai';
import fs from 'fs';

const portkey = new Portkey({
    apiKey: 'PORTKEY_API_KEY',
    provider: '@fireworks-ai'
});

const image = await portkey.images.generate({
    model: "accounts/fireworks/models/stable-diffusion-xl-1024-v1-0",
    prompt: "An orange elephant in a purple pond"
});

const imageData = image.data[0].b64_json;
fs.writeFileSync("generated.png", Buffer.from(imageData, 'base64'));
```

### Function Calling

Use function calling with Fireworks models:

> [card] [Function Calling Guide](https://docs.portkey.ai/docs/guides/getting-started/function-calling.md)
>
> Explore function calling examples and best practices

### Grammar Mode

Fireworks lets you define [formal grammars](https://en.wikipedia.org/wiki/Formal%5Fgrammar) to constrain model outputs. You can use it to force the model to generate valid JSON, speak only in emojis, or anything else. ([Originally created by GGML](https://github.com/ggerganov/llama.cpp/tree/master/grammars))

Grammar mode is set with the `response_format` param. Just pass your grammar definition with `{"type": "grammar", "grammar": grammar_definition}`

Let's say you want to classify patient requests into 3 pre-defined classes:

```python Python theme={"system"}
from portkey_ai import Portkey

portkey = Portkey(api_key="PORTKEY_API_KEY", provider="@fireworks-ai")

patient_classification = """
root      ::= diagnosis
diagnosis ::= "flu" | "dengue" | "malaria"
"""

response = portkey.chat.completions.create(
    model="accounts/fireworks/models/llama-v3-70b-instruct",
    messages=[{"role": "user", "content": "Patient has fever and chills"}],
    response_format={"type": "grammar", "grammar": patient_classification}
)

print(response.choices[0].message.content)
```

```javascript Node.js theme={"system"}
import Portkey from 'portkey-ai';

const portkey = new Portkey({
    apiKey: 'PORTKEY_API_KEY',
    provider: '@fireworks-ai'
});

const patient_classification = `
root      ::= diagnosis
diagnosis ::= "flu" | "dengue" | "malaria"
`;

const response = await portkey.chat.completions.create({
    model: "accounts/fireworks/models/llama-v3-70b-instruct",
    messages: [{ role: "user", content: "Patient has fever and chills" }],
    response_format: { type: "grammar", grammar: patient_classification }
});

console.log(response.choices[0].message.content);
```

> [card] [Fireworks Grammar Guide](https://readme.fireworks.ai/docs/structured-output-grammar-based)
>
> Learn more about grammar mode and examples

### JSON Mode

Force JSON output with optional schema validation:

```python Python theme={"system"}
from portkey_ai import Portkey
from pydantic import BaseModel
from typing import List

portkey = Portkey(api_key="PORTKEY_API_KEY", provider="@fireworks-ai")

class Recipe(BaseModel):
    title: str
    description: str
    steps: List[str]

response = portkey.chat.completions.create(
    model="accounts/fireworks/models/llama-v3-70b-instruct",
    messages=[{"role": "user", "content": "Give me a recipe for making Ramen"}],
    response_format={
        "type": "json_object",
        "schema": Recipe.schema_json()
    }
)

print(response.choices[0].message.content)
```

```javascript Node.js theme={"system"}
import Portkey from 'portkey-ai';

const portkey = new Portkey({
    apiKey: 'PORTKEY_API_KEY',
    provider: '@fireworks-ai'
});

const response = await portkey.chat.completions.create({
    model: "accounts/fireworks/models/llama-v3-70b-instruct",
    messages: [{ role: "user", content: "Give me a recipe for making Ramen" }],
    response_format: {
        type: "json_object",
        schema: {
            type: "object",
            properties: {
                title: { type: "string" },
                description: { type: "string" },
                steps: { type: "array" }
            }
        }
    }
});

console.log(response.choices[0].message.content);
```

> [card] [Fireworks JSON Mode Guide](https://readme.fireworks.ai/docs/structured-response-formatting)
>
> Learn more about JSON mode

***

## Supported Models

> [card] [Fireworks Models](https://readme.fireworks.ai/docs/querying-text-models)
>
> View the complete list of models available on Fireworks

***

## Next Steps

> [card] [Gateway Configs](https://docs.portkey.ai/docs/product/ai-gateway.md)
>
> Add fallbacks, load balancing, and more

> [card] [Observability](https://docs.portkey.ai/docs/product/observability.md)
>
> Monitor and trace your Fireworks requests

> [card] [Prompt Library](https://docs.portkey.ai/docs/product/prompt-engineering-studio.md)
>
> Manage and version your prompts

> [card] [Metadata](https://docs.portkey.ai/docs/product/observability/metadata.md)
>
> Add custom metadata to requests

For complete SDK documentation:

> [card] [SDK Reference](https://docs.portkey.ai/docs/api-reference/sdk/list.md)
>
> Complete Portkey SDK documentation

***

> [card] [Portkey is now PRISMA AIRS AI Gateway. See it in action.](https://www.paloaltonetworks.in/ai-security/ai-gateway?utm_source=portkey\&utm_medium=referral\&utm_campaign=prisma_airs\&utm_content=docs_nav#contact)
>
> Contact Us


## Related topics

- [Comparing DeepSeek Models Against OpenAI, Anthropic & More Using Portkey](https://docs.portkey.ai/docs/guides/use-cases/deepseek-r1.md)
- [Files](https://docs.portkey.ai/docs/integrations/llms/fireworks/files.md)
- [Fine-tune](https://docs.portkey.ai/docs/integrations/llms/fireworks/fine-tuning.md)
- [Data Service](https://docs.portkey.ai/docs/changelog/data-service.md)
- [Nitro Mode (Beta)](https://docs.portkey.ai/docs/product/ai-gateway/nitro-mode.md)


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.