> ## Documentation Index
> Fetch the complete documentation index at: https://docs.portkey.ai/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Together AI

> Integrate Together AI models with Portkey's AI Gateway

Portkey provides a robust and secure gateway to integrate various Large Language Models (LLMs) into applications, including [Together AI's hosted models](https://docs.together.ai/reference/inference).

With Portkey, take advantage of features like fast AI gateway access, observability, prompt management, and more, while securely managing API keys through [Model Catalog](https://docs.portkey.ai/docs/product/model-catalog.md).

## Quick Start

Get Together AI working in 3 steps:

```python Python icon="python" theme={"system"}
from portkey_ai import Portkey

# 1. Install: pip install portkey-ai
# 2. Add @together-ai provider in model catalog
# 3. Use it:

portkey = Portkey(api_key="PORTKEY_API_KEY")

response = portkey.chat.completions.create(
    model="@together-ai/meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo",
    messages=[{"role": "user", "content": "Say this is a test"}]
)

print(response.choices[0].message.content)
```

```js Javascript icon="square-js" theme={"system"}
import Portkey from 'portkey-ai'

// 1. Install: npm install portkey-ai
// 2. Add @together-ai provider in model catalog
// 3. Use it:

const portkey = new Portkey({
    apiKey: "PORTKEY_API_KEY"
})

const response = await portkey.chat.completions.create({
    model: "@together-ai/meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo",
    messages: [{ role: "user", content: "Say this is a test" }]
})

console.log(response.choices[0].message.content)
```

```python OpenAI Py icon="python" theme={"system"}
from openai import OpenAI
from portkey_ai import PORTKEY_GATEWAY_URL

# 1. Install: pip install openai portkey-ai
# 2. Add @together-ai provider in model catalog
# 3. Use it:

client = OpenAI(
    api_key="PORTKEY_API_KEY",  # Portkey API key
    base_url=PORTKEY_GATEWAY_URL
)

response = client.chat.completions.create(
    model="@together-ai/meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo",
    messages=[{"role": "user", "content": "Say this is a test"}]
)

print(response.choices[0].message.content)
```

```js OpenAI JS icon="square-js" theme={"system"}
import OpenAI from "openai"
import { PORTKEY_GATEWAY_URL } from "portkey-ai"

// 1. Install: npm install openai portkey-ai
// 2. Add @together-ai provider in model catalog
// 3. Use it:

const client = new OpenAI({
    apiKey: "PORTKEY_API_KEY",  // Portkey API key
    baseURL: PORTKEY_GATEWAY_URL
})

const response = await client.chat.completions.create({
    model: "@together-ai/meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo",
    messages: [{ role: "user", content: "Say this is a test" }]
})

console.log(response.choices[0].message.content)
```

```sh cURL icon="square-terminal" theme={"system"}
# 1. Add @together-ai provider in model catalog
# 2. Use it:

curl https://api.portkey.ai/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "x-portkey-api-key: $PORTKEY_API_KEY" \
  -d '{
    "model": "@together-ai/meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo",
    "messages": [
      { "role": "user", "content": "Say this is a test" }
    ]
  }'
```

> [!NOTE]
>
> **Tip:** You can also set `provider="@together-ai"` in `Portkey()` and use just `model="meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo"` in the request.

## Add Provider in Model Catalog

1. Go to [**Model Catalog → Add Provider**](https://app.portkey.ai/model-catalog/providers)
2. Select **Together AI**
3. Choose existing credentials or create new by entering your [Together AI API key](https://api.together.ai/settings/api-keys)
4. Name your provider (e.g., `together-ai-prod`)

> [card] [Complete Setup Guide →](https://docs.portkey.ai/docs/product/model-catalog.md)
>
> See all setup options, code examples, and detailed instructions

## Reasoning / Thinking Support

Together AI supports reasoning models that expose their internal chain of thought. Use the `reasoning_effort` parameter to control reasoning behavior, and set `strict_open_ai_compliance=False` to receive the thinking content in `content_blocks`.

```python Python theme={"system"}
from portkey_ai import Portkey

portkey = Portkey(
    api_key="PORTKEY_API_KEY",
    strict_open_ai_compliance=False
)

response = portkey.chat.completions.create(
    model="@together-ai/deepseek-ai/DeepSeek-R1",
    messages=[{"role": "user", "content": "Solve step by step: What is 23 * 47?"}],
    reasoning_effort="medium"
)

# Access thinking content from content_blocks
for block in response.choices[0].message.content_blocks:
    if block.get("type") == "thinking":
        print("Thinking:", block["thinking"])
    elif block.get("type") == "text":
        print("Response:", block["text"])
```

```js Javascript theme={"system"}
import Portkey from 'portkey-ai'

const portkey = new Portkey({
    apiKey: "PORTKEY_API_KEY",
    strictOpenAiCompliance: false
})

const response = await portkey.chat.completions.create({
    model: "@together-ai/deepseek-ai/DeepSeek-R1",
    messages: [{ role: "user", content: "Solve step by step: What is 23 * 47?" }],
    reasoning_effort: "medium"
})

// Access thinking content from content_blocks
for (const block of response.choices[0].message.content_blocks) {
    if (block.type === "thinking") {
        console.log("Thinking:", block.thinking)
    } else if (block.type === "text") {
        console.log("Response:", block.text)
    }
}
```

```sh cURL theme={"system"}
curl https://api.portkey.ai/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "x-portkey-api-key: $PORTKEY_API_KEY" \
  -H "x-portkey-strict-open-ai-compliance: false" \
  -d '{
    "model": "@together-ai/deepseek-ai/DeepSeek-R1",
    "messages": [
      { "role": "user", "content": "Solve step by step: What is 23 * 47?" }
    ],
    "reasoning_effort": "medium"
  }'
```

The reasoning response includes `content_blocks` with both the model's thinking process and the final answer. Streaming is also supported and returns reasoning chunks in the `content_blocks` field of the stream delta.

> [card] [Thinking Mode Documentation](https://docs.portkey.ai/docs/product/ai-gateway/multimodal-capabilities/thinking-mode.md)
>
> Learn more about thinking/reasoning support across providers

## Video Generation

Together AI supports video generation through Portkey's proxy. Generate videos from text prompts and track usage in your Portkey dashboard.

```python Python theme={"system"}
from portkey_ai import Portkey

portkey = Portkey(
    api_key="PORTKEY_API_KEY",
    provider="@together-ai"
)

body = {
    "prompt": "A serene sunset over the ocean with gentle waves",
    "model": "minimax/hailuo-02",
    "width": 1366,
    "height": 768
}

# Request to generate video
response = portkey.post(
    url="/v2/videos",
    **body
)

# Request to retrieve video status
response = portkey.get(
    path="/v2/videos/<video_id>",
)
```

```js Javascript icon="square-js" theme={"system"}
import Portkey from 'portkey-ai'

const portkey = new Portkey({
    apiKey: "PORTKEY_API_KEY",
    provider: "@together-ai"
})

const body = {
    prompt: "A serene sunset over the ocean with gentle waves",
    model: "minimax/hailuo-02",
    width: 1366,
    height: 768
}

// Request to generate video
const response = await portkey.post(
    "/v2/videos",
    body
)

// Request to retrieve video status (use the video id from the create response)
// const statusResponse = await portkey.get(`/v2/videos/${videoId}`)
```

```sh cURL theme={"system"}
curl --location 'https://api.portkey.ai/v1/v2/videos' \
  --header 'x-portkey-provider: @together-ai' \
  --header 'Content-Type: application/json' \
  --header 'x-portkey-api-key: PORTKEY_API_KEY' \
  --data '{
    "prompt": "A serene sunset over the ocean with gentle waves",
    "model": "minimax/hailuo-02",
    "width": 1366,
    "height": 768
  }'
```

Video generation pricing is logged in your Portkey dashboard for cost tracking.

***

## Managing Together AI Prompts

Manage all prompt templates to Together AI in the [Prompt Library](https://docs.portkey.ai/docs/product/prompt-library.md). All current Together AI models are supported, and you can easily test different prompts.

Use the `portkey.prompts.completions.create` interface to use the prompt in an application.

## Next Steps

> [card] [Add Metadata](https://docs.portkey.ai/docs/product/observability/metadata.md)
>
> Add metadata to your Together AI requests

> [card] [Gateway Configs](https://docs.portkey.ai/docs/product/ai-gateway/configs.md)
>
> Add gateway configs to your Together AI requests

> [card] [Tracing](https://docs.portkey.ai/docs/product/observability/traces.md)
>
> Trace your Together AI requests

> [card] [Fallbacks](https://docs.portkey.ai/docs/product/ai-gateway/fallbacks.md)
>
> Setup fallback from OpenAI to Together AI

For complete SDK documentation:

> [card] [SDK Reference](https://docs.portkey.ai/docs/api-reference/portkey-sdk-client)
>
> Complete Portkey SDK documentation

***

> [card] [Portkey is now PRISMA AIRS AI Gateway. See it in action.](https://www.paloaltonetworks.in/ai-security/ai-gateway?utm_source=portkey\&utm_medium=referral\&utm_campaign=prisma_airs\&utm_content=docs_nav#contact)
>
> Contact Us


## Related topics

- [Comparing DeepSeek Models Against OpenAI, Anthropic & More Using Portkey](https://docs.portkey.ai/docs/guides/use-cases/deepseek-r1.md)
- [Llama 3 on Portkey + Together AI](https://docs.portkey.ai/docs/guides/integrations/llama-3-on-portkey-+-together-ai.md)
- [Enforcing JSON Schema with Anyscale & Together](https://docs.portkey.ai/docs/guides/use-cases/enforcing-json-schema-with-anyscale-and-together.md)
- [Mixtral 8x22b](https://docs.portkey.ai/docs/guides/integrations/mixtral-8x22b.md)
- [Comparing Top10 LMSYS Models with Portkey](https://docs.portkey.ai/docs/guides/use-cases/comparing-top10-lmsys-models-with-portkey.md)


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.