SambaNova

Use SambaNova’s fast inference for Llama and other open-source models through Portkey.

Quick Start

Get started with SambaNova in under 2 minutes:

Python

from portkey_ai import Portkey

# 1. Install: pip install portkey-ai
# 2. Add @sambanova provider in model catalog
# 3. Use it:

portkey = Portkey(api_key="PORTKEY_API_KEY")

response = portkey.chat.completions.create(
    model="@sambanova/Meta-Llama-3.1-405B-Instruct",
    messages=[{"role": "user", "content": "Hello!"}]
)

print(response.choices[0].message.content)

Add Provider in Model Catalog

Before making requests, add SambaNova to your Model Catalog:

1. Go to Model Catalog → Add Provider
2. Select SambaNova
3. Enter your SambaNova API key
4. Name your provider (e.g., sambanova)

Complete Setup Guide

See all setup options and detailed configuration instructions

---

Supported Models

SambaNova provides fast inference for open-source models:

Supported Models

View the complete list of SambaNova models

Popular models include:

• Meta-Llama-3.1-405B-Instruct
• Meta-Llama-3.1-70B-Instruct
• Meta-Llama-3.1-8B-Instruct

---

Next Steps

Gateway Configs

Add fallbacks, load balancing, and more

Observability

Monitor and trace your SambaNova requests

Prompt Library

Manage and version your prompts

Metadata

Add custom metadata to requests

For complete SDK documentation:

SDK Reference

Complete Portkey SDK documentation