> ## Documentation Index
> Fetch the complete documentation index at: https://docs.portkey.ai/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Embeddings

> Get embeddings from Vertex AI

Vertex AI offers wide ranging support for embedding text, images and videos.
The Prisma AIRS AI Gateway provides a standardized interface for embedding multiple modalities.

***

## Gemini Embedding Models

The `gemini-embedding-2-preview` model supports embedding across multiple modalities — **text**, **image**, **video**, and **audio** — through a single unified endpoint.

| Input Type | Supported Formats |
| - | - |
| Text | Plain string or structured object |
| Image | GCS URI, HTTPS URL, base64, data URI |
| Video | GCS URI, HTTPS URL, base64 |
| Audio | GCS URI, HTTPS URL, base64 |

Additional supported parameters: `task_type`, `dimensions`

### Embedding Text

```sh cURL theme={"system"}
curl --location 'https://aigw.portkey.ai/v1/embeddings' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer PORTKEY_API_KEY' \
--header 'x-portkey-vertex-region: us-central1' \
--data '{
    "model": "@PORTKEY_PROVIDER/gemini-embedding-2-preview",
    "input": "What is the meaning of life?"
}'
```

```python OpenAI Python theme={"system"}
from openai import OpenAI

gateway_client = OpenAI(
    api_key="PORTKEY_API_KEY",
    base_url="https://aigw.portkey.ai/v1",
    default_headers={"x-portkey-vertex-region": "us-central1"}
)

embeddings = gateway_client.embeddings.create(
    model="@PORTKEY_PROVIDER/gemini-embedding-2-preview",
    input="What is the meaning of life?",
)
```

```js OpenAI NodeJS theme={"system"}
import OpenAI from 'openai';

const gatewayClient = new OpenAI({
  apiKey: "PORTKEY_API_KEY",
  baseURL: "https://aigw.portkey.ai/v1",
  defaultHeaders: { "x-portkey-vertex-region": "us-central1" }
});

const embedding = await gatewayClient.embeddings.create({
    input: "What is the meaning of life?",
    model: "@PORTKEY_PROVIDER/gemini-embedding-2-preview",
});

console.log(embedding);
```

### Embedding Images

```sh cURL theme={"system"}
curl --location 'https://aigw.portkey.ai/v1/embeddings' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer PORTKEY_API_KEY' \
--header 'x-portkey-vertex-region: us-central1' \
--data '{
    "model": "@PORTKEY_PROVIDER/gemini-embedding-2-preview",
    "input": [
        {
            "image": {
                "url": "gs://your-bucket/image.png",
                "mime_type": "image/png"
            }
        }
    ]
}'
```

```python OpenAI Python theme={"system"}
from openai import OpenAI

gateway_client = OpenAI(
    api_key="PORTKEY_API_KEY",
    base_url="https://aigw.portkey.ai/v1",
    default_headers={"x-portkey-vertex-region": "us-central1"}
)

embeddings = gateway_client.embeddings.create(
    model="@PORTKEY_PROVIDER/gemini-embedding-2-preview",
    input=[
        {
            "image": {
                "url": "gs://your-bucket/image.png",
                "mime_type": "image/png"
            }
        }
    ],
)
```

```js OpenAI NodeJS theme={"system"}
import OpenAI from 'openai';

const gatewayClient = new OpenAI({
  apiKey: "PORTKEY_API_KEY",
  baseURL: "https://aigw.portkey.ai/v1",
  defaultHeaders: { "x-portkey-vertex-region": "us-central1" }
});

const embedding = await gatewayClient.embeddings.create({
    input: [
        {
            image: {
                url: "gs://your-bucket/image.png",
                mime_type: "image/png"
            }
        }
    ],
    model: "@PORTKEY_PROVIDER/gemini-embedding-2-preview",
});

console.log(embedding);
```

You can also pass images as base64:

```json theme={"system"}
{
    "input": [
        {
            "image": {
                "base64": "iVBORw0KGgoAAAANSUhEUgAA...",
                "mime_type": "image/png"
            }
        }
    ]
}
```

Or as a data URI:

```json theme={"system"}
{
    "input": [
        {
            "image": {
                "url": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAA..."
            }
        }
    ]
}
```

### Embedding Videos

```sh cURL theme={"system"}
curl --location 'https://aigw.portkey.ai/v1/embeddings' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer PORTKEY_API_KEY' \
--header 'x-portkey-vertex-region: us-central1' \
--data '{
    "model": "@PORTKEY_PROVIDER/gemini-embedding-2-preview",
    "input": [
        {
            "video": {
                "url": "gs://cloud-samples-data/generative-ai/video/pixel8.mp4",
                "mime_type": "video/mp4"
            }
        }
    ]
}'
```

```python OpenAI Python theme={"system"}
from openai import OpenAI

gateway_client = OpenAI(
    api_key="PORTKEY_API_KEY",
    base_url="https://aigw.portkey.ai/v1",
    default_headers={"x-portkey-vertex-region": "us-central1"}
)

embeddings = gateway_client.embeddings.create(
    model="@PORTKEY_PROVIDER/gemini-embedding-2-preview",
    input=[
        {
            "video": {
                "url": "gs://cloud-samples-data/generative-ai/video/pixel8.mp4",
                "mime_type": "video/mp4"
            }
        }
    ],
)
```

```js OpenAI NodeJS theme={"system"}
import OpenAI from 'openai';

const gatewayClient = new OpenAI({
  apiKey: "PORTKEY_API_KEY",
  baseURL: "https://aigw.portkey.ai/v1",
  defaultHeaders: { "x-portkey-vertex-region": "us-central1" }
});

const embedding = await gatewayClient.embeddings.create({
    input: [
        {
            video: {
                url: "gs://cloud-samples-data/generative-ai/video/pixel8.mp4",
                mime_type: "video/mp4"
            }
        }
    ],
    model: "@PORTKEY_PROVIDER/gemini-embedding-2-preview",
});

console.log(embedding);
```

### Embedding Audio

```sh cURL theme={"system"}
curl --location 'https://aigw.portkey.ai/v1/embeddings' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer PORTKEY_API_KEY' \
--header 'x-portkey-vertex-region: us-central1' \
--data '{
    "model": "@PORTKEY_PROVIDER/gemini-embedding-2-preview",
    "input": [
        {
            "audio": {
                "url": "gs://cloud-samples-data/generative-ai/audio/Chirp-3-Docs-Dive.mp3",
                "mime_type": "audio/mpeg"
            }
        }
    ]
}'
```

```python OpenAI Python theme={"system"}
from openai import OpenAI

gateway_client = OpenAI(
    api_key="PORTKEY_API_KEY",
    base_url="https://aigw.portkey.ai/v1",
    default_headers={"x-portkey-vertex-region": "us-central1"}
)

embeddings = gateway_client.embeddings.create(
    model="@PORTKEY_PROVIDER/gemini-embedding-2-preview",
    input=[
        {
            "audio": {
                "url": "gs://cloud-samples-data/generative-ai/audio/Chirp-3-Docs-Dive.mp3",
                "mime_type": "audio/mpeg"
            }
        }
    ],
)
```

```js OpenAI NodeJS theme={"system"}
import OpenAI from 'openai';

const gatewayClient = new OpenAI({
  apiKey: "PORTKEY_API_KEY",
  baseURL: "https://aigw.portkey.ai/v1",
  defaultHeaders: { "x-portkey-vertex-region": "us-central1" }
});

const embedding = await gatewayClient.embeddings.create({
    input: [
        {
            audio: {
                url: "gs://cloud-samples-data/generative-ai/audio/Chirp-3-Docs-Dive.mp3",
                mime_type: "audio/mpeg"
            }
        }
    ],
    model: "@PORTKEY_PROVIDER/gemini-embedding-2-preview",
});

console.log(embedding);
```

### Multimodal Embedding (Mixed Inputs)

You can combine multiple input types in a single request:

```sh cURL theme={"system"}
curl --location 'https://aigw.portkey.ai/v1/embeddings' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer PORTKEY_API_KEY' \
--header 'x-portkey-vertex-region: us-central1' \
--data '{
    "model": "@PORTKEY_PROVIDER/gemini-embedding-2-preview",
    "input": [
        {
            "video": {
                "url": "gs://cloud-samples-data/generative-ai/video/pixel8.mp4",
                "mime_type": "video/mp4"
            }
        },
        {
            "audio": {
                "url": "gs://cloud-samples-data/generative-ai/audio/Chirp-3-Docs-Dive.mp3",
                "mime_type": "audio/mpeg"
            }
        }
    ]
}'
```

```python OpenAI Python theme={"system"}
from openai import OpenAI

gateway_client = OpenAI(
    api_key="PORTKEY_API_KEY",
    base_url="https://aigw.portkey.ai/v1",
    default_headers={"x-portkey-vertex-region": "us-central1"}
)

embeddings = gateway_client.embeddings.create(
    model="@PORTKEY_PROVIDER/gemini-embedding-2-preview",
    input=[
        {
            "video": {
                "url": "gs://cloud-samples-data/generative-ai/video/pixel8.mp4",
                "mime_type": "video/mp4"
            }
        },
        {
            "audio": {
                "url": "gs://cloud-samples-data/generative-ai/audio/Chirp-3-Docs-Dive.mp3",
                "mime_type": "audio/mpeg"
            }
        }
    ],
)
```

```js OpenAI NodeJS theme={"system"}
import OpenAI from 'openai';

const gatewayClient = new OpenAI({
  apiKey: "PORTKEY_API_KEY",
  baseURL: "https://aigw.portkey.ai/v1",
  defaultHeaders: { "x-portkey-vertex-region": "us-central1" }
});

const embedding = await gatewayClient.embeddings.create({
    input: [
        {
            video: {
                url: "gs://cloud-samples-data/generative-ai/video/pixel8.mp4",
                mime_type: "video/mp4"
            }
        },
        {
            audio: {
                url: "gs://cloud-samples-data/generative-ai/audio/Chirp-3-Docs-Dive.mp3",
                mime_type: "audio/mpeg"
            }
        }
    ],
    model: "@PORTKEY_PROVIDER/gemini-embedding-2-preview",
});

console.log(embedding);
```

### Setting Task Type and Dimensions

You can optionally specify `task_type` and `dimensions` to control the embedding behaviour:

```json theme={"system"}
{
    "model": "gemini-embedding-2-preview",
    "input": "What is the meaning of life?",
    "task_type": "RETRIEVAL_DOCUMENT",
    "dimensions": 768
}
```

***

## Legacy Embedding Models

The following sections cover the older Vertex AI embedding models like `textembedding-gecko@003` and `multimodalembedding@001`.

## Embedding Text

```sh cURL theme={"system"}
curl --location 'https://aigw.portkey.ai/v1/embeddings' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer PORTKEY_API_KEY' \
--header 'x-portkey-provider: PORTKEY_PROVIDER' \
--data-raw '{
    "model": "textembedding-gecko@003",
    "input": [
        "A HTTP 246 code is used to signify an AI response containing hallucinations or other inaccuracies",
        "246: Partially incorrect response"
    ],
    # "input": "Name the tallest buildings in Hawaii",
    "input_type": "classification"
}'
```

```python OpenAI Python theme={"system"}
from openai import OpenAI

gateway_client = OpenAI(
    api_key="PORTKEY_API_KEY",
    base_url="https://aigw.portkey.ai/v1",
    default_headers={"x-portkey-provider": "openai"}
)

embeddings = gateway_client.embeddings.create(
  model="textembedding-gecko@003",
  input_type="classification",
  input="The food was delicious and the waiter...",
  # input=["text to embed", "more text to embed"], # if you would like to embed multiple texts
)
```

```js OpenAI NodeJS theme={"system"}
import OpenAI from 'openai'; // We're using the v4 SDK

const gatewayClient = new OpenAI({
  apiKey: "PORTKEY_API_KEY", // defaults to process.env["OPENAI_API_KEY"],
  baseURL: "https://aigw.portkey.ai/v1"
});

const embedding = await gatewayClient.embeddings.create({
    input: 'Name the tallest buildings in Hawaii',
    // input: ['text to embed', 'more text to embed'], // if you would like to embed multiple texts
    model: '@PORTKEY_PROVIDER/textembedding-gecko@003'
});

console.log(embedding);
```

## Embeddings Images

```sh cURL theme={"system"}
curl --location 'https://aigw.portkey.ai/v1/embeddings' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer PORTKEY_API_KEY' \
--header 'x-portkey-provider: PORTKEY_PROVIDER' \
--data-raw '{
    "model": "multimodalembedding@001",
    "input": [
                  {
                      "text": "this is the caption of the image",
                      "image": {
                          "base64": "UklGRkacAABXRUJQVlA4IDqcAACQggKdASqpAn8B....."
                          # "url": "gcs://..." # if you want to use a url
                      }
                  }
              ]
}'
```

```python OpenAI Python theme={"system"}
from openai import OpenAI

gateway_client = OpenAI(
    api_key="PORTKEY_API_KEY",
    base_url="https://aigw.portkey.ai/v1",
    default_headers={"x-portkey-provider": "openai"}
)

embeddings = gateway_client.embeddings.create(
  model="multimodalembedding@001",
  input=[
          {
              "text": "this is the caption of the image",
              "image": {
                  "base64": "UklGRkacAABXRUJQVlA4IDqcAACQggKdASqpAn8B.....",
                  # "url": "gcs://..." # if you want to use a url
              }
          }
        ]
)
```

```js OpenAI NodeJS theme={"system"}
import OpenAI from 'openai'; // We're using the v4 SDK

const gatewayClient = new OpenAI({
  apiKey: "PORTKEY_API_KEY", // defaults to process.env["OPENAI_API_KEY"],
  baseURL: "https://aigw.portkey.ai/v1"
});

const embedding = await gatewayClient.embeddings.create({
    input: [
              {
                  "text": "this is the caption of the image",
                  "image": {
                      "base64": "UklGRkacAABXRUJQVlA4IDqcAACQggKdASqpAn8B.....",
                      // "url": "gcs://..." // if you want to use a url
                  }
              }
            ],
    model: '@PORTKEY_PROVIDER/multimodalembedding@001'
});

console.log(embedding);
```

## Embeddings Videos

```sh cURL theme={"system"}
curl --location 'https://aigw.portkey.ai/v1/embeddings' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer PORTKEY_API_KEY' \
--header 'x-portkey-provider: PORTKEY_PROVIDER' \
--data-raw '{
    "model": "multimodalembedding@001",
    "input": [
                  {
                      "text": "this is the caption of the video",
                      "video": {
                          "base64": "UklGRkacAABXRUJQVlA4IDqcAACQggKdASqpAn8B.....",
                          "start_offset": 0,
                          "end_offset": 10,
                          "interval": 5
                          # "url": "gcs://..." # if you want to use a url
                      }
                  }
              ]
}'
```

```python OpenAI Python theme={"system"}
from openai import OpenAI

gateway_client = OpenAI(
    api_key="PORTKEY_API_KEY",
    base_url="https://aigw.portkey.ai/v1",
    default_headers={"x-portkey-provider": "openai"}
)

embeddings = gateway_client.embeddings.create(
  model="multimodalembedding@001",
  input=[
          {
              "text": "this is the caption of the video",
              "video": {
                  "base64": "UklGRkacAABXRUJQVlA4IDqcAACQggKdASqpAn8B.....",
                  "start_offset": 0,
                  "end_offset": 10,
                  "interval": 5,
                  # "url": "gcs://..." # if you want to use a url
              }
          }
        ]
)
```

```js OpenAI NodeJS theme={"system"}
import OpenAI from 'openai'; // We're using the v4 SDK

const gatewayClient = new OpenAI({
  apiKey: "PORTKEY_API_KEY", // defaults to process.env["OPENAI_API_KEY"],
  baseURL: "https://aigw.portkey.ai/v1"
});

const embedding = await gatewayClient.embeddings.create({
    input: [
              {
                  "text": "this is the caption of the video",
                  "video": {
                      "base64": "UklGRkacAABXRUJQVlA4IDqcAACQggKdASqpAn8B.....",
                      "start_offset": 0,
                      "end_offset": 10,
                      "interval": 5,
                      // "url": "gcs://..." // if you want to use a url
                  }
              }
            ],
    model: '@PORTKEY_PROVIDER/multimodalembedding@001'
});

console.log(embedding);
```


## Related topics

- [Embeddings](https://docs.portkey.ai/docs/aigw/integrations/llms/openai/embeddings.md)
- [Create embedding](https://docs.portkey.ai/docs/aigw/api-reference/embeddings/create-embedding.md)
- [Guardrails for Embedding Requests](https://docs.portkey.ai/docs/aigw/product/guardrails/embedding-guardrails.md)
- [Langchain (JS/TS)](https://docs.portkey.ai/docs/aigw/integrations/libraries/langchain-js.md)


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.