> ## Documentation Index
> Fetch the complete documentation index at: https://docs.portkey.ai/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Batches

> Perform batch inference with OpenAI

The Prisma AIRS AI Gateway exposes [OpenAI’s Batch API](https://platform.openai.com/docs/guides/batch) through one consistent endpoint, so you can run large, asynchronous evaluation jobs at 50 % lower cost.

Use batches when you need to run large jobs offline — e.g. nightly evals, A/B tests, or bulk embeddings.

## Create Batch Job

Upload your .jsonl file first — see [Files API](https://docs.portkey.ai/docs/aigw/integrations/llms/openai/files.md) for more details and then use the following code to create a batch job.

```bash curl theme={"system"}
curl --location 'https://aigw.portkey.ai/v1/batches' \
--header 'Authorization: Bearer <portkey_api_key>' \
--header 'x-portkey-provider: @provider' \
--header 'Content-Type: application/json' \
--data '{
    "input_file_id": "<file_id>",
    "endpoint": "/v1/chat/completions",
    "completion_window": "24h",
    "metadata": {}
}'
```

```javascript OpenAI NodeJS theme={"system"}
import OpenAI from 'openai'; // We're using the v4 SDK

const openai = new OpenAI({
  apiKey: "PORTKEY_API_KEY",  // defaults to process.env["PORTKEY_API_KEY"]
  baseURL: "https://aigw.portkey.ai/v1",
  defaultHeaders: {
      "x-portkey-provider": "openai",
  }
});

const startBatch = async () => {
  const startBatchResponse = await openai.batches.create({
    input_file_id: "file_id", // file id of the input file
    endpoint: "/v1/chat/completions",
    completion_window: "24h",
    metadata: {} // metadata for the batch
  });

  console.log(startBatchResponse);
}

await startBatch();
```

```python OpenAI Python theme={"system"}
from openai import OpenAI

openai = OpenAI(
    api_key="PORTKEY_API_KEY",
    base_url="https://aigw.portkey.ai/v1",
    default_headers={"x-portkey-provider": "openai"}
)

start_batch_response = openai.batches.create(
  input_file_id="file_id", # file id of the input file
  endpoint="/v1/chat/completions",
  completion_window="24h",
  metadata={} # metadata for the batch
)

print(start_batch_response)
```

## List Batch Jobs

```bash curl theme={"system"}
curl --location 'https://aigw.portkey.ai/v1/batches' \
--header 'Authorization: Bearer <portkey_api_key>' \
--header 'x-portkey-provider: @provider'
```

```javascript OpenAI NodeJS theme={"system"}
import OpenAI from 'openai'; // We're using the v4 SDK

const openai = new OpenAI({
  apiKey: "PORTKEY_API_KEY",  // defaults to process.env["PORTKEY_API_KEY"]
  baseURL: "https://aigw.portkey.ai/v1",
  defaultHeaders: {
      "x-portkey-provider": "openai",
  }
});

const listBatches = async () => {
  const batches = await openai.batches.list();

  console.log(batches);
}

await listBatches();
```

```python OpenAI Python theme={"system"}
from openai import OpenAI

openai = OpenAI(
    api_key="PORTKEY_API_KEY",
    base_url="https://aigw.portkey.ai/v1",
    default_headers={"x-portkey-provider": "openai"}
)

batches = openai.batches.list()

print(batches)
```

## Get Batch Job Details

```bash curl theme={"system"}
curl --location 'https://aigw.portkey.ai/v1/batches/<batch_id>' \
--header 'Authorization: Bearer <portkey_api_key>' \
--header 'x-portkey-provider: @provider'
```

```javascript OpenAI NodeJS theme={"system"}
import OpenAI from 'openai'; // We're using the v4 SDK

const openai = new OpenAI({
  apiKey: "PORTKEY_API_KEY",  // defaults to process.env["PORTKEY_API_KEY"]
  baseURL: "https://aigw.portkey.ai/v1",
  defaultHeaders: {
      "x-portkey-provider": "openai",
  }
});

const getBatch = async () => {
  const batch = await openai.batches.retrieve(batch_id="batch_id");

  console.log(batch);
}

await getBatch();
```

```python OpenAI Python theme={"system"}
from openai import OpenAI

openai = OpenAI(
    api_key="PORTKEY_API_KEY",
    base_url="https://aigw.portkey.ai/v1",
    default_headers={"x-portkey-provider": "openai"}
)

batch = openai.batches.retrieve(batch_id="batch_id")

print(batch)
```

The status of a given Batch object can be any of the following:

| Status | Description |
| - | - |
| validating | the input file is being validated before the batch can begin |
| failed | the input file has failed the validation process |
| file\_upload\_failed | the batch job failed to upload the file to the provider after validation |
| batch\_start\_failed | the file upload succeeded but the batch job submission to the provider failed |
| in\_progress | the input file was successfully validated and the batch is currently being run |
| finalizing | the batch has completed and the results are being prepared |
| completed | the batch has been completed and the results are ready |
| expired | the batch was not able to be completed within the 24-hour time window |
| cancelling | the batch is being cancelled (may take up to 10 minutes) |
| cancelled | the batch was cancelled |

## Get Batch Output

```bash curl theme={"system"}
curl --location 'https://aigw.portkey.ai/v1/batches/<batch_id>/output' \
--header 'Authorization: Bearer <portkey_api_key>' \
--header 'x-portkey-provider: @provider'
```

## Cancel Batch Job

```bash curl theme={"system"}
curl --request POST --location 'https://aigw.portkey.ai/v1/batches/<batch_id>/cancel' \
--header 'Authorization: Bearer <portkey_api_key>' \
--header 'x-portkey-provider: @provider'
```

```javascript OpenAI NodeJS theme={"system"}
import OpenAI from 'openai'; // We're using the v4 SDK

const openai = new OpenAI({
  apiKey: "PORTKEY_API_KEY",  // defaults to process.env["PORTKEY_API_KEY"]
  baseURL: "https://aigw.portkey.ai/v1",
  defaultHeaders: {
      "x-portkey-provider": "openai",
  }
});

const cancelBatch = async () => {
  const cancel_batch_response = await openai.batches.cancel(batch_id="batch_id");

  console.log(cancel_batch_response);
}

await cancelBatch();
```

```python OpenAI Python theme={"system"}
from openai import OpenAI

openai = OpenAI(
    api_key="PORTKEY_API_KEY",
    base_url="https://aigw.portkey.ai/v1",
    default_headers={"x-portkey-provider": "openai"}
)

cancel_batch_response = openai.batches.cancel(batch_id="batch_id")

print(cancel_batch_response)
```


## Related topics

- [Batches](https://docs.portkey.ai/docs/aigw/integrations/llms/claude-platform-aws/batches.md)


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.