> ## Documentation Index
> Fetch the complete documentation index at: https://docs.portkey.ai/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Guardrails for Batches

> Secure batch inference with Prisma AIRS AI Gateway Guardrails

> [!NOTE]
>
> Gateway >= `2.9.0`
> Data Service >= `1.8.0`

Imagine usecases where you want to redact PII, remove some requests and redact responses when making a bulk batch inference call to providers like OpenAI, Azure OpenAI, Bedrock, Vertex AI, etc.

The AI Gateway supports running [guardrails](https://docs.portkey.ai/docs/aigw/product/guardrails.md) on [provider batch requests](https://docs.portkey.ai/docs/aigw/api-reference/batch/create-batch.md). You pass guardrails through a [config](https://docs.portkey.ai/docs/aigw/product/ai-gateway/configs.md),
and the AI Gateway applies them asynchronously — pre-processing inputs before sending to the provider, and post-processing outputs after the provider completes.

```mermaid theme={"system"}
flowchart LR
    A(["📄 Upload JSONL\n(input file)"])
    B(["⚙️ Create Config\nwith guardrails"])
    C(["🚀 Create Batch\n(file ID + config)"])

    subgraph portkey["AI Gateway"]
        direction TB
        D["🛡️ Input Guardrails\n(filter / transform requests)"]
        E[("pk_file_* v2\n(processed input)")]
        D --> E
    end

    subgraph provider["Upstream Provider"]
        F["🤖 LLM Inference\n(batch processing)"]
    end

    subgraph postprocess["AI Gateway"]
        direction TB
        G["🛡️ Output Guardrails\n(filter / transform responses)"]
        H[("pk_file_* final\n(output file)")]
        G --> H
    end

    I(["⬇️ Download\noutput file"])

    A --> C
    B --> C
    C --> D
    E --> F
    F --> G
    H --> I

    style portkey fill:#f0f4ff,stroke:#346df1,color:#1a1a1a
    style provider fill:#f5f5f5,stroke:#888,color:#1a1a1a
    style postprocess fill:#f0f4ff,stroke:#346df1,color:#1a1a1a
```

## How It Works

**1. Upload a batch input file**

Upload a JSONL file in OpenAI chat completions format to the AI Gateway.

```sh theme={"system"}
curl --location 'https://aigw.portkey.ai/v1/files' \
  --header 'Authorization: Bearer {{PORTKEY_API_KEY}}' \
  --header 'x-portkey-provider: portkey' \
  --form 'file=@"/path/to/batch_requests.jsonl"' \
  --form 'purpose="batch"'
```

Each line in the file should be a valid chat completions request. [Download an example JSONL file](https://github.com/Portkey-AI/docs-core/blob/main/files/chat_completions_101_requests.jsonl).

The response returns an AI Gateway file ID (`pk_file_*`):

```json theme={"system"}
{
  "object": "file",
  "id": "pk_file_1db86eb1-22b5-4440-8810-e45ddd1e5aca",
  "purpose": "batch",
  "filename": "d2d44065-0f52-4780-a216-eb0122a25d78/pk_file_1db86eb1-22b5-4440-8810-e45ddd1e5aca.jsonl",
  "bytes": 26204,
  "created_at": 1777975975402,
  "status": "processed",
  "status_details": null
}
```

**2. Create a config with guardrails**

Guardrails are the **only** supported way to pass guardrail checks to a provider batch request. Define them in a [AI Gateway config](https://docs.portkey.ai/docs/aigw/product/ai-gateway/configs.md) [here](https://stratacloudmanager.paloaltonetworks.com/):

```json theme={"system"}
{
  "retry": {
    "attempts": 3
  },
  "input_guardrails": ["pg-sync-g-0663db"],
  "output_guardrails": ["pg-mutato-9760bc"]
}
```

Save this config in Strata Cloud Manager and note the config ID (e.g., `pc-sync-g-1009a3`).

**3. Create a provider batch**

Reference your uploaded file ID and config when creating the batch:

```sh theme={"system"}
curl --location 'https://aigw.portkey.ai/v1/batches' \
  --header 'Authorization: Bearer {{PORTKEY_API_KEY}}' \
  --header 'x-portkey-provider: portkey' \
  --header 'Content-Type: application/json' \
  --data '{
      "input_file_id": "pk_file_1db86eb1-22b5-4440-8810-e45ddd1e5aca",
      "endpoint": "/v1/chat/completions",
      "completion_window": "24h",
      "model": "gpt-3.5-turbo",
      "portkey_options": {
          "x-portkey-config": "pc-sync-g-1009a3",
          "x-portkey-virtual-key": "openai-virtual-key"
      }
  }'
```

The response contains an AI Gateway batch ID (`pk_batch_*`) and the initial guardrail results in `hook_results`:

```json theme={"system"}
{
  "id": "pk_batch_cd7f0e97-37b6-48cb-abd7-1bb699300585",
  "object": "batch",
  "status": "validating",
  "input_file_id": "pk_file_1db86eb1-22b5-4440-8810-e45ddd1e5aca",
  "hook_results": {
    "before_request_hooks": [],
    "after_request_hooks": [
      {
        "verdict": true,
        "id": "portkey-dataservice",
        "type": "guardrail",
        "deny": false
      }
    ]
  }
}
```

**4. Poll for batch status**

Poll the batch to track progress. Once input guardrails are applied, the `input_file_id` is replaced with a new processed file. Use `include_provider_details=true` to see full upstream details:

```sh theme={"system"}
curl --location 'https://aigw.portkey.ai/v1/batches/pk_batch_cd7f0e97-37b6-48cb-abd7-1bb699300585?include_provider_details=true' \
  --header 'Authorization: Bearer {{PORTKEY_API_KEY}}'
```

If your config includes `output_guardrails`, the batch enters an intermediate `processing_post_request_guardrails` state after the upstream provider finishes. Once complete, `output_file_id` reflects the final guardrail-processed output file.

**5. Download the output file**

Once the batch status is `completed`, retrieve the output using the gateway-exclusive file content endpoint:

```sh theme={"system"}
curl --location 'https://aigw.portkey.ai/v1/files/{{OUTPUT_FILE_ID}}/content' \
  --header 'Authorization: Bearer {{PORTKEY_API_KEY}}' \
  --header 'x-portkey-provider: portkey'
```

## Batch Lifecycle with Guardrails

| State | Description |
| :- | :- |
| `validating` | Batch created; input guardrails being evaluated |
| `in_progress` | Input guardrails passed; provider is processing requests |
| `processing_post_request_guardrails` | Provider finished; output guardrails being evaluated |
| `completed` | All guardrails applied; output file ready |

> [!NOTE]
>
> The `input_file_id` in the batch response is updated after input guardrails
> process the file — the new ID points to the transformed input sent to the
> upstream provider.


## Related topics

- [Unified Batch Inference](https://docs.portkey.ai/docs/aigw/product/ai-gateway/batches.md)
- [Batches](https://docs.portkey.ai/docs/aigw/integrations/llms/claude-platform-aws/batches.md)


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.