Documentation

OpenAI

Trace OpenAI requests and responses.

Overview

The OpenAI integration automatically traces every chat completion request made through an OpenAI client. It captures prompts, responses, model names, latency, token usage, streaming chunks, and retry attempts — all as structured steps within a TraceLLM trace.

Two API surfaces are available: wrap_openai() monkey-patches an existing OpenAI instance, and TraceOpenAI is a drop-in replacement class that self-wraps on initialization.

Installation

terminalCopy

bash

pip install "tracellm[openai]"

This installs openai>=1.6.0 alongside TraceLLM. Alternatively, install TraceLLM and OpenAI separately:

terminalCopy

bash

pip install tracellm openai

Setup

Set your OpenAI API key and wrap your client:

setup.pyCopy

python

from openai import OpenAI
from tracellm import trace
from tracellm.integrations.openai import wrap_openai

client = OpenAI()
client = wrap_openai(client)

Or use the TraceOpenAI class directly:

TraceOpenAI.pyCopy

python

from tracellm.integrations.openai import TraceOpenAI

client = TraceOpenAI()  # auto-wrapped

Example

The standard pattern pairs @trace with a wrapped client. The decorator provides the outer trace container (project, environment, metadata), and every chat completion call inside it is captured as a step:

openai_example.pyCopy

python

from openai import OpenAI
from tracellm import trace
from tracellm.integrations.openai import wrap_openai

client = OpenAI()
client = wrap_openai(client)

@trace(
    prompt="ask_llm",
    model_name="gpt-4o",
    project="customer-support",
    environment="production",
)
def ask_llm(prompt: str) -> str:
    response = client.chat.completions.create(
        model="gpt-4o",
        messages=[
            {"role": "system", "content": "You are a helpful assistant."},
            {"role": "user", "content": prompt},
        ],
        max_tokens=500,
    )
    return response.choices[0].message.content

result = ask_llm("What is the capital of France?")
print(result)

Streaming

Streaming is supported transparently. The integration collects chunks as they arrive, reconstructs the full response, and records the stream as a single openai_chat_stream step with chunk count metadata:

openai_stream.pyCopy

python

@trace(prompt="ask_stream", project="streaming-demo")
def ask_stream(prompt: str) -> str:
    stream = client.chat.completions.create(
        model="gpt-4o-mini",
        messages=[{"role": "user", "content": prompt}],
        stream=True,
    )
    full = ""
    for chunk in stream:
        if chunk.choices and chunk.choices[0].delta.content:
            content = chunk.choices[0].delta.content
            print(content, end="", flush=True)
            full += content
    print()
    return full

Info

During streaming, the trace is finalized after all chunks are consumed. The trace payload includes the complete response, total latency, and estimated token count (one token per content-bearing chunk).

Retries and Error Handling

The integration respects the max_retries parameter on chat completion calls. Each retry attempt is recorded as a separate step with exponential backoff (0.5 * 2^attempt, capped at 5s). Failed attempts are marked with success=False, and the final exception is re-raised if all retries are exhausted:

openai_retry.pyCopy

python

@trace(prompt="unreliable_api", project="retry-demo")
def call_with_retries(prompt: str) -> str:
    # max_retries is passed directly to the OpenAI SDK
    response = client.chat.completions.create(
        model="gpt-4o-mini",
        messages=[{"role": "user", "content": prompt}],
        max_retries=3,          # <-- TraceLLM captures each retry as a step
    )
    return response.choices[0].message.content

What the Trace Captures

Each OpenAI call produces a trace containing:

Data	Non-Streaming	Streaming
Prompt	Full messages array	Full messages array
Response	Full message content	Full reconstructed content
Model	From response metadata	From request kwargs
Latency	perf_counter delta (ms)	perf_counter delta (ms)
Token count	OpenAI usage.total_tokens	Estimated from chunks
Finish reason	From choices[0].finish_reason	N/A
Steps	openai_chat step	openai_chat_stream step
Retries	Separate step per attempt	Separate step per attempt

Expected Output

After running a traced OpenAI call, the console shows a trace summary panel and the dashboard displays the trace with its captured step:

Console outputCopy

text

  ╭── TraceLLM Trace ───────────────────────────── SUCCESS ──╮
  │                                                              │
  │  Trace ID     tr_a1b2c3d4                                    │
  │  Prompt       What is the capital of France?                 │
  │  Model        gpt-4o                                         │
  │  Project      customer-support                               │
  │  Environment  production                                     │
  │  Latency      1,240.00 ms                                    │
  │  Token Count  87                                             │
  │  Retries      0                                              │
  │  Steps        1                                              │
  │  Status        SUCCESS                                       │
  │                                                              │
  ╰──────────────────────────────────────────────────────────────╯

  #  Tool              Duration  Status  Detail
  1  openai_chat         1240ms     OK

Verification

Check the console for the trace summary panel after each call
Open the dashboard at http://localhost:3000 and find your trace
Inspect the trace details — the step should show model, messages, and finish reason
Verify token counts match the OpenAI response metadata

Troubleshooting

Issue	Cause	Fix
No traces appear	wrap_openai() not called or client created after wrapping	Ensure wrap_openai(client) is called and the wrapped client is used
Streaming traces have 0 tokens	Stream ended before chunks were collected	Check that the stream is fully consumed before the @trace function returns
OpenAI import error	openai package not installed	Run pip install 'tracellm[openai]' or pip install openai>=1.6.0
API key not set	OPENAI_API_KEY environment variable is missing	Set export OPENAI_API_KEY=sk-... or pass api_key= to OpenAI()