-
Notifications
You must be signed in to change notification settings - Fork 1
Description
Gap
The OpenAI Python SDK provides a client.chat.completions.stream() helper method that is distinct from client.chat.completions.create(stream=True). This helper is not instrumented — calling it on a wrapped client produces no Braintrust span.
CompletionsV1Wrapper (line 776 of oai.py) only defines create(). When .stream() is called, it falls through via NamedWrapper.__getattr__ (line 24) to the underlying unwrapped Completions resource. The request succeeds but is invisible to tracing.
How .stream() differs from .create(stream=True)
The .stream() helper (documented in the openai-python helpers.md):
- Returns a context manager (
ChatCompletionStream) - Provides an event-based API with structured event objects instead of raw chunks
- Automatically accumulates deltas across streaming chunks
- Offers convenience methods like
.get_final_completion()and.until_done() - Is the SDK-recommended pattern for streaming in many OpenAI guides
# Pattern 1 — instrumented today
stream = client.chat.completions.create(stream=True, ...)
for chunk in stream:
...
# Pattern 2 — NOT instrumented
with client.chat.completions.stream(...) as stream:
for event in stream:
...
final = stream.get_final_completion()The same gap likely applies to AsyncCompletionsV1Wrapper (line 803) for the async variant.
What is missing
client.chat.completions.stream()(sync) — needs astream()method onCompletionsV1Wrapperclient.chat.completions.stream()(async) — needs astream()method onAsyncCompletionsV1Wrapper
Braintrust docs status
unclear — the OpenAI integration page mentions streaming support but does not distinguish between .create(stream=True) and .stream().
Upstream sources
- OpenAI Python SDK streaming helpers: https://github.com/openai/openai-python/blob/main/helpers.md
openai.resources.chat.completions.Completions.streammethod (available since openai-python ~v1.26)
Local files inspected
py/src/braintrust/oai.py:CompletionsV1Wrapper(line 776) — only definescreate()AsyncCompletionsV1Wrapper(line 803) — only defines asynccreate()NamedWrapper.__getattr__(line 24) — delegates unknown attrs to the unwrapped object
py/src/braintrust/wrappers/test_openai.py— no tests forchat.completions.stream(), all streaming tests usecreate(stream=True)py/noxfile.py— tests openai versions latest, 1.77.0, 1.71, 1.91, 1.92;.stream()helper exists in all of these