Skip to content

[FEATURE] Add model streaming event hooks to Python SDK #1819

@zachazhu

Description

@zachazhu

Problem Statement

The TypeScript Strands SDK exposes hooks into model streamed chunk events, but the Python SDK has no equivalent. There is currently no way to observe or act on individual chunks during model streaming, and no mechanism to cancel an in-flight model call mid-stream.

The Python SDK's hook system provides BeforeModelCallEvent and AfterModelCallEvent, but nothing fires between them while the stream is active. Once stream_messages() starts, the only way to stop it is to cancel the entire asyncio task, which is is the approach taken to resolve #81.

Proposed Solution

Add mid-stream chunk hooks to the Python SDK (matching TypeScript SDK parity), with the ability to signal cancellation from within the hook. For example:

@dataclass
class ModelStreamChunkHookEvent(HookEvent):
    chunk: StreamEvent
    cancel: bool = False  # Set to True to stop the stream

When cancel is set to True, the stream should:

  • Stop consuming further chunks
  • Clean up server-side resources (close the HTTP connection)
  • Fall through to AfterModelCallEvent where hooks can set retry = True

Use Case

We need to cancel a specific in-flight model call based on an external signal (e.g., a background validation check that determines the current call is invalid) and then retry via AfterModelCallEvent.retry = True. This
requires:

  1. A way to observe or intercept the stream mid-flight
  2. A way to signal cancellation that cleanly stops the stream
  3. Execution continuing to AfterModelCallEvent so hooks can decide whether to retry

This is distinct from agent-level cancellation (#81, #1772), which kills the entire agent invocation. We need per-model-call granularity where the event loop continues running after cancellation.

Alternatives Solutions

A prototype implementation using AbortSignal + CancellableStream was explored but mid-stream hooks are a more general solution that also enables other use cases (logging, metrics, content filtering)

Additional Context

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions