Skip to content

fix: make streaming parser more stable and add a fallback#5808

Open
OEvgeny wants to merge 10 commits intomainfrom
fix/stabilize-streaming-parser
Open

fix: make streaming parser more stable and add a fallback#5808
OEvgeny wants to merge 10 commits intomainfrom
fix/stabilize-streaming-parser

Conversation

@OEvgeny
Copy link
Copy Markdown
Collaborator

@OEvgeny OEvgeny commented Apr 15, 2026

Fixes #

Changelog Entry

  • 🧪 Added incremental streaming Markdown renderer for livestreaming, in PR #5799, by @OEvgeny
    • Fixed streaming Markdown renderer to preserve link reference definitions during incremental rendering and recover on error, in PR #5808, by @OEvgeny

Description

This change stabilizes createStreamingRenderer by improving incremental Markdown parsing and adding a fallback on error. It also preserves link reference definitions across streaming commits so links continue to resolve correctly as streamed content grows.

Design

The main idea is to fed known definitions back to the parser instead of trying to resume the parser state which is hidden in the closure.

Specific Changes

  • createStreamingRenderer.ts

    • Added STREAMING_ERROR symbol and wrapper error metadata.
    • Added extractDefinitions, step, commit, cleanup, and revert helper logic.
    • Improved incremental rendering path to preserve definitions and commit newly finished blocks.
  • createStreamingRenderer.spec.ts

    • Added coverage for:
      • definition extraction during full reparse,
      • active reference resolution while tail grows,
      • incremental commits with later definitions,
      • shared link definitions across committed and active blocks,
      • definition preservation when blocks become committed.

Checklist

  • I have added tests and executed them locally
  • I have updated CHANGELOG.md
  • I have updated documentation

Review Checklist

This section is for contributors to review your work.

  • Accessibility reviewed (tab order, content readability, alt text, color contrast)
  • Browser and platform compatibilities reviewed
  • CSS styles reviewed (minimal rules, no z-index)
  • Documents reviewed (docs, samples, live demo)
  • Internationalization reviewed (strings, unit formatting)
  • package.json and package-lock.json reviewed
  • Security reviewed (no data URIs, check for nonce leak)
  • Tests reviewed (coverage, legitimacy)

@OEvgeny OEvgeny marked this pull request as ready for review April 16, 2026 00:25
throw new Error('Failed to extract definition token offset');
}

lastStepDefinitionOffset = lastDefinitionTokenOffset;
Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No reliable way to flip it so definitions appear at the end allowing us to drop the offset.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant