Skip to content

Add grammar reference with railroad syntax diagrams#372

Open
Sleepful wants to merge 3 commits intomainfrom
grammar-reference
Open

Add grammar reference with railroad syntax diagrams#372
Sleepful wants to merge 3 commits intomainfrom
grammar-reference

Conversation

@Sleepful
Copy link

@Sleepful Sleepful commented Mar 4, 2026

This PR goes in hand with powersync-ja/powersync-service#535

Summary

  • Add railroad diagram documentation for Sync Rules (15 productions) and Sync Streams (26 productions)
    • Mintlify does not support inline SVGs (strips <style> and <text> elements), so SVGs are static files referenced via <img> tags
  • Each grammar gets a single flat page with all productions, operator tables, and lexical rules
  • Each production section includes Used by (parent terms) and References (child terms) cross-links
  • Lexical rules section includes a token summary table and per-rule descriptions
  • Adds "Grammar Reference" navigation group in docs.json

Structure

  • sync/grammar/sync-rules/index.mdx — flat page with all 15 Sync Rules productions + lexical rules
  • sync/grammar/sync-rules/*.svg — 15 co-located SVG diagrams
  • sync/grammar/sync-streams/index.mdx — flat page with all 26 Sync Streams productions + lexical rules
  • sync/grammar/sync-streams/*.svg — 26 co-located SVG diagrams

Generated from

These pages are generated by packages/sync-rules/scripts/generate-grammar-docs.ts in the powersync-service repo (branch grammar-svgs):

pnpm exec tsx scripts/generate-grammar-docs.ts --outdir /path/to/powersync-docs/sync/grammar --base-url /sync/grammar

Sleepful added 3 commits March 4, 2026 22:52
Generated split-mode MDX pages with co-located SVG syntax diagrams
for Sync Rules (15 productions) and Sync Streams (26 productions).
Includes index pages, lexical rules pages, and docs.json nav entries.
…ammar reference

Replace 85 per-production MDX/SVG files with 2 flat index.mdx pages
(one per grammar) plus co-located SVGs. Each production section now
includes Used by / References cross-links. Simplify docs.json nav.
@Sleepful Sleepful force-pushed the grammar-reference branch from c95d411 to beaabb4 Compare March 5, 2026 04:52
@rkistner
Copy link
Contributor

rkistner commented Mar 5, 2026

I'm happy with the general approach. @benitav What do you think about this? Any tweaks you'd like? Keep in mind it's auto-generated, but we have a lot of control in how it's generated.

@benitav
Copy link
Collaborator

benitav commented Mar 5, 2026

Not sure if this level of feedback is useful (mostly polish in the page, not the generated content) but as someone who isn’t familiar with railroad diagrams and can’t easily parse them, but got a high-level understanding now.

  1. I’d organize the pages a bit differently, to hide the Sync Rules grammar under the Sync Rules section, and perhaps group this page with the Supported SQL page. Related, both pages are technically “Supported SQL”, so making the distinction a bit clearer - I don’t know enough about the distinction yet though, except one is more lower-level diagrams.
  2. What the “Compiler” prefix (e.g. CompilerStreamQuery) means isn’t intuitive, since I think “compiler” is some kind of internal reference? Why not just StreamQuery?
  3. An example in SQL paired with each diagram would make this easier to grok for a user like me. I think for CompilerStreamQuery it would be e.g. SELECT projects.* FROM projects JOIN memberships ON memberships.project_id = projects.id WHERE memberships.user_id = auth.user_id() (and/or an example with the comma-based join syntax if I understand it correctly)
  4. This long list of “expressions” (not sure that’s the correct term) seems overwhelming to navigate and the titles aren't very clear. Maybe some kind of table at the top with the most common expressions and their corresponding “use-case” in words. E.g. for a user who wants to understand whether/how subqueries in where clauses are possible, link to the relevant diagram(s). The “On this page” list on the right helps if you know what you’re looking for, but I don’t have a good sense if readers would be able to parse those names.
  5. A question: how would this be maintained? manually run some kind of script and paste results in here like we do for the generated Error Codes page? OK I see the script in the PR description now.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants