Skip to content

perf: cut per-execution Deno overhead (CPL-264)#304

Open
GTC6244 wants to merge 5 commits intonextfrom
feature/cpl-264-deno-loading-v2
Open

perf: cut per-execution Deno overhead (CPL-264)#304
GTC6244 wants to merge 5 commits intonextfrom
feature/cpl-264-deno-loading-v2

Conversation

@GTC6244
Copy link
Copy Markdown
Contributor

@GTC6244 GTC6244 commented Apr 20, 2026

Summary

Five complementary changes to lit-actions-server shrink the per-request hot path so cached actions reuse more work across runs:

Snapshot-bake bootstrap deletions (Option A)
delete Deno.{build,permissions,version} and delete globalThis.Worker now live in LitNamespace.js, so each request runs one execute_script for namespace setup + bootstrap cleanup instead of two. Removes the dedicated PatchDeno.js round-trip.

Cache Permissions globally (Option B)
The RuntimePermissionDescriptorParser and the base Permissions instance are identical for every execution (deny everything except outbound fetch). Both now live in LazyLocks and the per-call build_main_worker_and_inject_sdk clones the struct instead of rebuilding via Permissions::from_options every time.

Drop dead RewriteResult.code rebuild (Option C)
Production callers never read the stripped form (the bundler consumes the original code with imports intact). Gated behind #[cfg(test)] so the per-miss string allocation stays out of the hot path. Tests still pin scanner behavior.

V8 code cache (Option H)
New v8_code_cache.rs provides a SharedV8CodeCache plumbed through Serverexecute_jsWorkerServiceOptions.v8_code_cache. Wired so V8 reuses compiled bytecode for cached actions across requests.

The cache is only consumed by Deno's eval_context_code_cache_cbsexecute_script bypasses it entirely. So a new __litEvalCached helper (baked into 99_patches.js) routes user code through op_eval_context. The outer stub still parses per execution but its body is one string literal, so V8 only pays for source-string scanning rather than compiling the bundled action body.

The helper deletes itself from globalThis on its first call so user code (which runs inside op_eval_context below) cannot reach it — preserves the --disallow-code-generation-from-strings posture by not handing actions a string-eval primitive. Each request runs in a fresh worker so the delete has no cross-request effect.

Inline literal dynamic imports (CPL-264 / F3 fix)
bundler::rewrite_literal_dynamic_imports walks the SWC AST, replaces literal import("X") with Promise.resolve(__litDyn_<i>), and prepends import * as __litDyn_<i> from "X"; declarations. The static-import bundler then inlines them like any other dependency. Non-literal import(expr) calls surface a clear bundler error pointing at where to refactor.

prepare_action_code now also routes through the bundler when the source contains import( so dynamic imports get picked up. The substring check is intentionally permissive — false positives (the literal text in a string or comment) just route through the bundler harmlessly.

Test Coverage

  • 74 server unit tests pass (5 new DynamicImportRewriter tests, 3 new v8_code_cache tests)
  • 23 integration tests pass + 2 ignored (1 new: lit_eval_cached_is_hidden_from_user_code verifies user code cannot reach the eval helper)
  • Release build clean
cargo test -p lit-actions-server   # 74/74
cargo test -p lit-actions-tests    # 23/23 + 2 ignored
cargo build -p lit-actions-server --release

Pre-Landing Review

Codex adversarial review run pre-commit. Three findings addressed:

  • F2 (__litEvalCached exposure): helper now self-deletes from globalThis as its first action; integration test verifies invisibility from user code.
  • F3 (dynamic-import behavior change): extended bundler with DynamicImportRewriter so literal import("X") continues to work after the cache-time bundling switch in CPL-262.
  • F6 (unsubstantiated perf claim): reworded comment in 99_patches.js:34-46 to describe the mechanism (eval-context cache wiring) rather than asserting a specific speedup.

Two findings flagged by Codex (F1 jsDelivr .. traversal at full-URL path, F4 hand scanner gate) confirmed pre-existing on next and out of scope for this branch.

TODOS

No TODOS.md items relevant to this branch.

Notes

  • No VERSION / CHANGELOG.md in this repo; lit-actions workspace stays at 0.1.0.
  • Commit was created with --no-gpg-sign due to a missing pinentry-mac in the local environment. Repo doesn't enforce signed commits (most recent commits on next are also unsigned).

Test plan

  • cargo test -p lit-actions-server (74/74)
  • cargo test -p lit-actions-tests (23 pass, 2 ignored)
  • cargo build -p lit-actions-server --release clean
  • Smoke-test in dev env (cached action runs faster than uncached on second invocation)

🤖 Generated with Claude Code

Five complementary changes shrink the per-request hot path inside
lit-actions-server so cached actions reuse more work across runs:

- Snapshot-bake `delete Deno.{build,permissions,version}` and
  `delete globalThis.Worker` into LitNamespace.js so each request
  runs one `execute_script` for namespace setup + bootstrap cleanup
  instead of two.
- Cache the `RuntimePermissionDescriptorParser` and a base
  `Permissions` instance in `LazyLock`s so build_main_worker_and_inject_sdk
  clones the struct instead of re-running `Permissions::from_options`
  per call.
- Add `SharedV8CodeCache` (new `v8_code_cache.rs`) and wire it into
  `WorkerServiceOptions.v8_code_cache` so V8 reuses compiled bytecode
  for cached actions across requests.
- Route user code through `op_eval_context` via the `__litEvalCached`
  helper baked into 99_patches.js so Deno's `eval_context_code_cache_cbs`
  (the only consumer of the V8 code cache) actually sees the source.
  The helper deletes itself from globalThis on first call so user code
  never sees a string-eval primitive (preserves the
  `--disallow-code-generation-from-strings` posture).
- Inline literal `import("X")` calls at bundle time
  (`bundler::rewrite_literal_dynamic_imports`) so dynamic imports flow
  through the same SWC pipeline as static ones; non-literal dynamic
  imports surface a clear bundler error.

Plus a small cleanup: gate the dead `RewriteResult.code` rebuild
behind `#[cfg(test)]` so the production scanner skips the per-miss
string allocation.

Tests: 74 server unit + 23 integration pass (including 5 new bundler
tests for `DynamicImportRewriter`, 3 for `v8_code_cache`, and a new
`lit_eval_cached_is_hidden_from_user_code` integration test that
verifies user code cannot reach the eval helper).

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR reduces per-execution overhead in lit-actions-server by moving more work into shared/process-wide caches and by routing user code through Deno’s eval-context code-cache pathway so cached actions can reuse compiled V8 bytecode across requests.

Changes:

  • Introduces a process-wide SharedV8CodeCache and plumbs it through Server → execute_js → WorkerServiceOptions.
  • Routes user code execution through a snapshot-baked __litEvalCached helper that calls op_eval_context, enabling Deno/V8 eval-context code caching.
  • Extends the SWC bundler to rewrite literal import("...") into static imports for cache-time bundling, and optimizes import rewriting to avoid dead allocations on the hot path.

Reviewed changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated 5 comments.

Show a summary per file
File Description
lit-actions/tests/it.rs Adds an integration test ensuring __litEvalCached is not observable from user code; updates stack trace expectations.
lit-actions/server/v8_code_cache.rs Adds a new in-memory V8 bytecode cache implementing deno_runtime::code_cache::CodeCache.
lit-actions/server/server.rs Stores and passes a shared V8 code cache into each execution.
lit-actions/server/runtime.rs Caches permissions globally; removes PatchDeno.js round-trip; routes user code through __litEvalCached(op_eval_context); wires the V8 code cache into WorkerServiceOptions.
lit-actions/server/lib.rs Registers the new v8_code_cache module.
lit-actions/server/import_rewriter.rs Gates RewriteResult.code behind #[cfg(test)] to avoid rebuild/allocation on production cache misses.
lit-actions/server/bundler.rs Adds a pre-bundle rewrite for literal dynamic imports and new unit tests.
lit-actions/ext/js/99_patches.js Adds the __litEvalCached helper that self-deletes and evaluates code via op_eval_context.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread lit-actions/server/bundler.rs
Comment thread lit-actions/server/bundler.rs Outdated
Comment thread lit-actions/server/v8_code_cache.rs Outdated
Comment thread lit-actions/server/v8_code_cache.rs Outdated
Comment thread lit-actions/server/v8_code_cache.rs Outdated
GTC6244 and others added 4 commits April 20, 2026 12:22
Local rustfmt 1.88.0 missed five formatting drifts that CI's 1.91.1
flagged. No functional changes.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
- v8_code_cache: probe-then-decide in set_sync so a too-large replacement
  no longer drops the existing cached bytecode (would have turned a cache
  hit into a permanent miss). Recover by clearing the cache on fresh insert
  past the 100MB ceiling instead of becoming permanently full. Add 2 unit
  tests for the edge cases.
- v8_code_cache: borrowed-key (V8CodeCacheKeyRef) lookup in get_sync via a
  trait-object Borrow trick, so V8's per-module-load cache probe no longer
  allocates a String for the specifier on the hot path.
- bundler: capture spans of non-literal import(...) offenders and surface
  the first one's line/col + snippet in the error so users see exactly
  where to refactor, instead of just a count.
- bundler: explicit doc note on the eager-evaluation semantic change of
  the literal import() rewrite.
- Drive-by: switch extend(drain) to append() to satisfy clippy 1.91.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Resolved bundler.rs by keeping both branches' new test additions:
- This branch: rewrite_dynamic_imports_* + run_swc_bundler_inlines_literal_dynamic_import
  (CPL-264, F3 literal dynamic-import bundling)
- next: cached_loader helper + walk_deps_collects_transitive_graph +
  walk_deps_enforces_max_module_count_on_wide_layer (CPL-263 parallel
  dep-graph walk)

Drive-by: rewrote slice literal at walk_deps_collects_transitive_graph
to satisfy clippy::cloned_ref_to_slice_refs (1.91).

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Integrate CPL-265 pre-warmed worker pool with CPL-264 V8 code cache:
- Plumb SharedV8CodeCache through PoolSharedState so pool workers
  benefit from the same code-cache hits as the legacy cold path.
- Wire v8_code_cache from DispatchState into spawn_legacy.
- Remove now-dead PRIVACY_MODE_HEADER_KEY static (next's split helpers
  call HEADER_KEY_X_PRIVACY_MODE.to_ascii_lowercase() inline).
- Take next's build_worker_base / inject_lit_namespace /
  execute_patch_deno / inject_params_globals split.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants