perf: cut per-execution Deno overhead (CPL-264)#304
Open
Conversation
Five complementary changes shrink the per-request hot path inside
lit-actions-server so cached actions reuse more work across runs:
- Snapshot-bake `delete Deno.{build,permissions,version}` and
`delete globalThis.Worker` into LitNamespace.js so each request
runs one `execute_script` for namespace setup + bootstrap cleanup
instead of two.
- Cache the `RuntimePermissionDescriptorParser` and a base
`Permissions` instance in `LazyLock`s so build_main_worker_and_inject_sdk
clones the struct instead of re-running `Permissions::from_options`
per call.
- Add `SharedV8CodeCache` (new `v8_code_cache.rs`) and wire it into
`WorkerServiceOptions.v8_code_cache` so V8 reuses compiled bytecode
for cached actions across requests.
- Route user code through `op_eval_context` via the `__litEvalCached`
helper baked into 99_patches.js so Deno's `eval_context_code_cache_cbs`
(the only consumer of the V8 code cache) actually sees the source.
The helper deletes itself from globalThis on first call so user code
never sees a string-eval primitive (preserves the
`--disallow-code-generation-from-strings` posture).
- Inline literal `import("X")` calls at bundle time
(`bundler::rewrite_literal_dynamic_imports`) so dynamic imports flow
through the same SWC pipeline as static ones; non-literal dynamic
imports surface a clear bundler error.
Plus a small cleanup: gate the dead `RewriteResult.code` rebuild
behind `#[cfg(test)]` so the production scanner skips the per-miss
string allocation.
Tests: 74 server unit + 23 integration pass (including 5 new bundler
tests for `DynamicImportRewriter`, 3 for `v8_code_cache`, and a new
`lit_eval_cached_is_hidden_from_user_code` integration test that
verifies user code cannot reach the eval helper).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Contributor
There was a problem hiding this comment.
Pull request overview
This PR reduces per-execution overhead in lit-actions-server by moving more work into shared/process-wide caches and by routing user code through Deno’s eval-context code-cache pathway so cached actions can reuse compiled V8 bytecode across requests.
Changes:
- Introduces a process-wide
SharedV8CodeCacheand plumbs it throughServer → execute_js → WorkerServiceOptions. - Routes user code execution through a snapshot-baked
__litEvalCachedhelper that callsop_eval_context, enabling Deno/V8 eval-context code caching. - Extends the SWC bundler to rewrite literal
import("...")into static imports for cache-time bundling, and optimizes import rewriting to avoid dead allocations on the hot path.
Reviewed changes
Copilot reviewed 8 out of 8 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
| lit-actions/tests/it.rs | Adds an integration test ensuring __litEvalCached is not observable from user code; updates stack trace expectations. |
| lit-actions/server/v8_code_cache.rs | Adds a new in-memory V8 bytecode cache implementing deno_runtime::code_cache::CodeCache. |
| lit-actions/server/server.rs | Stores and passes a shared V8 code cache into each execution. |
| lit-actions/server/runtime.rs | Caches permissions globally; removes PatchDeno.js round-trip; routes user code through __litEvalCached(op_eval_context); wires the V8 code cache into WorkerServiceOptions. |
| lit-actions/server/lib.rs | Registers the new v8_code_cache module. |
| lit-actions/server/import_rewriter.rs | Gates RewriteResult.code behind #[cfg(test)] to avoid rebuild/allocation on production cache misses. |
| lit-actions/server/bundler.rs | Adds a pre-bundle rewrite for literal dynamic imports and new unit tests. |
| lit-actions/ext/js/99_patches.js | Adds the __litEvalCached helper that self-deletes and evaluates code via op_eval_context. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Local rustfmt 1.88.0 missed five formatting drifts that CI's 1.91.1 flagged. No functional changes. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
- v8_code_cache: probe-then-decide in set_sync so a too-large replacement no longer drops the existing cached bytecode (would have turned a cache hit into a permanent miss). Recover by clearing the cache on fresh insert past the 100MB ceiling instead of becoming permanently full. Add 2 unit tests for the edge cases. - v8_code_cache: borrowed-key (V8CodeCacheKeyRef) lookup in get_sync via a trait-object Borrow trick, so V8's per-module-load cache probe no longer allocates a String for the specifier on the hot path. - bundler: capture spans of non-literal import(...) offenders and surface the first one's line/col + snippet in the error so users see exactly where to refactor, instead of just a count. - bundler: explicit doc note on the eager-evaluation semantic change of the literal import() rewrite. - Drive-by: switch extend(drain) to append() to satisfy clippy 1.91. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Resolved bundler.rs by keeping both branches' new test additions: - This branch: rewrite_dynamic_imports_* + run_swc_bundler_inlines_literal_dynamic_import (CPL-264, F3 literal dynamic-import bundling) - next: cached_loader helper + walk_deps_collects_transitive_graph + walk_deps_enforces_max_module_count_on_wide_layer (CPL-263 parallel dep-graph walk) Drive-by: rewrote slice literal at walk_deps_collects_transitive_graph to satisfy clippy::cloned_ref_to_slice_refs (1.91). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Integrate CPL-265 pre-warmed worker pool with CPL-264 V8 code cache: - Plumb SharedV8CodeCache through PoolSharedState so pool workers benefit from the same code-cache hits as the legacy cold path. - Wire v8_code_cache from DispatchState into spawn_legacy. - Remove now-dead PRIVACY_MODE_HEADER_KEY static (next's split helpers call HEADER_KEY_X_PRIVACY_MODE.to_ascii_lowercase() inline). - Take next's build_worker_base / inject_lit_namespace / execute_patch_deno / inject_params_globals split.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Five complementary changes to
lit-actions-servershrink the per-request hot path so cached actions reuse more work across runs:Snapshot-bake bootstrap deletions (Option A)
delete Deno.{build,permissions,version}anddelete globalThis.Workernow live inLitNamespace.js, so each request runs oneexecute_scriptfor namespace setup + bootstrap cleanup instead of two. Removes the dedicatedPatchDeno.jsround-trip.Cache Permissions globally (Option B)
The
RuntimePermissionDescriptorParserand the basePermissionsinstance are identical for every execution (deny everything except outboundfetch). Both now live inLazyLocks and the per-callbuild_main_worker_and_inject_sdkclones the struct instead of rebuilding viaPermissions::from_optionsevery time.Drop dead
RewriteResult.coderebuild (Option C)Production callers never read the stripped form (the bundler consumes the original code with imports intact). Gated behind
#[cfg(test)]so the per-miss string allocation stays out of the hot path. Tests still pin scanner behavior.V8 code cache (Option H)
New
v8_code_cache.rsprovides aSharedV8CodeCacheplumbed throughServer→execute_js→WorkerServiceOptions.v8_code_cache. Wired so V8 reuses compiled bytecode for cached actions across requests.The cache is only consumed by Deno's
eval_context_code_cache_cbs—execute_scriptbypasses it entirely. So a new__litEvalCachedhelper (baked into99_patches.js) routes user code throughop_eval_context. The outer stub still parses per execution but its body is one string literal, so V8 only pays for source-string scanning rather than compiling the bundled action body.The helper deletes itself from globalThis on its first call so user code (which runs inside
op_eval_contextbelow) cannot reach it — preserves the--disallow-code-generation-from-stringsposture by not handing actions a string-eval primitive. Each request runs in a fresh worker so the delete has no cross-request effect.Inline literal dynamic imports (CPL-264 / F3 fix)
bundler::rewrite_literal_dynamic_importswalks the SWC AST, replaces literalimport("X")withPromise.resolve(__litDyn_<i>), and prependsimport * as __litDyn_<i> from "X";declarations. The static-import bundler then inlines them like any other dependency. Non-literalimport(expr)calls surface a clear bundler error pointing at where to refactor.prepare_action_codenow also routes through the bundler when the source containsimport(so dynamic imports get picked up. The substring check is intentionally permissive — false positives (the literal text in a string or comment) just route through the bundler harmlessly.Test Coverage
DynamicImportRewritertests, 3 newv8_code_cachetests)lit_eval_cached_is_hidden_from_user_codeverifies user code cannot reach the eval helper)Pre-Landing Review
Codex adversarial review run pre-commit. Three findings addressed:
__litEvalCachedexposure): helper now self-deletes from globalThis as its first action; integration test verifies invisibility from user code.DynamicImportRewriterso literalimport("X")continues to work after the cache-time bundling switch in CPL-262.99_patches.js:34-46to describe the mechanism (eval-context cache wiring) rather than asserting a specific speedup.Two findings flagged by Codex (
F1jsDelivr..traversal at full-URL path,F4hand scanner gate) confirmed pre-existing onnextand out of scope for this branch.TODOS
No
TODOS.mditems relevant to this branch.Notes
VERSION/CHANGELOG.mdin this repo; lit-actions workspace stays at0.1.0.--no-gpg-signdue to a missingpinentry-macin the local environment. Repo doesn't enforce signed commits (most recent commits onnextare also unsigned).Test plan
cargo test -p lit-actions-server(74/74)cargo test -p lit-actions-tests(23 pass, 2 ignored)cargo build -p lit-actions-server --releaseclean🤖 Generated with Claude Code