Skip to content

Add LSP server for integration package validation#3405

Draft
ilyannn wants to merge 12 commits intomainfrom
worktree-typed-prancing-rainbow
Draft

Add LSP server for integration package validation#3405
ilyannn wants to merge 12 commits intomainfrom
worktree-typed-prancing-rainbow

Conversation

@ilyannn
Copy link
Copy Markdown

@ilyannn ilyannn commented Mar 24, 2026

Summary

  • Add elastic-package lsp command — a Language Server Protocol server (stdio) that wraps existing package-spec validation to surface lint errors inline in editors
  • Zed editor extension at zed-elastic-package launches the LSP
  • Field completions in ingest pipelines (from fields/*.yml), manifest key completions, and field type completions
  • Hover documentation for manifest keys (with nested path resolution), field types, units, and metric types
  • Validation errors mapped to exact YAML line positions

Test plan

  • go test ./internal/lsp/... — unit tests for error parsing, position lookup, URI conversion, document store, manifest schema
  • ./scripts/lsp-demo.sh ./elastic-package — end-to-end demo: valid package (no errors), broken package (errors routed to correct files), nested file triggers package-root discovery
  • Install Zed dev extension, open an integration package, verify diagnostics appear on save
  • Hover over manifest keys (including nested like streams.input) — verify docs shown
  • Type field: in a pipeline file — verify field name completions appear

🤖 Generated with Claude Code / Opus 4.6

ilyannn and others added 4 commits March 20, 2026 16:42
Wrap existing package-spec validation in an LSP server (stdio) so editors
can surface lint errors inline on open/save. Uses tliron/glsp for the
JSON-RPC transport. Includes debounced per-package validation, error-to-file
routing, and diagnostic clearing.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…guide

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
… LSP

YAML field path resolution maps validation errors to exact lines.
Document store tracks open file contents for hover/completions.
Notify function captured at initialize for safe async use in debouncer.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Completions: field names from fields/*.yml in pipelines, field types in
field definitions, manifest keys with context awareness.
Hover: manifest key docs (nested paths resolved via indentation), field
type/unit/metric_type docs, field reference info from field index.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Comment thread internal/lsp/completion.go
Comment thread internal/lsp/position.go Outdated
Comment thread internal/lsp/hover.go
Comment thread internal/lsp/workspace.go
Comment thread scripts/lsp-demo.sh Outdated
Comment thread scripts/lsp-demo.sh Outdated
ilyannn and others added 2 commits March 24, 2026 13:37
Better error attribution for dashboard refs and folder items. Completion
prefix filtering for types and manifest keys. Overlay FS enables
validation with unsaved file contents. Expose ValidateAndFilterFromFS.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Tests for completions, hover, field index, workspace, overlay FS,
document store, server lifecycle, and live diagnostics with real
package validation.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Comment thread internal/lsp/position.go
ilyannn and others added 2 commits March 24, 2026 13:51
Remove unused validatePackage function, range over string directly
instead of []rune(string), align struct tags, regenerate README.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Manifest completions now resolve nested YAML context (e.g. completing
keys inside policy_templates.inputs or streams.vars). Value completions
suggest enum values and booleans from the package-spec JSON schemas.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Comment thread internal/lsp/manifest_schema.go
ilyannn and others added 2 commits April 7, 2026 15:48
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- position.go: use len(quotedValue)-2 for JSON range end to handle
  escaped characters correctly
- manifest_schema.go: store normalized schemaPath in valueCandidates
  so recursive branch calls use the resolved base path
- lsp-demo.sh: fix \n in sed for BSD/macOS compatibility; fix trap
  quoting to defer $BROKEN_DIR expansion

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Comment thread internal/lsp/position.go
lines := splitLines(text)
quotedValue := strconv.Quote(value)

for lineIndex, line := range lines {
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Low lsp/position.go:194

findJSONPropertyValueRange uses strings.Index byte offsets for protocol.Position.Character, but LSP requires UTF-16 code unit offsets. When the line contains multi-byte UTF-8 characters before the matched value, the computed positions are incorrect, causing diagnostic highlights to misalign in editors. Consider converting the byte offset to a UTF-16 code unit offset before constructing the position.

🚀 Reply "fix it for me" or copy this AI Prompt for your agent:
In file internal/lsp/position.go around line 194:

`findJSONPropertyValueRange` uses `strings.Index` byte offsets for `protocol.Position.Character`, but LSP requires UTF-16 code unit offsets. When the line contains multi-byte UTF-8 characters before the matched value, the computed positions are incorrect, causing diagnostic highlights to misalign in editors. Consider converting the byte offset to a UTF-16 code unit offset before constructing the position.

Evidence trail:
1. internal/lsp/position.go lines 190-213 at REVIEWED_COMMIT: shows `findJSONPropertyValueRange` using `strings.Index` (returns byte offset) and `len(quotedValue)` (returns byte count) directly as `Character` values in `protocol.Position`.
2. LSP specification reference from https://github.com/microsoft/language-server-protocol/issues/872 and https://github.com/microsoft/language-server-protocol/issues/376: Confirms LSP Position.Character requires UTF-16 code unit offsets by default, not byte offsets.
3. Go documentation confirms `strings.Index` returns byte index and `len(string)` returns byte count.

ilyannn and others added 2 commits April 7, 2026 16:47
…ions

Editors without YAML auto-indent (e.g. Neovim) place the cursor at
column 0 on a new blank line. With pos.Character=0 the line prefix was
empty, yamlIndent returned 0, and path resolution fell back to root
level — showing all top-level keys instead of the current block's
children.

Fix: when the cursor is at column 0 on a blank/whitespace-only line,
infer the expected indent from the nearest preceding non-blank line. A
bare key (no inline value) implies indent+2; a key-value pair implies
the same indent (sibling position).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…and subscription

Three related completion fixes:

1. Root keys shown even at non-zero column: linePrefixAtPosition clamps
   to line length, so pos.Character>0 on a truly blank line still yields
   linePrefix="". Widen the blank-line inference guard from
   pos.Character==0 to yamlIndent(linePrefix)==0 to cover this.

2. categories (and other array-of-enum fields): valueCandidates only
   looked at the top-level schema node. For array types the enum lives
   in items, not the array node itself. Add items traversal so both
   "- |" list-item and inline "categories: sec" positions return the
   full enum list.

3. When completing in list-item position and the array holds scalar
   values (no object keys), fall back to completeArrayEnumItems instead
   of returning nil.

conditions.elastic.subscription values were already working; added a
regression test to pin the behaviour.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@elasticmachine
Copy link
Copy Markdown
Collaborator

elasticmachine commented Apr 7, 2026

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants