Improve telemetry error classification to reduce 'unknown' error types#1409
Merged
eleanorjboyd merged 2 commits intomicrosoft:mainfrom Mar 27, 2026
Merged
Improve telemetry error classification to reduce 'unknown' error types#1409eleanorjboyd merged 2 commits intomicrosoft:mainfrom
eleanorjboyd merged 2 commits intomicrosoft:mainfrom
Conversation
eleanorjboyd
approved these changes
Mar 27, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The
classifyError()function in the telemetry error classifier had limited coverage — only 6 error categories were recognized. Many errors from manager registration (MANAGER_REGISTRATION_FAILED) were falling into the'unknown'bucket, making it difficult to diagnose failures from telemetry data.After tracing every code path that leads to
MANAGER_REGISTRATION_FAILEDtelemetry (pipenv, pyenv, poetry, conda, system, and shellStartupVars registration), we identified 6 common error patterns that were unclassified.Changes
Added 6 new error types to
DiscoveryErrorTypeand corresponding classification logic:tool_not_foundcommand_failedFailed to run "conda ...",Failed to run poetry ...,Error spawning conda: ...connection_errorinstanceof rpc.ConnectionErrorrpc_errorinstanceof rpc.ResponseErrorprocess_crashalready_registeredinstanceof BaseErrorEvery message pattern is based on exact
throw new Error(...)strings found in the codebase — no speculative matching.Ordering
The check order is designed to avoid misclassification:
instanceofchecks first (most reliable, no false positives)error.codechecks for Node.js errno codes'not found'is after'json'to avoid catching JSON validation errors)'unknown'as final fallbackTesting
Added 7 new test cases covering all new error types with real-world error messages from the codebase. All 15 tests pass.