Update kube-state-metrics from v2.13 to v2.18#49412
Open
Conversation
Update the kube-state-metrics dependency from v2.13 (fork based on CONS-7514 branch) to v2.18 (fork based on lenaic/public-discovery branch). All cherry-picked commits from the old fork (panic fixes, cronjob timezone features) are now in upstream. The only remaining fork change is moving internal/discovery/ to pkg/discovery/ so that external consumers can use customresourcestate.FromConfig(). Adapt to breaking API changes in KSM v2.18: - BuildStoresFunc/BuildCustomResourceStoresFunc: add limit int64 param - allowdenylist.IsIncluded/IsExcluded: handle new (bool, error) return - CRDiscoverer: add CRDsUpdateEventsCounter field (new in v2.18) - DefaultResources: endpoints replaced by endpointslices - Remove stale kube_endpoint_address_available and kube_endpoint_address_not_ready metric mapper entries (metrics removed upstream in v2.14) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Contributor
Go Package Import DifferencesBaseline: 695e39f
|
Contributor
Files inventory check summaryFile checks results against ancestor 695e39f7: Results for datadog-agent_7.79.0~devel.git.854.8f6ef10.pipeline.108530413-1_amd64.deb:No change detected |
Regression DetectorRegression Detector ResultsMetrics dashboard Baseline: 695e39f Optimization Goals: ✅ No significant changes detected
|
| perf | experiment | goal | Δ mean % | Δ mean % CI | trials | links |
|---|---|---|---|---|---|---|
| ➖ | docker_containers_cpu | % cpu utilization | -2.53 | [-5.48, +0.43] | 1 | Logs |
Fine details of change detection per experiment
| perf | experiment | goal | Δ mean % | Δ mean % CI | trials | links |
|---|---|---|---|---|---|---|
| ➖ | quality_gate_logs | % cpu utilization | +0.86 | [-0.77, +2.48] | 1 | Logs bounds checks dashboard |
| ➖ | file_tree | memory utilization | +0.74 | [+0.68, +0.80] | 1 | Logs |
| ➖ | quality_gate_metrics_logs | memory utilization | +0.72 | [+0.48, +0.96] | 1 | Logs bounds checks dashboard |
| ➖ | quality_gate_idle | memory utilization | +0.50 | [+0.45, +0.55] | 1 | Logs bounds checks dashboard |
| ➖ | quality_gate_idle_all_features | memory utilization | +0.33 | [+0.29, +0.37] | 1 | Logs bounds checks dashboard |
| ➖ | ddot_metrics_sum_cumulative | memory utilization | +0.31 | [+0.16, +0.46] | 1 | Logs |
| ➖ | uds_dogstatsd_20mb_12k_contexts_20_senders | memory utilization | +0.23 | [+0.17, +0.30] | 1 | Logs |
| ➖ | docker_containers_memory | memory utilization | +0.22 | [+0.13, +0.32] | 1 | Logs |
| ➖ | ddot_metrics_sum_cumulativetodelta_exporter | memory utilization | +0.12 | [-0.11, +0.34] | 1 | Logs |
| ➖ | tcp_syslog_to_blackhole | ingress throughput | +0.08 | [-0.13, +0.28] | 1 | Logs |
| ➖ | file_to_blackhole_100ms_latency | egress throughput | +0.07 | [-0.03, +0.18] | 1 | Logs |
| ➖ | otlp_ingest_logs | memory utilization | +0.07 | [-0.04, +0.17] | 1 | Logs |
| ➖ | file_to_blackhole_0ms_latency | egress throughput | +0.06 | [-0.46, +0.59] | 1 | Logs |
| ➖ | file_to_blackhole_1000ms_latency | egress throughput | +0.01 | [-0.41, +0.44] | 1 | Logs |
| ➖ | tcp_dd_logs_filter_exclude | ingress throughput | +0.01 | [-0.12, +0.13] | 1 | Logs |
| ➖ | uds_dogstatsd_to_api_v3 | ingress throughput | -0.00 | [-0.20, +0.19] | 1 | Logs |
| ➖ | uds_dogstatsd_to_api | ingress throughput | -0.01 | [-0.21, +0.18] | 1 | Logs |
| ➖ | file_to_blackhole_500ms_latency | egress throughput | -0.09 | [-0.49, +0.30] | 1 | Logs |
| ➖ | ddot_metrics | memory utilization | -0.12 | [-0.30, +0.07] | 1 | Logs |
| ➖ | ddot_metrics_sum_delta | memory utilization | -0.19 | [-0.37, -0.02] | 1 | Logs |
| ➖ | ddot_logs | memory utilization | -0.28 | [-0.35, -0.22] | 1 | Logs |
| ➖ | otlp_ingest_metrics | memory utilization | -1.05 | [-1.20, -0.90] | 1 | Logs |
| ➖ | docker_containers_cpu | % cpu utilization | -2.53 | [-5.48, +0.43] | 1 | Logs |
Bounds Checks: ✅ Passed
| perf | experiment | bounds_check_name | replicates_passed | observed_value | links |
|---|---|---|---|---|---|
| ✅ | docker_containers_cpu | simple_check_run | 10/10 | 713 ≥ 26 | |
| ✅ | docker_containers_memory | memory_usage | 10/10 | 275.92MiB ≤ 370MiB | |
| ✅ | docker_containers_memory | simple_check_run | 10/10 | 688 ≥ 26 | |
| ✅ | file_to_blackhole_0ms_latency | memory_usage | 10/10 | 0.19GiB ≤ 1.20GiB | |
| ✅ | file_to_blackhole_0ms_latency | missed_bytes | 10/10 | 0B = 0B | |
| ✅ | file_to_blackhole_1000ms_latency | memory_usage | 10/10 | 0.24GiB ≤ 1.20GiB | |
| ✅ | file_to_blackhole_1000ms_latency | missed_bytes | 10/10 | 0B = 0B | |
| ✅ | file_to_blackhole_100ms_latency | memory_usage | 10/10 | 0.20GiB ≤ 1.20GiB | |
| ✅ | file_to_blackhole_100ms_latency | missed_bytes | 10/10 | 0B = 0B | |
| ✅ | file_to_blackhole_500ms_latency | memory_usage | 10/10 | 0.22GiB ≤ 1.20GiB | |
| ✅ | file_to_blackhole_500ms_latency | missed_bytes | 10/10 | 0B = 0B | |
| ✅ | quality_gate_idle | intake_connections | 10/10 | 4 = 4 | bounds checks dashboard |
| ✅ | quality_gate_idle | memory_usage | 10/10 | 175.24MiB ≤ 181MiB | bounds checks dashboard |
| ✅ | quality_gate_idle_all_features | intake_connections | 10/10 | 4 = 4 | bounds checks dashboard |
| ✅ | quality_gate_idle_all_features | memory_usage | 10/10 | 500.68MiB ≤ 550MiB | bounds checks dashboard |
| ✅ | quality_gate_logs | intake_connections | 10/10 | 4 ≤ 6 | bounds checks dashboard |
| ✅ | quality_gate_logs | memory_usage | 10/10 | 210.76MiB ≤ 220MiB | bounds checks dashboard |
| ✅ | quality_gate_logs | missed_bytes | 10/10 | 0B = 0B | bounds checks dashboard |
| ✅ | quality_gate_metrics_logs | cpu_usage | 10/10 | 365.49 ≤ 2000 | bounds checks dashboard |
| ✅ | quality_gate_metrics_logs | intake_connections | 10/10 | 4 ≤ 6 | bounds checks dashboard |
| ✅ | quality_gate_metrics_logs | memory_usage | 10/10 | 420.20MiB ≤ 475MiB | bounds checks dashboard |
| ✅ | quality_gate_metrics_logs | missed_bytes | 10/10 | 0B = 0B | bounds checks dashboard |
Explanation
Confidence level: 90.00%
Effect size tolerance: |Δ mean %| ≥ 5.00%
Performance changes are noted in the perf column of each table:
- ✅ = significantly better comparison variant performance
- ❌ = significantly worse comparison variant performance
- ➖ = no significant change in performance
A regression test is an A/B test of target performance in a repeatable rig, where "performance" is measured as "comparison variant minus baseline variant" for an optimization goal (e.g., ingress throughput). Due to intrinsic variability in measuring that goal, we can only estimate its mean value for each experiment; we report uncertainty in that value as a 90.00% confidence interval denoted "Δ mean % CI".
For each experiment, we decide whether a change in performance is a "regression" -- a change worth investigating further -- if all of the following criteria are true:
-
Its estimated |Δ mean %| ≥ 5.00%, indicating the change is big enough to merit a closer look.
-
Its 90.00% confidence interval "Δ mean % CI" does not contain zero, indicating that if our statistical model is accurate, there is at least a 90.00% chance there is a difference in performance between baseline and comparison variants.
-
Its configuration does not mark it "erratic".
CI Pass/Fail Decision
✅ Passed. All Quality Gates passed.
- quality_gate_logs, bounds check missed_bytes: 10/10 replicas passed. Gate passed.
- quality_gate_logs, bounds check intake_connections: 10/10 replicas passed. Gate passed.
- quality_gate_logs, bounds check memory_usage: 10/10 replicas passed. Gate passed.
- quality_gate_metrics_logs, bounds check missed_bytes: 10/10 replicas passed. Gate passed.
- quality_gate_metrics_logs, bounds check intake_connections: 10/10 replicas passed. Gate passed.
- quality_gate_metrics_logs, bounds check memory_usage: 10/10 replicas passed. Gate passed.
- quality_gate_metrics_logs, bounds check cpu_usage: 10/10 replicas passed. Gate passed.
- quality_gate_idle, bounds check memory_usage: 10/10 replicas passed. Gate passed.
- quality_gate_idle, bounds check intake_connections: 10/10 replicas passed. Gate passed.
- quality_gate_idle_all_features, bounds check intake_connections: 10/10 replicas passed. Gate passed.
- quality_gate_idle_all_features, bounds check memory_usage: 10/10 replicas passed. Gate passed.
The kube_endpoint_address_available and kube_endpoint_address_not_ready
KSM metrics were removed in v2.14 and replaced by kube_endpoint_address
with a "ready" label ("true" for available, "false" for not ready).
Add a transformer that splits kube_endpoint_address back into the two
legacy Datadog metrics (endpoint.address_available and
endpoint.address_not_ready) to maintain backward compatibility.
The stale metric mapper entries are removed since the transformer now
handles the metric emission directly.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
KSM v2.18 exposes endpointslice metrics alongside endpoint metrics. Add support for the new kube_endpointslice_endpoints metric by following the same pattern as the existing endpoint metrics: - Add endpointSliceEndpointsTransformer that splits kube_endpointslice_endpoints by its "ready" label into endpointslice.address_available and endpointslice.address_not_ready - Add "endpointslice" label mapper to produce kube_endpointslice tags The remaining endpointslice KSM metrics follow the same behavior as their endpoint counterparts: kube_endpointslice_info is used for label joins, kube_endpointslice_created is denied by default, and kube_endpointslice_ports is not mapped (same as kube_endpoint_ports). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Replace deprecated netipx.AddrNext() with netip.Addr.Next() in npcollector_test.go (go4.org/netipx was bumped transitively) - Regenerate LICENSE-3rdparty.csv for new/updated dependencies Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
L3n41c
added
qa/done
Member
Author
|
@codex review |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: bbc79c065a
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
… release note - Treat empty "ready" label as "true" in endpointSliceEndpointsTransformer (per K8s API, nil Ready means the endpoint is ready to receive traffic) - Update go.mod replace comment to reference upstream PR kubernetes/kube-state-metrics#2928 - Add release note documenting new endpointslice metrics and the endpoints → endpointslices default collector change Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Upstream KSM v2.18 replaced "endpoints" with "endpointslices" in its default resource list. To avoid a breaking change for users who rely on endpoint metrics without explicit collector configuration, add "endpoints" back to the default collectors when it is not already present in the upstream defaults. This applies to both the KSM check (kubernetes_state.go) and the sharding logic (ksm_sharding.go). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Member
Author
|
@codex review |
|
Codex Review: Didn't find any major issues. Keep it up! ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
L3n41c
changed the title
kube-state-metrics from v2.13 to v2.18
davidor
approved these changes
Apr 16, 2026
Member
davidor
left a comment
davidor
left a comment
There was a problem hiding this comment.
👍 for container-platform (we only own 1 file)
…ate-metrics-v2.18 # Conflicts: # comp/otelcol/collector-contrib/impl/go.sum # comp/otelcol/ddflareextension/impl/go.sum # comp/otelcol/status/impl/go.sum # go.sum # internal/tools/go.sum # pkg/util/prometheus/go.sum
…ate-metrics-v2.18
…ate-metrics-v2.18
lavigne958
approved these changes
Apr 17, 2026
ken-schneider
approved these changes
Apr 17, 2026
L3n41c
force-pushed
the
lenaic/update-kube-state-metrics-v2.18
branch
from
38b4121 to
8f6ef10
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Updates the
kube-state-metricsdependency from v2.13 to v2.18.The previous fork (
L3n41c/kube-state-metrics, branchCONS-7514, based on v2.13.0) contained:GVRFromType— now in upstreaminternal/discovery/public — still neededThe new fork (
L3n41c/kube-state-metrics, branchlenaic/public-discovery, based on v2.18.0) contains only the single commit movinginternal/discovery/topkg/discovery/, which is required becausecustomresourcestate.FromConfig()takes a*discovery.CRDiscovererparameter from an internal package.The use of this branch is temporary until kubernetes/kube-state-metrics#2928 is merged upstream.
Code changes to adapt to KSM v2.18 breaking API changes:
BuildStoresFunc/BuildCustomResourceStoresFunc: addedlimit int64parameter (unused — the agent does not use KSM's object limiting)allowdenylist.IsIncluded()/IsExcluded(): now return(bool, error)— updated test assertionsCRDiscoverer: addedCRDsUpdateEventsCounterfield (new in v2.18)DefaultResources: upstream replacedendpointswithendpointslices— updated test expectationskube_endpoint_address_availableandkube_endpoint_address_not_ready(metrics removed upstream in v2.14)Motivation
The fork was significantly behind upstream (v2.13 vs v2.18). All cherry-picked changes from the old fork are now in upstream, so this is a good time to rebase on the latest release and minimize the fork delta to a single commit.
Describe how you validated your changes
dda inv cluster-agent.build— builds successfullydda inv test --targets=./pkg/collector/corechecks/cluster/ksm/...— 301 tests passeddda inv test --targets=./pkg/kubestatemetrics/...— 15 tests passeddda inv test --targets=./pkg/clusteragent/clusterchecks/...— 118 tests passedAdditional Notes
The
replacedirective still points toL3n41c/kube-state-metricsbecause upstream KSM keeps thediscoverypackage underinternal/. A PR to make it public should be submitted tokubernetes/kube-state-metricsto eventually remove the fork entirely.The
sum.golang.orgchecksum database may not have indexed the fork'sv2.18.0tag yet. CI may needGONOSUMDB=github.com/L3n41c/kube-state-metrics/v2andGONOSUMCHECK=github.com/L3n41c/kube-state-metrics/v2temporarily.