perf: optimize count distinct fast path and sliding count distinct#21782
perf: optimize count distinct fast path and sliding count distinct#21782lyne7-sc wants to merge 6 commits intoapache:mainfrom
Conversation
|
run benchmarks |
|
🤖 Benchmark running (GKE) | trigger CPU Details (lscpu)Comparing perf-count-distinct-clean (d71d55b) to 4bff17e (merge-base) diff using: clickbench_partitioned File an issue against this benchmark runner |
|
🤖 Benchmark running (GKE) | trigger CPU Details (lscpu)Comparing perf-count-distinct-clean (d71d55b) to 4bff17e (merge-base) diff using: tpcds File an issue against this benchmark runner |
|
🤖 Benchmark running (GKE) | trigger CPU Details (lscpu)Comparing perf-count-distinct-clean (d71d55b) to 4bff17e (merge-base) diff using: tpch File an issue against this benchmark runner |
|
🤖 Benchmark completed (GKE) | trigger Instance: CPU Details (lscpu)Details
Resource Usagetpch — base (merge-base)
tpch — branch
File an issue against this benchmark runner |
|
🤖 Benchmark completed (GKE) | trigger Instance: CPU Details (lscpu)Details
Resource Usageclickbench_partitioned — base (merge-base)
clickbench_partitioned — branch
File an issue against this benchmark runner |
|
🤖 Benchmark completed (GKE) | trigger Instance: CPU Details (lscpu)Details
Resource Usagetpcds — base (merge-base)
tpcds — branch
File an issue against this benchmark runner |
|
synced with |
Which issue does this PR close?
Rationale for this change
This PR improves two hot paths for count distinct.
For the primitive non-sliding path, it adds a fast path to avoid per-row null handling when the input batch contains no nulls.
For the sliding path, it adds a primitive accumulator and keeps other types on the generic path for now.
What changes are included in this PR?
This pr adds a null-free fast path for primitive
COUNT(DISTINCT)and a primitive sliding distinct accumulator for integer types.Benchmarks
Are these changes tested?
yes
Are there any user-facing changes?
no