Skip to content

[WIP] Experimental OTEL profiling support#1754

Draft
danielsn wants to merge 4 commits intomainfrom
dsn/otel-impl-for-python
Draft

[WIP] Experimental OTEL profiling support#1754
danielsn wants to merge 4 commits intomainfrom
dsn/otel-impl-for-python

Conversation

@danielsn
Copy link
Contributor

@danielsn danielsn commented Mar 17, 2026

What does this PR do?

This is an experimental branch adding a new "otel" feature to the profiler, which

  1. Uses an "otel shaped" internal representation
  2. adds an otel protobuf definition
  3. has code to generate either pprof or otel

Motivation

OTEL is coming, lets see what it would take to support it.

Additional Notes

Doens't currently support

  1. upscaling
  2. profilers that have more than two sample types at the same time
  3. streaming generation of the otel proto: we build the entire thing then emit it.

How to test the change?

Describe here in detail how the change can be validated.

@dd-octo-sts
Copy link
Contributor

dd-octo-sts bot commented Mar 17, 2026

Artifact Size Benchmark Report

aarch64-alpine-linux-musl
Artifact Baseline Commit Change
/aarch64-alpine-linux-musl/lib/libdatadog_profiling.a 91.49 MB 91.51 MB +.02% (+21.50 KB) 🔍
/aarch64-alpine-linux-musl/lib/libdatadog_profiling.so 8.38 MB 8.38 MB 0% (0 B) 👌
aarch64-unknown-linux-gnu
Artifact Baseline Commit Change
/aarch64-unknown-linux-gnu/lib/libdatadog_profiling.a 106.22 MB 106.23 MB +0% (+7.82 KB) 👌
/aarch64-unknown-linux-gnu/lib/libdatadog_profiling.so 10.75 MB 10.75 MB +0% (+528 B) 👌
libdatadog-x64-windows
Artifact Baseline Commit Change
/libdatadog-x64-windows/debug/dynamic/datadog_profiling_ffi.dll 24.97 MB 24.97 MB +.01% (+3.00 KB) 🔍
/libdatadog-x64-windows/debug/dynamic/datadog_profiling_ffi.lib 75.94 KB 75.94 KB 0% (0 B) 👌
/libdatadog-x64-windows/debug/dynamic/datadog_profiling_ffi.pdb 166.41 MB 166.44 MB +.01% (+32.00 KB) 🔍
/libdatadog-x64-windows/debug/static/datadog_profiling_ffi.lib 837.48 MB 837.51 MB +0% (+28.96 KB) 👌
/libdatadog-x64-windows/release/dynamic/datadog_profiling_ffi.dll 9.55 MB 9.55 MB +0% (+512 B) 👌
/libdatadog-x64-windows/release/dynamic/datadog_profiling_ffi.lib 75.94 KB 75.94 KB 0% (0 B) 👌
/libdatadog-x64-windows/release/dynamic/datadog_profiling_ffi.pdb 22.99 MB 23.00 MB +.03% (+8.00 KB) 🔍
/libdatadog-x64-windows/release/static/datadog_profiling_ffi.lib 48.35 MB 48.36 MB +.01% (+5.50 KB) 🔍
libdatadog-x86-windows
Artifact Baseline Commit Change
/libdatadog-x86-windows/debug/dynamic/datadog_profiling_ffi.dll 21.05 MB 21.05 MB +.01% (+3.50 KB) 🔍
/libdatadog-x86-windows/debug/dynamic/datadog_profiling_ffi.lib 77.12 KB 77.12 KB 0% (0 B) 👌
/libdatadog-x86-windows/debug/dynamic/datadog_profiling_ffi.pdb 170.15 MB 170.19 MB +.02% (+40.00 KB) 🔍
/libdatadog-x86-windows/debug/static/datadog_profiling_ffi.lib 823.53 MB 823.36 MB --.02% (-177.83 KB) 💪
/libdatadog-x86-windows/release/dynamic/datadog_profiling_ffi.dll 7.23 MB 7.23 MB +.01% (+1.00 KB) 🔍
/libdatadog-x86-windows/release/dynamic/datadog_profiling_ffi.lib 77.12 KB 77.12 KB 0% (0 B) 👌
/libdatadog-x86-windows/release/dynamic/datadog_profiling_ffi.pdb 24.58 MB 24.59 MB +.03% (+8.00 KB) 🔍
/libdatadog-x86-windows/release/static/datadog_profiling_ffi.lib 44.09 MB 44.10 MB +.01% (+5.66 KB) 🔍
x86_64-alpine-linux-musl
Artifact Baseline Commit Change
/x86_64-alpine-linux-musl/lib/libdatadog_profiling.a 80.04 MB 80.04 MB +0% (+6.47 KB) 👌
/x86_64-alpine-linux-musl/lib/libdatadog_profiling.so 9.86 MB 9.86 MB 0% (0 B) 👌
x86_64-unknown-linux-gnu
Artifact Baseline Commit Change
/x86_64-unknown-linux-gnu/lib/libdatadog_profiling.a 100.17 MB 100.17 MB +0% (+7.46 KB) 👌
/x86_64-unknown-linux-gnu/lib/libdatadog_profiling.so 11.43 MB 11.43 MB +0% (+384 B) 👌

@codecov-commenter
Copy link

Codecov Report

❌ Patch coverage is 57.22022% with 237 lines in your changes missing coverage. Please review.
✅ Project coverage is 69.80%. Comparing base (27de9f3) to head (bca3ebd).
⚠️ Report is 68 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1754      +/-   ##
==========================================
- Coverage   71.17%   69.80%   -1.38%     
==========================================
  Files         423      429       +6     
  Lines       62063    64056    +1993     
==========================================
+ Hits        44173    44712     +539     
- Misses      17890    19344    +1454     
Components Coverage Δ
libdd-crashtracker 63.98% <ø> (+1.43%) ⬆️
libdd-crashtracker-ffi 18.19% <ø> (+2.39%) ⬆️
libdd-alloc 98.77% <ø> (ø)
libdd-data-pipeline 87.94% <ø> (+1.00%) ⬆️
libdd-data-pipeline-ffi 74.85% <ø> (+0.45%) ⬆️
libdd-common 79.73% <ø> (-0.86%) ⬇️
libdd-common-ffi 73.40% <ø> (-0.35%) ⬇️
libdd-telemetry 62.48% <ø> (-0.04%) ⬇️
libdd-telemetry-ffi 16.75% <ø> (ø)
libdd-dogstatsd-client 82.64% <ø> (ø)
datadog-ipc 80.47% <ø> (-0.37%) ⬇️
libdd-profiling 70.27% <57.22%> (-11.27%) ⬇️
libdd-profiling-ffi 63.65% <ø> (-0.02%) ⬇️
datadog-sidecar 33.10% <ø> (+0.32%) ⬆️
datdog-sidecar-ffi 10.79% <ø> (+1.29%) ⬆️
spawn-worker 54.69% <ø> (ø)
libdd-tinybytes 93.16% <ø> (ø)
libdd-trace-normalization 81.71% <ø> (ø)
libdd-trace-obfuscation 91.80% <ø> (-2.41%) ⬇️
libdd-trace-protobuf 68.25% <ø> (+0.25%) ⬆️
libdd-trace-utils 88.98% <ø> (-0.12%) ⬇️
datadog-tracer-flare 90.46% <ø> (+1.50%) ⬆️
libdd-log 74.69% <ø> (ø)
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@github-actions
Copy link

Clippy Allow Annotation Report

Comparing clippy allow annotations between branches:

  • Base Branch: origin/main
  • PR Branch: origin/dsn/otel-impl-for-python

Summary by Rule

Rule Base Branch PR Branch Change
expect_used 2 2 No change (0%)
unwrap_used 1 7 ⚠️ +6 (+600.0%)
Total 3 9 ⚠️ +6 (+200.0%)

Annotation Counts by File

File Base Branch PR Branch Change
libdd-profiling/src/internal/location.rs 1 1 No change (0%)
libdd-profiling/src/internal/mapping.rs 1 1 No change (0%)
libdd-profiling/src/internal/otel_style_observation.rs 0 6 ⚠️ +6 (N/A)
libdd-profiling/src/internal/profile/mod.rs 1 1 No change (0%)

Annotation Stats by Crate

Crate Base Branch PR Branch Change
clippy-annotation-reporter 5 5 No change (0%)
datadog-ffe-ffi 1 1 No change (0%)
datadog-ipc 28 27 ✅ -1 (-3.6%)
datadog-live-debugger 6 6 No change (0%)
datadog-live-debugger-ffi 10 10 No change (0%)
datadog-profiling-replayer 4 4 No change (0%)
datadog-remote-config 3 3 No change (0%)
datadog-sidecar 59 59 No change (0%)
libdd-common 10 10 No change (0%)
libdd-common-ffi 12 12 No change (0%)
libdd-crashtracker 0 12 ⚠️ +12 (N/A)
libdd-data-pipeline 5 5 No change (0%)
libdd-ddsketch 2 2 No change (0%)
libdd-dogstatsd-client 1 1 No change (0%)
libdd-profiling 13 19 ⚠️ +6 (+46.2%)
libdd-telemetry 19 19 No change (0%)
libdd-tinybytes 4 4 No change (0%)
libdd-trace-normalization 2 2 No change (0%)
libdd-trace-obfuscation 9 9 No change (0%)
libdd-trace-utils 15 15 No change (0%)
Total 208 225 ⚠️ +17 (+8.2%)

About This Report

This report tracks Clippy allow annotations for specific rules, showing how they've changed in this PR. Decreasing the number of these annotations generally improves code quality.

@github-actions
Copy link

📚 Documentation Check Results

⚠️ 645 documentation warning(s) found

📦 libdd-profiling - 645 warning(s)


Updated: 2026-03-17 22:29:50 UTC | Commit: 9de1314 | missing-docs job results

@github-actions
Copy link

🔒 Cargo Deny Results

No issues found!

📦 libdd-profiling - ✅ No issues


Updated: 2026-03-17 22:32:58 UTC | Commit: 9de1314 | dependency-check job results

@danielsn danielsn changed the title Dsn/otel impl for python [WIP] Experimental OTEL profiling support Mar 18, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants