[procmgr] Write DDOT procmgr YAML in post-install hooks#49591
Open
jose-manuel-almaza wants to merge 4 commits intojose/procmgr-build-defaultfrom
Open
[procmgr] Write DDOT procmgr YAML in post-install hooks#49591jose-manuel-almaza wants to merge 4 commits intojose/procmgr-build-defaultfrom
jose-manuel-almaza wants to merge 4 commits intojose/procmgr-build-defaultfrom
Conversation
Embed all four DDOT process manager YAML variants (oci/debrpm x stable/experiment) and write the appropriate one to the agent's processes.d/ directory during DDOT post-install. This causes dd-procmgrd to manage DDOT instead of systemd, since the systemd unit's ConditionPathExists=! skips itself when the YAML is present. If dd-procmgrd is not installed (binary not found), the write is skipped and DDOT falls back to systemd management. On package removal, the YAML is deleted so systemd can resume managing DDOT if needed.
jose-manuel-almaza
added
changelog/no-changelog
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 8f6d26f3ed
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Contributor
Gitlab CI Configuration ChangesModified Jobs.agent_build_common .agent_build_common:
artifacts:
expire_in: 2 weeks
paths:
- $OMNIBUS_PACKAGE_DIR
before_script:
- mkdir -p .cargo
- 'cat > .cargo/config.toml << ''EOF''
[registries]
datadog = { index = "sparse+https://depot-read-api-rust.us1.ddbuild.io/datadog/rust/"
}
[registry]
default = "datadog"
[source]
[source.crates-io]
replace-with = "datadog"
EOF
'
cache:
- key:
files:
- omnibus/Gemfile
- release.json
prefix: omnibus-deps-$CI_JOB_IMAGE-$CI_JOB_NAME-$OMNIBUS_RUBY_VERSION
paths:
- omnibus/vendor/bundle
- key:
files:
- .bazelversion
prefix: bazelversion-$CI_RUNNER_DESCRIPTION
paths:
- .cache/bazelisk
- .cache/bazel/*/install
policy: pull$BAZEL_CACHE_POLICY_SUFFIX
when: on_success
- key:
files:
- .go-version
- .python-version
prefix: bazel-$CI_JOB_NAME
paths:
- .cache/bazel/*/cache
- .cache/bazel/disk-cache
- .cache/go
- .cache/ms-go
- .cache/pip
policy: pull$BAZEL_CACHE_POLICY_SUFFIX
when: on_success
id_tokens:
BUILDBARN_ID_TOKEN:
aud: buildbarn.us1.ddbuild.io
needs: []
rules:
- if: $CI_COMMIT_BRANCH =~ /^mq-working-branch-/
when: never
- when: on_success
script:
- mkdir -p $GOPATH/pkg/mod/cache && zstd -dc modcache.tar.zst | tar xf - -C $GOPATH/pkg/mod/cache
- rm -f modcache.tar.zst
- pushd omnibus && bundle config set --local path 'vendor/bundle' && popd
- rm -rf $OMNIBUS_PACKAGE_DIR/*
- tar -xf $CI_PROJECT_DIR/sysprobe-build-outputs.tar.xz
- mkdir -p /tmp/system-probe
- ${S3_CP_CMD} "${S3_PERMANENT_ARTIFACTS_URI}/clang-${CLANG_LLVM_VER}.${PACKAGE_ARCH}.${CLANG_BUILD_VERSION}"
/tmp/system-probe/clang-bpf
- ${S3_CP_CMD} "${S3_PERMANENT_ARTIFACTS_URI}/llc-${CLANG_LLVM_VER}.${PACKAGE_ARCH}.${CLANG_BUILD_VERSION}"
/tmp/system-probe/llc-bpf
- cp $CI_PROJECT_DIR/minimized-btfs.tar.xz /tmp/system-probe/minimized-btfs.tar.xz
- chmod 0744 /tmp/system-probe/clang-bpf /tmp/system-probe/llc-bpf
- dda inv -- -e omnibus.build --base-dir $OMNIBUS_BASE_DIR --skip-deps --go-mod-cache="$GOPATH/pkg/mod"
- --system-probe-bin=/tmp/system-probe --with-dd-procmgrd --flavor "$FLAVOR" --config-directory
? -------------------
+ --system-probe-bin=/tmp/system-probe --flavor "$FLAVOR" --config-directory "$CONFIG_DIR"
? ++++++++++++++
- "$CONFIG_DIR" --install-directory "$INSTALL_DIR"
? --------------
+ --install-directory "$INSTALL_DIR"
- ls -la $OMNIBUS_PACKAGE_DIR
- $S3_CP_CMD $OMNIBUS_PACKAGE_DIR/version-manifest.json $S3_SBOM_STORAGE_URI/$CI_JOB_NAME/version-manifest.json
stage: package_build
variables:
BAZELISK_HOME: $XDG_CACHE_HOME/bazelisk
KUBERNETES_CPU_REQUEST: 16
KUBERNETES_MEMORY_LIMIT: 32Gi
KUBERNETES_MEMORY_REQUEST: 32Gi
XDG_CACHE_HOME: $CI_PROJECT_DIR/.cache.agent_build_script .agent_build_script:
- mkdir -p $GOPATH/pkg/mod/cache && zstd -dc modcache.tar.zst | tar xf - -C $GOPATH/pkg/mod/cache
- rm -f modcache.tar.zst
- pushd omnibus && bundle config set --local path 'vendor/bundle' && popd
- rm -rf $OMNIBUS_PACKAGE_DIR/*
- tar -xf $CI_PROJECT_DIR/sysprobe-build-outputs.tar.xz
- mkdir -p /tmp/system-probe
- ${S3_CP_CMD} "${S3_PERMANENT_ARTIFACTS_URI}/clang-${CLANG_LLVM_VER}.${PACKAGE_ARCH}.${CLANG_BUILD_VERSION}"
/tmp/system-probe/clang-bpf
- ${S3_CP_CMD} "${S3_PERMANENT_ARTIFACTS_URI}/llc-${CLANG_LLVM_VER}.${PACKAGE_ARCH}.${CLANG_BUILD_VERSION}"
/tmp/system-probe/llc-bpf
- cp $CI_PROJECT_DIR/minimized-btfs.tar.xz /tmp/system-probe/minimized-btfs.tar.xz
- chmod 0744 /tmp/system-probe/clang-bpf /tmp/system-probe/llc-bpf
- dda inv -- -e omnibus.build --base-dir $OMNIBUS_BASE_DIR --skip-deps --go-mod-cache="$GOPATH/pkg/mod"
- --system-probe-bin=/tmp/system-probe --with-dd-procmgrd --flavor "$FLAVOR" --config-directory
? -------------------
+ --system-probe-bin=/tmp/system-probe --flavor "$FLAVOR" --config-directory "$CONFIG_DIR"
? ++++++++++++++
- "$CONFIG_DIR" --install-directory "$INSTALL_DIR"
? --------------
+ --install-directory "$INSTALL_DIR"
- ls -la $OMNIBUS_PACKAGE_DIR
- $S3_CP_CMD $OMNIBUS_PACKAGE_DIR/version-manifest.json $S3_SBOM_STORAGE_URI/$CI_JOB_NAME/version-manifest.jsondatadog-agent-7-arm64 datadog-agent-7-arm64:
artifacts:
expire_in: 2 weeks
paths:
- $OMNIBUS_PACKAGE_DIR
before_script:
- mkdir -p .cargo
- 'cat > .cargo/config.toml << ''EOF''
[registries]
datadog = { index = "sparse+https://depot-read-api-rust.us1.ddbuild.io/datadog/rust/"
}
[registry]
default = "datadog"
[source]
[source.crates-io]
replace-with = "datadog"
EOF
'
cache:
- key:
files:
- omnibus/Gemfile
- release.json
prefix: omnibus-deps-$CI_JOB_IMAGE-$CI_JOB_NAME-$OMNIBUS_RUBY_VERSION
paths:
- omnibus/vendor/bundle
- key:
files:
- .bazelversion
prefix: bazelversion-$CI_RUNNER_DESCRIPTION
paths:
- .cache/bazelisk
- .cache/bazel/*/install
policy: pull$BAZEL_CACHE_POLICY_SUFFIX
when: on_success
- key:
files:
- .go-version
- .python-version
prefix: bazel-$CI_JOB_NAME
paths:
- .cache/bazel/*/cache
- .cache/bazel/disk-cache
- .cache/go
- .cache/ms-go
- .cache/pip
policy: pull$BAZEL_CACHE_POLICY_SUFFIX
when: on_success
id_tokens:
BUILDBARN_ID_TOKEN:
aud: buildbarn.us1.ddbuild.io
image: registry.ddbuild.io/ci/datadog-agent-buildimages/linux$CI_IMAGE_LINUX_SUFFIX:$CI_IMAGE_LINUX
needs:
- build_system-probe-arm64
- go_deps
- generate_minimized_btfs_arm64
rules:
- when: on_success
script:
- mkdir -p $GOPATH/pkg/mod/cache && zstd -dc modcache.tar.zst | tar xf - -C $GOPATH/pkg/mod/cache
- rm -f modcache.tar.zst
- pushd omnibus && bundle config set --local path 'vendor/bundle' && popd
- rm -rf $OMNIBUS_PACKAGE_DIR/*
- tar -xf $CI_PROJECT_DIR/sysprobe-build-outputs.tar.xz
- mkdir -p /tmp/system-probe
- ${S3_CP_CMD} "${S3_PERMANENT_ARTIFACTS_URI}/clang-${CLANG_LLVM_VER}.${PACKAGE_ARCH}.${CLANG_BUILD_VERSION}"
/tmp/system-probe/clang-bpf
- ${S3_CP_CMD} "${S3_PERMANENT_ARTIFACTS_URI}/llc-${CLANG_LLVM_VER}.${PACKAGE_ARCH}.${CLANG_BUILD_VERSION}"
/tmp/system-probe/llc-bpf
- cp $CI_PROJECT_DIR/minimized-btfs.tar.xz /tmp/system-probe/minimized-btfs.tar.xz
- chmod 0744 /tmp/system-probe/clang-bpf /tmp/system-probe/llc-bpf
- dda inv -- -e omnibus.build --base-dir $OMNIBUS_BASE_DIR --skip-deps --go-mod-cache="$GOPATH/pkg/mod"
- --system-probe-bin=/tmp/system-probe --with-dd-procmgrd --flavor "$FLAVOR" --config-directory
? -------------------
+ --system-probe-bin=/tmp/system-probe --flavor "$FLAVOR" --config-directory "$CONFIG_DIR"
? ++++++++++++++
- "$CONFIG_DIR" --install-directory "$INSTALL_DIR"
? --------------
+ --install-directory "$INSTALL_DIR"
- ls -la $OMNIBUS_PACKAGE_DIR
- $S3_CP_CMD $OMNIBUS_PACKAGE_DIR/version-manifest.json $S3_SBOM_STORAGE_URI/$CI_JOB_NAME/version-manifest.json
stage: package_build
tags:
- arch:arm64
- specific:true
variables:
BAZELISK_HOME: $XDG_CACHE_HOME/bazelisk
DD_CC: aarch64-unknown-linux-gnu-gcc
DD_CXX: aarch64-unknown-linux-gnu-g++
FLAVOR: base
KUBERNETES_CPU_REQUEST: 16
KUBERNETES_MEMORY_LIMIT: 32Gi
KUBERNETES_MEMORY_REQUEST: 32Gi
PACKAGE_ARCH: arm64
XDG_CACHE_HOME: $CI_PROJECT_DIR/.cachedatadog-agent-7-arm64-fips datadog-agent-7-arm64-fips:
artifacts:
expire_in: 2 weeks
paths:
- $OMNIBUS_PACKAGE_DIR
before_script:
- mkdir -p .cargo
- 'cat > .cargo/config.toml << ''EOF''
[registries]
datadog = { index = "sparse+https://depot-read-api-rust.us1.ddbuild.io/datadog/rust/"
}
[registry]
default = "datadog"
[source]
[source.crates-io]
replace-with = "datadog"
EOF
'
cache:
- key:
files:
- omnibus/Gemfile
- release.json
prefix: omnibus-deps-$CI_JOB_IMAGE-$CI_JOB_NAME-$OMNIBUS_RUBY_VERSION
paths:
- omnibus/vendor/bundle
- key:
files:
- .bazelversion
prefix: bazelversion-$CI_RUNNER_DESCRIPTION
paths:
- .cache/bazelisk
- .cache/bazel/*/install
policy: pull$BAZEL_CACHE_POLICY_SUFFIX
when: on_success
- key:
files:
- .go-version
- .python-version
prefix: bazel-$CI_JOB_NAME
paths:
- .cache/bazel/*/cache
- .cache/bazel/disk-cache
- .cache/go
- .cache/ms-go
- .cache/pip
policy: pull$BAZEL_CACHE_POLICY_SUFFIX
when: on_success
id_tokens:
BUILDBARN_ID_TOKEN:
aud: buildbarn.us1.ddbuild.io
image: registry.ddbuild.io/ci/datadog-agent-buildimages/linux$CI_IMAGE_LINUX_SUFFIX:$CI_IMAGE_LINUX
needs:
- build_system-probe-arm64
- go_deps
- generate_minimized_btfs_arm64
rules:
- when: on_success
script:
- mkdir -p $GOPATH/pkg/mod/cache && zstd -dc modcache.tar.zst | tar xf - -C $GOPATH/pkg/mod/cache
- rm -f modcache.tar.zst
- pushd omnibus && bundle config set --local path 'vendor/bundle' && popd
- rm -rf $OMNIBUS_PACKAGE_DIR/*
- tar -xf $CI_PROJECT_DIR/sysprobe-build-outputs.tar.xz
- mkdir -p /tmp/system-probe
- ${S3_CP_CMD} "${S3_PERMANENT_ARTIFACTS_URI}/clang-${CLANG_LLVM_VER}.${PACKAGE_ARCH}.${CLANG_BUILD_VERSION}"
/tmp/system-probe/clang-bpf
- ${S3_CP_CMD} "${S3_PERMANENT_ARTIFACTS_URI}/llc-${CLANG_LLVM_VER}.${PACKAGE_ARCH}.${CLANG_BUILD_VERSION}"
/tmp/system-probe/llc-bpf
- cp $CI_PROJECT_DIR/minimized-btfs.tar.xz /tmp/system-probe/minimized-btfs.tar.xz
- chmod 0744 /tmp/system-probe/clang-bpf /tmp/system-probe/llc-bpf
- dda inv -- -e omnibus.build --base-dir $OMNIBUS_BASE_DIR --skip-deps --go-mod-cache="$GOPATH/pkg/mod"
- --system-probe-bin=/tmp/system-probe --with-dd-procmgrd --flavor "$FLAVOR" --config-directory
? -------------------
+ --system-probe-bin=/tmp/system-probe --flavor "$FLAVOR" --config-directory "$CONFIG_DIR"
? ++++++++++++++
- "$CONFIG_DIR" --install-directory "$INSTALL_DIR"
? --------------
+ --install-directory "$INSTALL_DIR"
- ls -la $OMNIBUS_PACKAGE_DIR
- $S3_CP_CMD $OMNIBUS_PACKAGE_DIR/version-manifest.json $S3_SBOM_STORAGE_URI/$CI_JOB_NAME/version-manifest.json
stage: package_build
tags:
- arch:arm64
- specific:true
variables:
BAZELISK_HOME: $XDG_CACHE_HOME/bazelisk
DD_CC: aarch64-unknown-linux-gnu-gcc
DD_CXX: aarch64-unknown-linux-gnu-g++
FLAVOR: fips
KUBERNETES_CPU_REQUEST: 16
KUBERNETES_MEMORY_LIMIT: 32Gi
KUBERNETES_MEMORY_REQUEST: 32Gi
PACKAGE_ARCH: arm64
XDG_CACHE_HOME: $CI_PROJECT_DIR/.cachedatadog-agent-7-x64 datadog-agent-7-x64:
artifacts:
expire_in: 2 weeks
paths:
- $OMNIBUS_PACKAGE_DIR
before_script:
- mkdir -p .cargo
- 'cat > .cargo/config.toml << ''EOF''
[registries]
datadog = { index = "sparse+https://depot-read-api-rust.us1.ddbuild.io/datadog/rust/"
}
[registry]
default = "datadog"
[source]
[source.crates-io]
replace-with = "datadog"
EOF
'
cache:
- key:
files:
- omnibus/Gemfile
- release.json
prefix: omnibus-deps-$CI_JOB_IMAGE-$CI_JOB_NAME-$OMNIBUS_RUBY_VERSION
paths:
- omnibus/vendor/bundle
- key:
files:
- .bazelversion
prefix: bazelversion-$CI_RUNNER_DESCRIPTION
paths:
- .cache/bazelisk
- .cache/bazel/*/install
policy: pull$BAZEL_CACHE_POLICY_SUFFIX
when: on_success
- key:
files:
- .go-version
- .python-version
prefix: bazel-$CI_JOB_NAME
paths:
- .cache/bazel/*/cache
- .cache/bazel/disk-cache
- .cache/go
- .cache/ms-go
- .cache/pip
policy: pull$BAZEL_CACHE_POLICY_SUFFIX
when: on_success
id_tokens:
BUILDBARN_ID_TOKEN:
aud: buildbarn.us1.ddbuild.io
image: registry.ddbuild.io/ci/datadog-agent-buildimages/linux$CI_IMAGE_LINUX_SUFFIX:$CI_IMAGE_LINUX
needs:
- build_system-probe-x64
- go_deps
- generate_minimized_btfs_x64
rules:
- when: on_success
script:
- mkdir -p $GOPATH/pkg/mod/cache && zstd -dc modcache.tar.zst | tar xf - -C $GOPATH/pkg/mod/cache
- rm -f modcache.tar.zst
- pushd omnibus && bundle config set --local path 'vendor/bundle' && popd
- rm -rf $OMNIBUS_PACKAGE_DIR/*
- tar -xf $CI_PROJECT_DIR/sysprobe-build-outputs.tar.xz
- mkdir -p /tmp/system-probe
- ${S3_CP_CMD} "${S3_PERMANENT_ARTIFACTS_URI}/clang-${CLANG_LLVM_VER}.${PACKAGE_ARCH}.${CLANG_BUILD_VERSION}"
/tmp/system-probe/clang-bpf
- ${S3_CP_CMD} "${S3_PERMANENT_ARTIFACTS_URI}/llc-${CLANG_LLVM_VER}.${PACKAGE_ARCH}.${CLANG_BUILD_VERSION}"
/tmp/system-probe/llc-bpf
- cp $CI_PROJECT_DIR/minimized-btfs.tar.xz /tmp/system-probe/minimized-btfs.tar.xz
- chmod 0744 /tmp/system-probe/clang-bpf /tmp/system-probe/llc-bpf
- dda inv -- -e omnibus.build --base-dir $OMNIBUS_BASE_DIR --skip-deps --go-mod-cache="$GOPATH/pkg/mod"
- --system-probe-bin=/tmp/system-probe --with-dd-procmgrd --flavor "$FLAVOR" --config-directory
? -------------------
+ --system-probe-bin=/tmp/system-probe --flavor "$FLAVOR" --config-directory "$CONFIG_DIR"
? ++++++++++++++
- "$CONFIG_DIR" --install-directory "$INSTALL_DIR"
? --------------
+ --install-directory "$INSTALL_DIR"
- ls -la $OMNIBUS_PACKAGE_DIR
- $S3_CP_CMD $OMNIBUS_PACKAGE_DIR/version-manifest.json $S3_SBOM_STORAGE_URI/$CI_JOB_NAME/version-manifest.json
stage: package_build
tags:
- arch:amd64
- specific:true
variables:
BAZELISK_HOME: $XDG_CACHE_HOME/bazelisk
DD_CC: x86_64-unknown-linux-gnu-gcc
DD_CXX: x86_64-unknown-linux-gnu-g++
FLAVOR: base
KUBERNETES_CPU_REQUEST: 16
KUBERNETES_MEMORY_LIMIT: 32Gi
KUBERNETES_MEMORY_REQUEST: 32Gi
PACKAGE_ARCH: amd64
XDG_CACHE_HOME: $CI_PROJECT_DIR/.cachedatadog-agent-7-x64-fips datadog-agent-7-x64-fips:
artifacts:
expire_in: 2 weeks
paths:
- $OMNIBUS_PACKAGE_DIR
before_script:
- mkdir -p .cargo
- 'cat > .cargo/config.toml << ''EOF''
[registries]
datadog = { index = "sparse+https://depot-read-api-rust.us1.ddbuild.io/datadog/rust/"
}
[registry]
default = "datadog"
[source]
[source.crates-io]
replace-with = "datadog"
EOF
'
cache:
- key:
files:
- omnibus/Gemfile
- release.json
prefix: omnibus-deps-$CI_JOB_IMAGE-$CI_JOB_NAME-$OMNIBUS_RUBY_VERSION
paths:
- omnibus/vendor/bundle
- key:
files:
- .bazelversion
prefix: bazelversion-$CI_RUNNER_DESCRIPTION
paths:
- .cache/bazelisk
- .cache/bazel/*/install
policy: pull$BAZEL_CACHE_POLICY_SUFFIX
when: on_success
- key:
files:
- .go-version
- .python-version
prefix: bazel-$CI_JOB_NAME
paths:
- .cache/bazel/*/cache
- .cache/bazel/disk-cache
- .cache/go
- .cache/ms-go
- .cache/pip
policy: pull$BAZEL_CACHE_POLICY_SUFFIX
when: on_success
id_tokens:
BUILDBARN_ID_TOKEN:
aud: buildbarn.us1.ddbuild.io
image: registry.ddbuild.io/ci/datadog-agent-buildimages/linux$CI_IMAGE_LINUX_SUFFIX:$CI_IMAGE_LINUX
needs:
- build_system-probe-x64
- go_deps
- generate_minimized_btfs_x64
rules:
- when: on_success
script:
- mkdir -p $GOPATH/pkg/mod/cache && zstd -dc modcache.tar.zst | tar xf - -C $GOPATH/pkg/mod/cache
- rm -f modcache.tar.zst
- pushd omnibus && bundle config set --local path 'vendor/bundle' && popd
- rm -rf $OMNIBUS_PACKAGE_DIR/*
- tar -xf $CI_PROJECT_DIR/sysprobe-build-outputs.tar.xz
- mkdir -p /tmp/system-probe
- ${S3_CP_CMD} "${S3_PERMANENT_ARTIFACTS_URI}/clang-${CLANG_LLVM_VER}.${PACKAGE_ARCH}.${CLANG_BUILD_VERSION}"
/tmp/system-probe/clang-bpf
- ${S3_CP_CMD} "${S3_PERMANENT_ARTIFACTS_URI}/llc-${CLANG_LLVM_VER}.${PACKAGE_ARCH}.${CLANG_BUILD_VERSION}"
/tmp/system-probe/llc-bpf
- cp $CI_PROJECT_DIR/minimized-btfs.tar.xz /tmp/system-probe/minimized-btfs.tar.xz
- chmod 0744 /tmp/system-probe/clang-bpf /tmp/system-probe/llc-bpf
- dda inv -- -e omnibus.build --base-dir $OMNIBUS_BASE_DIR --skip-deps --go-mod-cache="$GOPATH/pkg/mod"
- --system-probe-bin=/tmp/system-probe --with-dd-procmgrd --flavor "$FLAVOR" --config-directory
? -------------------
+ --system-probe-bin=/tmp/system-probe --flavor "$FLAVOR" --config-directory "$CONFIG_DIR"
? ++++++++++++++
- "$CONFIG_DIR" --install-directory "$INSTALL_DIR"
? --------------
+ --install-directory "$INSTALL_DIR"
- ls -la $OMNIBUS_PACKAGE_DIR
- $S3_CP_CMD $OMNIBUS_PACKAGE_DIR/version-manifest.json $S3_SBOM_STORAGE_URI/$CI_JOB_NAME/version-manifest.json
stage: package_build
tags:
- arch:amd64
- specific:true
variables:
BAZELISK_HOME: $XDG_CACHE_HOME/bazelisk
DD_CC: x86_64-unknown-linux-gnu-gcc
DD_CXX: x86_64-unknown-linux-gnu-g++
FLAVOR: fips
KUBERNETES_CPU_REQUEST: 16
KUBERNETES_MEMORY_LIMIT: 32Gi
KUBERNETES_MEMORY_REQUEST: 32Gi
PACKAGE_ARCH: amd64
XDG_CACHE_HOME: $CI_PROJECT_DIR/.cacheChanges Summary
ℹ️ Diff available in the job log. |
Contributor
Files inventory check summaryFile checks results against ancestor 8e2a37e5: Results for datadog-agent_7.80.0~devel.git.47.c4f50db.pipeline.108550077-1_amd64.deb:Detected file changes:
|
Regression DetectorRegression Detector ResultsMetrics dashboard Baseline: 8e2a37e Optimization Goals: ✅ No significant changes detected
|
| perf | experiment | goal | Δ mean % | Δ mean % CI | trials | links |
|---|---|---|---|---|---|---|
| ➖ | docker_containers_cpu | % cpu utilization | -1.69 | [-4.72, +1.33] | 1 | Logs |
Fine details of change detection per experiment
| perf | experiment | goal | Δ mean % | Δ mean % CI | trials | links |
|---|---|---|---|---|---|---|
| ➖ | file_tree | memory utilization | +0.52 | [+0.47, +0.57] | 1 | Logs |
| ➖ | docker_containers_memory | memory utilization | +0.25 | [+0.17, +0.34] | 1 | Logs |
| ➖ | quality_gate_idle | memory utilization | +0.15 | [+0.10, +0.20] | 1 | Logs bounds checks dashboard |
| ➖ | ddot_metrics_sum_cumulativetodelta_exporter | memory utilization | +0.14 | [-0.09, +0.36] | 1 | Logs |
| ➖ | file_to_blackhole_100ms_latency | egress throughput | +0.02 | [-0.08, +0.13] | 1 | Logs |
| ➖ | file_to_blackhole_500ms_latency | egress throughput | +0.02 | [-0.38, +0.42] | 1 | Logs |
| ➖ | uds_dogstatsd_to_api | ingress throughput | +0.01 | [-0.19, +0.22] | 1 | Logs |
| ➖ | uds_dogstatsd_to_api_v3 | ingress throughput | +0.00 | [-0.21, +0.21] | 1 | Logs |
| ➖ | tcp_dd_logs_filter_exclude | ingress throughput | +0.00 | [-0.11, +0.11] | 1 | Logs |
| ➖ | ddot_metrics_sum_delta | memory utilization | -0.02 | [-0.20, +0.15] | 1 | Logs |
| ➖ | file_to_blackhole_1000ms_latency | egress throughput | -0.03 | [-0.46, +0.40] | 1 | Logs |
| ➖ | file_to_blackhole_0ms_latency | egress throughput | -0.04 | [-0.56, +0.48] | 1 | Logs |
| ➖ | uds_dogstatsd_20mb_12k_contexts_20_senders | memory utilization | -0.05 | [-0.12, +0.01] | 1 | Logs |
| ➖ | ddot_metrics | memory utilization | -0.08 | [-0.26, +0.10] | 1 | Logs |
| ➖ | ddot_metrics_sum_cumulative | memory utilization | -0.18 | [-0.33, -0.03] | 1 | Logs |
| ➖ | quality_gate_idle_all_features | memory utilization | -0.24 | [-0.27, -0.20] | 1 | Logs bounds checks dashboard |
| ➖ | tcp_syslog_to_blackhole | ingress throughput | -0.32 | [-0.49, -0.16] | 1 | Logs |
| ➖ | otlp_ingest_metrics | memory utilization | -0.36 | [-0.52, -0.20] | 1 | Logs |
| ➖ | ddot_logs | memory utilization | -0.42 | [-0.48, -0.35] | 1 | Logs |
| ➖ | quality_gate_logs | % cpu utilization | -0.50 | [-2.14, +1.14] | 1 | Logs bounds checks dashboard |
| ➖ | quality_gate_metrics_logs | memory utilization | -0.76 | [-0.99, -0.52] | 1 | Logs bounds checks dashboard |
| ➖ | otlp_ingest_logs | memory utilization | -1.35 | [-1.45, -1.25] | 1 | Logs |
| ➖ | docker_containers_cpu | % cpu utilization | -1.69 | [-4.72, +1.33] | 1 | Logs |
Bounds Checks: ✅ Passed
| perf | experiment | bounds_check_name | replicates_passed | observed_value | links |
|---|---|---|---|---|---|
| ✅ | docker_containers_cpu | simple_check_run | 10/10 | 706 ≥ 26 | |
| ✅ | docker_containers_memory | memory_usage | 10/10 | 279.51MiB ≤ 370MiB | |
| ✅ | docker_containers_memory | simple_check_run | 10/10 | 717 ≥ 26 | |
| ✅ | file_to_blackhole_0ms_latency | memory_usage | 10/10 | 0.19GiB ≤ 1.20GiB | |
| ✅ | file_to_blackhole_0ms_latency | missed_bytes | 10/10 | 0B = 0B | |
| ✅ | file_to_blackhole_1000ms_latency | memory_usage | 10/10 | 0.24GiB ≤ 1.20GiB | |
| ✅ | file_to_blackhole_1000ms_latency | missed_bytes | 10/10 | 0B = 0B | |
| ✅ | file_to_blackhole_100ms_latency | memory_usage | 10/10 | 0.20GiB ≤ 1.20GiB | |
| ✅ | file_to_blackhole_100ms_latency | missed_bytes | 10/10 | 0B = 0B | |
| ✅ | file_to_blackhole_500ms_latency | memory_usage | 10/10 | 0.22GiB ≤ 1.20GiB | |
| ✅ | file_to_blackhole_500ms_latency | missed_bytes | 10/10 | 0B = 0B | |
| ✅ | quality_gate_idle | intake_connections | 10/10 | 4 = 4 | bounds checks dashboard |
| ✅ | quality_gate_idle | memory_usage | 10/10 | 174.88MiB ≤ 181MiB | bounds checks dashboard |
| ✅ | quality_gate_idle_all_features | intake_connections | 10/10 | 4 = 4 | bounds checks dashboard |
| ✅ | quality_gate_idle_all_features | memory_usage | 10/10 | 497.27MiB ≤ 550MiB | bounds checks dashboard |
| ✅ | quality_gate_logs | intake_connections | 10/10 | 4 ≤ 6 | bounds checks dashboard |
| ✅ | quality_gate_logs | memory_usage | 10/10 | 209.67MiB ≤ 220MiB | bounds checks dashboard |
| ✅ | quality_gate_logs | missed_bytes | 10/10 | 0B = 0B | bounds checks dashboard |
| ✅ | quality_gate_metrics_logs | cpu_usage | 10/10 | 347.60 ≤ 2000 | bounds checks dashboard |
| ✅ | quality_gate_metrics_logs | intake_connections | 10/10 | 4 ≤ 6 | bounds checks dashboard |
| ✅ | quality_gate_metrics_logs | memory_usage | 10/10 | 407.44MiB ≤ 475MiB | bounds checks dashboard |
| ✅ | quality_gate_metrics_logs | missed_bytes | 10/10 | 0B = 0B | bounds checks dashboard |
Explanation
Confidence level: 90.00%
Effect size tolerance: |Δ mean %| ≥ 5.00%
Performance changes are noted in the perf column of each table:
- ✅ = significantly better comparison variant performance
- ❌ = significantly worse comparison variant performance
- ➖ = no significant change in performance
A regression test is an A/B test of target performance in a repeatable rig, where "performance" is measured as "comparison variant minus baseline variant" for an optimization goal (e.g., ingress throughput). Due to intrinsic variability in measuring that goal, we can only estimate its mean value for each experiment; we report uncertainty in that value as a 90.00% confidence interval denoted "Δ mean % CI".
For each experiment, we decide whether a change in performance is a "regression" -- a change worth investigating further -- if all of the following criteria are true:
-
Its estimated |Δ mean %| ≥ 5.00%, indicating the change is big enough to merit a closer look.
-
Its 90.00% confidence interval "Δ mean % CI" does not contain zero, indicating that if our statistical model is accurate, there is at least a 90.00% chance there is a difference in performance between baseline and comparison variants.
-
Its configuration does not mark it "erratic".
CI Pass/Fail Decision
✅ Passed. All Quality Gates passed.
- quality_gate_logs, bounds check missed_bytes: 10/10 replicas passed. Gate passed.
- quality_gate_logs, bounds check intake_connections: 10/10 replicas passed. Gate passed.
- quality_gate_logs, bounds check memory_usage: 10/10 replicas passed. Gate passed.
- quality_gate_metrics_logs, bounds check intake_connections: 10/10 replicas passed. Gate passed.
- quality_gate_metrics_logs, bounds check missed_bytes: 10/10 replicas passed. Gate passed.
- quality_gate_metrics_logs, bounds check cpu_usage: 10/10 replicas passed. Gate passed.
- quality_gate_metrics_logs, bounds check memory_usage: 10/10 replicas passed. Gate passed.
- quality_gate_idle_all_features, bounds check intake_connections: 10/10 replicas passed. Gate passed.
- quality_gate_idle_all_features, bounds check memory_usage: 10/10 replicas passed. Gate passed.
- quality_gate_idle, bounds check intake_connections: 10/10 replicas passed. Gate passed.
- quality_gate_idle, bounds check memory_usage: 10/10 replicas passed. Gate passed.
The embedded YAML uses extension-layout paths (.../ext/ddot/...) which are correct for the extension flow but not for the standalone DDOT package. Apply the same path rewrites as the systemd unit backwards compatibility function: remove /ext/ddot from all paths and, for OCI, replace the agent package path with the DDOT package path.
dd-procmgrd does not watch the config directory for changes; it requires an explicit restart or reload to detect new process definitions. Restart the datadog-agent-procmgrd.service after writing the DDOT YAML so procmgrd picks it up immediately. The restart is best-effort (logged as warning on failure) since procmgrd may not be running yet if the agent was just installed.
Contributor
Author
|
@codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: c4f50db1b7
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Comment on lines
+333
to
+335
| return filepath.Join(paths.PackagesPath, "datadog-agent", "stable", "processes.d") | ||
| } | ||
| return filepath.Join(agentInstallDirDebRpm, "processes.d") |
There was a problem hiding this comment.
writeDDOTProcessConfig uses this helper for both install and removal, but these branches target /opt/datadog-packages/.../processes.d or /opt/datadog-agent/processes.d. The generated procmgrd service sets DD_PM_CONFIG_DIR=/etc/datadog-agent/processes.d and the DDOT systemd unit's skip condition also checks /etc/datadog-agent/processes.d/datadog-agent-ddot.yaml (pkg/fleet/installer/packages/embedded/tmpl/gen/debrpm/datadog-agent-procmgrd.service:13, pkg/fleet/installer/packages/embedded/tmpl/gen/debrpm/datadog-agent-ddot.service:7). On hosts with dd-procmgrd installed, this either fails post-install with ENOENT because the Agent only creates the /etc directory, or writes a file that procmgrd never reads, leaving DDOT under systemd instead of procmgrd.
Useful? React with 👍 / 👎.
| // preRemoveDatadogAgentDDOT performs pre-removal steps for the DDOT package | ||
| // All the steps are allowed to fail | ||
| func preRemoveDatadogAgentDDOT(ctx HookContext) error { | ||
| removeDDOTProcessConfig(ctx.PackageType) |
There was a problem hiding this comment.
When DDOT has been managed by procmgrd, deleting the config file here is not enough to stop it: the daemon loads configs on startup/reload, and handle_reload_config is the path that stops processes whose YAML was removed (pkg/procmgr/rust/src/manager.rs:273). The subsequent agentDDOTService.Stop* calls only address the systemd DDOT units, which were skipped while the YAML existed, so uninstalling/removing the package can leave the old otel-agent process supervised by procmgrd until the Agent service is restarted. Reload or restart datadog-agent-procmgrd.service after removing the config.
Useful? React with 👍 / 👎.
With dd-procmgrd now managing the DDOT process, the systemd unit datadog-agent-ddot.service is intentionally skipped via ConditionPathExists=!. Update E2E tests to verify the new expected state: procmgr YAML present, procmgrd active, DDOT systemd unit inactive. Also restart procmgrd in the pre-remove hook so it stops supervising the DDOT process when the config is deleted.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Embeds all four DDOT process manager YAML variants (oci/debrpm × stable/experiment) in the installer binary and writes the appropriate one to the agent's
processes.d/directory during DDOT post-install. This causesdd-procmgrdto manage DDOT instead of systemd, since the systemd unit'sConditionPathExists=!skips itself when the YAML is present.On package removal, the YAML is deleted so systemd can resume managing DDOT if needed.
If
dd-procmgrdis not installed (binary not found), the write is skipped and DDOT falls back to systemd management.Motivation
Make DDOT managed by
dd-procmgrdby default on Linux bare metal hosts instead of systemd. This is the installer-side change that places the configuration file triggering the switch.Describe how you validated your changes
dd-procmgrdis present.Additional Notes