Agent version
7.77.1 (system-probe / security-agent as shipped in chart datadog Helm 3.196.0). Likely affects other 7.x builds using the same CWS mount probes without MatchFuncName.
Bug Report
On Talos Linux v1.12.5 (kernel 6.18.15), Cloud Workload Security (CWS) runtime does not start: the event_monitor module fails during probe activation because the kprobe target attach_recursive_mnt is not found. The kernel exports the function as attach_recursive_mnt.isra.0 (GCC IPA-SRA), while the agent resolves the hook to an exact symbol name attach_recursive_mnt via /sys/kernel/debug/tracing/available_filter_functions (no MatchFuncName regex on this probe).
Because event_monitor never loads, /var/run/sysprobe/runtime-security.sock is never created and security-agent loops on:
error while connecting to the runtime security module ... dial unix /var/run/sysprobe/runtime-security.sock: connect: no such file or directory
Expected: CWS runtime loads on Talos the same as on kernels that retain the unsuffixed attach_recursive_mnt symbol.
Actual: Probe validation fails; full log line from system-probe:
error registering HTTP endpoints for module event_monitor: failed to init probe:
probes activation validation failed: AllOf requirement failed, the following probes
are not running [{UID:security EBPFFuncName:hook_attach_recursive_mnt}:
symbol 'attach_recursive_mnt' not found: invalid argument]
Root cause (concise): hook_attach_recursive_mnt is registered in pkg/security/ebpf/probes/mount.go without MatchFuncName, while pkg/security/ebpf/probes/event_types.go places it in an AllOf group — one failed attach aborts the whole module. Same class of issue as existing network tracer handling for suffixed symbols, e.g. MatchFuncName: "^ctnetlink_fill_info(\\.constprop\\.0)?$" in pkg/network/tracer/ebpf_conntracker.go.
Suggested fix (for discussion): Add MatchFuncName for the mount kprobe so both attach_recursive_mnt and attach_recursive_mnt.isra.0 resolve, mirroring the conntracker literal-.0 style:
MatchFuncName: "^attach_recursive_mnt(\\.isra\\.0)?$",
Reproduction Steps
- Boot a Talos v1.12.5 cluster (or use a kernel where
attach_recursive_mnt appears only as attach_recursive_mnt.isra.N in kallsyms / available_filter_functions).
- Install the Datadog Agent with
datadog.securityAgent.runtime.enabled: true (and system-probe privileged as required).
- Inspect system-probe logs: confirm
symbol 'attach_recursive_mnt' not found and security-agent errors on missing runtime-security.sock.
Public minimal repro (CI + scripts): ChipWolf/datadog-cws-talos-repro — GitHub Action live-repro.yml boots Talos in QEMU, installs the agent, and asserts the failure; utility scripts under scripts/ extract Talos vmlinuz kallsyms and compare to expected CWS probe names.
Agent configuration
Relevant Helm values excerpt (repro):
datadog:
securityAgent:
runtime:
enabled: true
compliance:
enabled: false # optional; runtime is the failing piece
Also set explicit hostname / DD_HOSTNAME if the node has no resolvable hostname, so system-probe reaches probe init (optional; unrelated to symbol suffix but needed in minimal QEMU CI).
Mitigation (not a fix): datadog.securityAgent.runtime.enabled: false avoids loading event_monitor; CWS runtime features remain off until probe matching is fixed.
Operating System
Talos Linux v1.12.5, Linux 6.18.15 (amd64). Example symbol table lines from published Talos vmlinuz (kallsyms extraction):
ffffffff8192c7f0 t attach_recursive_mnt.isra.0
Other environment details
Agent version
7.77.1 (system-probe / security-agent as shipped in chart
datadogHelm 3.196.0). Likely affects other 7.x builds using the same CWS mount probes withoutMatchFuncName.Bug Report
On Talos Linux v1.12.5 (kernel 6.18.15), Cloud Workload Security (CWS) runtime does not start: the event_monitor module fails during probe activation because the kprobe target
attach_recursive_mntis not found. The kernel exports the function asattach_recursive_mnt.isra.0(GCC IPA-SRA), while the agent resolves the hook to an exact symbol nameattach_recursive_mntvia/sys/kernel/debug/tracing/available_filter_functions(noMatchFuncNameregex on this probe).Because event_monitor never loads,
/var/run/sysprobe/runtime-security.sockis never created and security-agent loops on:error while connecting to the runtime security module ... dial unix /var/run/sysprobe/runtime-security.sock: connect: no such file or directoryExpected: CWS runtime loads on Talos the same as on kernels that retain the unsuffixed
attach_recursive_mntsymbol.Actual: Probe validation fails; full log line from system-probe:
Root cause (concise):
hook_attach_recursive_mntis registered inpkg/security/ebpf/probes/mount.gowithoutMatchFuncName, whilepkg/security/ebpf/probes/event_types.goplaces it in anAllOfgroup — one failed attach aborts the whole module. Same class of issue as existing network tracer handling for suffixed symbols, e.g.MatchFuncName: "^ctnetlink_fill_info(\\.constprop\\.0)?$"inpkg/network/tracer/ebpf_conntracker.go.Suggested fix (for discussion): Add
MatchFuncNamefor the mount kprobe so bothattach_recursive_mntandattach_recursive_mnt.isra.0resolve, mirroring the conntracker literal-.0style:Reproduction Steps
attach_recursive_mntappears only asattach_recursive_mnt.isra.Nin kallsyms /available_filter_functions).datadog.securityAgent.runtime.enabled: true(and system-probe privileged as required).symbol 'attach_recursive_mnt' not foundand security-agent errors on missingruntime-security.sock.Public minimal repro (CI + scripts): ChipWolf/datadog-cws-talos-repro — GitHub Action
live-repro.ymlboots Talos in QEMU, installs the agent, and asserts the failure; utility scripts underscripts/extract Talosvmlinuzkallsyms and compare to expected CWS probe names.Agent configuration
Relevant Helm values excerpt (repro):
Also set explicit hostname /
DD_HOSTNAMEif the node has no resolvable hostname, so system-probe reaches probe init (optional; unrelated to symbol suffix but needed in minimal QEMU CI).Mitigation (not a fix):
datadog.securityAgent.runtime.enabled: falseavoids loading event_monitor; CWS runtime features remain off until probe matching is fixed.Operating System
Talos Linux v1.12.5, Linux 6.18.15 (amd64). Example symbol table lines from published Talos
vmlinuz(kallsyms extraction):Other environment details
pkg/security/ebpf/c/include/hooks/mount.h(HOOK_ENTRY("attach_recursive_mnt")); Go probes —pkg/security/ebpf/probes/mount.go; selector —pkg/security/ebpf/probes/event_types.go(mountAllOfblock)..isra.Nsuffix — see GCC IPA-SRA documentation.