Policy tests fail on 9.4.0+ due to new beatsauth OTel fields injected by Fleet

[mrodm]

Since elastic/kibana#259308 (present in 9.4.0-SNAPSHOT onwards), Fleet injects a new beatsauth authentication extension into every OTel policy that uses an Elasticsearch output. This
causes policy tests to fail when run against a 9.4.0+ stack because the downloaded policy contains fields that existing fixtures don't include.

The three new fields injected are:

# 1. auth reference added to the elasticsearch exporter
exporters:
  elasticsearch/componentid-0:
    auth:                                                                                                                                                                                                       
      authenticator: beatsauth/default
                                                                                                                                                                                                                
# 2. new top-level extension with deployment-specific SSL data
extensions:
  beatsauth/componentid-0:
    ssl:
      ca_trusted_fingerprint: 2611966E95E1F6B5A4C0A3094B8BFBB83B7AD1A72F195C35A54432C8B563A73A # <dynamic hash>
                                                                                                                                                                                                                
# 3. extension registered in service
service:                                                                                                                                                                                                        
  extensions:   
    - beatsauth/componentid-0

Packages tested against older stacks (e.g. 9.2.0) don't have these fields, so fixtures created on older stacks fail on newer ones and vice versa.

A concrete example of this breakage is visible in elastic/integrations#18321.

Proposed direction

These fields are deployment-specific (the ca_trusted_fingerprint changes per installation, and the auth wiring is controlled by Fleet, not the package) and should be stripped by cleanPolicy before comparison,
following the existing pattern in policyEntryFilters.

• exporters[*].auth — can be filtered with the existing mapValues mechanism (add {name: "auth"} to the exporters filter).
• extensions keys matching beatsauth/* — requires a new capability to remove map entries by key pattern.
• service.extensions list entries matching beatsauth/* — requires a new capability to remove list elements by value pattern (or by prefix).

The existing onlyIfEmpty/ignoreValues mechanism is close but not sufficient for pruning specific entries from a list while keeping others.

Related issues

• [Stack 9.4.0-SNAPSHOT] Too many packages failing in daily job integrations#18321
• [Fleet] Extend OTel exporter configuration kibana#259308

---

Generated with the assistance of Claude