vSphere logs: map client.ip to ECS source.ip and related.ip

[ishleenk17]

Summary

The vSphere integration ingest pipelines parse a client IP into client.ip for several event types, but do not populate ECS fields source.ip or related.ip. This breaks ECS alignment and reduces cross-source IP correlation (dashboards/detections that rely on source.ip/related.ip miss the vSphere client IP).

Affected package / data stream

• Package: vsphere
• Data stream: log

Current behavior (examples)

Ingest pipelines extract an IP into client.ip:

• Login/logout/failed login / connection-related messages:
  packages/vsphere/data_stream/log/elasticsearch/ingest_pipeline/login.yml
• Upload/file messages:
  packages/vsphere/data_stream/log/elasticsearch/ingest_pipeline/file.yml

However, neither pipeline copies client.ip to:

• source.ip
• related.ip

Expected behavior

When client.ip is present:

• source.ip should be set to client.ip
• related.ip should include client.ip (no duplicates)

[github-actions]

tl;dr: This is a valid issue, but it is already being addressed in draft PR #18536; I recommend using #18536 as the single implementation path, closing duplicate PR #18535, and fixing the incorrect changelog link before merge.

Recommendation

Use PR #18536 as the canonical fix for this issue and close PR #18535 as duplicate work. In #18536, update the vSphere changelog entry link from pull/16753 to the correct PR reference before merging.

Findings

• Current vSphere log pipelines parse client.ip but do not map ECS source.ip or related.ip:
  • packages/vsphere/data_stream/log/elasticsearch/ingest_pipeline/login.yml:11,16,41,54,60,66,71 (extracts client.ip), and no source.ip/related.ip mapping at pipeline end (login.yml:1-159).
  • packages/vsphere/data_stream/log/elasticsearch/ingest_pipeline/file.yml:6 (extracts client.ip), and no source.ip/related.ip mapping in the file (file.yml:1-16).
• Expected pipeline outputs currently assert client.ip without corresponding source/related blocks (example):
  • packages/vsphere/data_stream/log/_dev/test/pipeline/test-format-common.log-expected.json:73-75
• Two draft PRs already implement this exact fix scope:
  • PR #18535 (draft) and PR #18536 (draft) both modify:
    • .../ingest_pipeline/login.yml
    • .../ingest_pipeline/file.yml
    • .../test-format-common.log-expected.json
    • packages/vsphere/changelog.yml
    • packages/vsphere/manifest.yml
• Both draft PR changelog patches currently point to https://github.com/elastic/integrations/pull/16753:
  • from PR file patch for packages/vsphere/changelog.yml in #18535 and #18536.

Verification

I ran repository checks to confirm current behavior in main:

$ grep -nE "client\.ip|source\.ip|related\.ip" packages/vsphere/data_stream/log/elasticsearch/ingest_pipeline/login.yml packages/vsphere/data_stream/log/elasticsearch/ingest_pipeline/file.yml
packages/vsphere/data_stream/log/elasticsearch/ingest_pipeline/login.yml:11:    pattern: "%{}User %{user.name}@%{client.ip} %{_tmp.event} as %{user_agent.original}"
packages/vsphere/data_stream/log/elasticsearch/ingest_pipeline/login.yml:16:    pattern: "%{} Shared secret from %{client.ip} %{_tmp.event} as %{user_agent.original}"
packages/vsphere/data_stream/log/elasticsearch/ingest_pipeline/login.yml:41:    pattern: "%{}User %{user.name}@%{client.ip} %{_tmp.event} out (login time: %{event.start},
packages/vsphere/data_stream/log/elasticsearch/ingest_pipeline/login.yml:54:    pattern: "[%{}] [] [%{}] [%{event.outcome} login %{user.name} from %{client.ip} %{}"
packages/vsphere/data_stream/log/elasticsearch/ingest_pipeline/login.yml:60:    pattern: 'Received %{_tmp.status} from %{client.ip} port %{client.port}:%{destination.port}:
packages/vsphere/data_stream/log/elasticsearch/ingest_pipeline/login.yml:66:    pattern: "%{_tmp.status} from %{client.ip} port %{client.port} %{}"
packages/vsphere/data_stream/log/elasticsearch/ingest_pipeline/login.yml:71:    pattern: Connection %{_tmp.status} by %{client.ip} port %{client.port} %{}
packages/vsphere/data_stream/log/elasticsearch/ingest_pipeline/file.yml:6:      pattern: "%{} 'Upload' for path '%{vsphere.log.file.path}' %{} '%{client.ip}' %{} '%{event.outcome}'"

PR patch metadata confirms #18536 includes the expected processors and version bump:

• packages/vsphere/data_stream/log/elasticsearch/ingest_pipeline/login.yml adds set source.ip + append related.ip
• packages/vsphere/data_stream/log/elasticsearch/ingest_pipeline/file.yml adds same
• packages/vsphere/manifest.yml bumps 1.23.1 -> 1.24.0

Detailed Action Plan

1. Keep PR #18536 as the active implementation PR for issue #18537.
2. Close PR #18535 as duplicate to avoid parallel review and conflicting release metadata.
3. In #18536, correct packages/vsphere/changelog.yml link in the 1.24.0 entry (currently pull/16753) to the correct PR link.
4. Ensure final PR retains these mappings in both pipelines:
   • packages/vsphere/data_stream/log/elasticsearch/ingest_pipeline/login.yml (tail section, after parsing)
   • packages/vsphere/data_stream/log/elasticsearch/ingest_pipeline/file.yml (after existing file-event processors)
   • processors:
     • set source.ip from client.ip guarded by ctx.client?.ip != null
     • append related.ip with allow_duplicates: false guarded by same condition
5. Keep fixture updates in packages/vsphere/data_stream/log/_dev/test/pipeline/test-format-common.log-expected.json so events with client.ip also assert source.ip and related.ip.
6. Merge #18536, then close issue #18537 as fixed by that PR.

Related Items

Type	Link / File	Relevance
Issue	#18537	Tracking issue for ECS IP mapping in vSphere logs
PR	#18536	Draft PR implementing requested mapping; should be canonical
PR	#18535	Duplicate draft PR with same scope
File	packages/vsphere/data_stream/log/elasticsearch/ingest_pipeline/login.yml	Current missing mapping; target for source.ip/related.ip
File	packages/vsphere/data_stream/log/elasticsearch/ingest_pipeline/file.yml	Current missing mapping; target for source.ip/related.ip
File	packages/vsphere/data_stream/log/_dev/test/pipeline/test-format-common.log-expected.json	Fixture assertions for expected output
File	packages/vsphere/changelog.yml	Changelog entry includes incorrect PR link
File	packages/vsphere/manifest.yml	Version bump expected with enhancement

Note

🔒 Integrity filter blocked 11 items

The following items were blocked because they don't meet the GitHub integrity level.

• #14943 search_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
• #16335 search_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
• #18505 search_pull_requests: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
• #17573 search_pull_requests: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
• #15505 search_pull_requests: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
• #4320 search_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
• #7422 search_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
• #15235 search_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
• #15421 search_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
• #12241 search_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
• #11049 search_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".

To allow these resources, lower min-integrity in your GitHub frontmatter:

tools:
  github:
    min-integrity: approved  # merged | approved | unapproved | none

---

What is this? | From workflow: Issue Triage

Give us feedback! React with 🚀 if perfect, 👍 if helpful, 👎 if not.