fix(source-gong): fix extensiveCalls, answeredScorecards, callTranscripts incremental sync not filtering server-side#74091
Conversation
…orecards, callTranscripts - extensiveCalls: change start_time_option from request_parameter to body_json with field_path [filter, fromDateTime], and remove hardcoded filter.fromDateTime from request_body_json - answeredScorecards: remove stale request_parameters.fromDateTime that is ignored on POST endpoints - callTranscripts: add incremental_sync with DatetimeBasedCursor and body_json injection, remove hardcoded filter.fromDateTime from request_body_json Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
|
Note 📝 PR Converted to Draft More info...Thank you for creating this PR. As a policy to protect our engineers' time, Airbyte requires all PRs to be created first in draft status. Your PR has been automatically converted to draft status in respect for this policy. As soon as your PR is ready for formal review, you can proceed to convert the PR to "ready for review" status by clicking the "Ready for review" button at the bottom of the PR page. To skip draft status in future PRs, please include |
👋 Welcome to Airbyte!Thank you for your contribution from humungasaurus/airbyte! We're excited to have you in the Airbyte community. If you have any questions, feel free to ask in the PR comments or join our Slack community. 💡 Show Tips and TricksPR Slash CommandsAs needed or by request, Airbyte Maintainers can execute the following slash commands on your PR:
Tips for Working with CI
📚 Show Repo GuidanceHelpful Resources
|
Update callTranscripts from Full Refresh to Incremental in the docs and add changelog entry for the incremental sync fix. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Summary
Fixes #74090
Three POST-based streams in
source-gongclaim incremental sync but fetch all records on every sync due to incorrectfromDateTimeinjection. This is the same class of bug fixed in PR #71344 foransweredScorecards, but the fixes forextensiveCallsandcallTranscriptswere missed, and a stalerequest_parametersentry was left behind onansweredScorecards.Changes
extensiveCalls: Changedstart_time_optionfrominject_into: request_parametertoinject_into: body_jsonwithfield_path: [filter, fromDateTime]. Removed hardcodedrequest_body_json.filter.fromDateTimeso the cursor value fromDatetimeBasedCursoris used instead ofconfig['start_date']on every sync.answeredScorecards: Removed stalerequest_parameters.fromDateTime: "{{ config['start_date'] }}"from the requester. This is a POST endpoint so Gong ignores query params — the incremental cursor already correctly injects intobody_json.callTranscripts: Removed hardcodedrequest_body_json.filter.fromDateTime. Addedincremental_syncwithDatetimeBasedCursorandstart_time_optionusinginject_into: body_jsonwithfield_path: [filter, fromDateTime].Root cause
Gong's POST endpoints ignore query parameters. The
start_time_optionwas configured withinject_into: request_parameter, which appendsfromDateTimeas a query param — silently ignored by Gong. Additionally,request_body_json.filter.fromDateTimewas hardcoded toconfig['start_date'], preventing the cursor from ever advancing past the initial start date.Test plan
extensiveCallsincremental sync only fetches records newer than the last cursor stateansweredScorecardsstill works correctly without the stalerequest_parameterscallTranscriptsincremental sync filters server-side viafilter.fromDateTimein the POST body🤖 Generated with Claude Code