-
Notifications
You must be signed in to change notification settings - Fork 70
Description
Summary
Enhance PyAirbyte MCP (Coral MCP) tools to support cross-workspace connector duplication and migration workflows (e.g., self-managed → Cloud, Cloud → Cloud). This was identified during an exploration of the end-to-end migration process using Coral MCP and Ops MCP tools.
Requested by: Aaron ("AJ") Steers (@aaronsteers)
Related Devin session: https://app.devin.ai/sessions/eb636b8d14fc4895b3b894ea399f998e
Note: Setting catalog/state artifacts is intentionally out of scope here — those write operations are handled by the Ops MCP
connection_medictools (behindAIRBYTE_OPS_MEDIC_MODE).
Context: Migration Workflow
A connector migration requires these steps:
- Read source connector config (type, name, credentials/config)
- Read destination connector config
- Read connection metadata (streams, sync modes, schedule, enabled status)
- Read configured catalog (already supported via
get_connection_artifact) - Read connection state (already supported via
get_connection_artifact) - Deploy source to target workspace (supported via
deploy_source_to_cloud) - Deploy destination to target workspace (supported via
deploy_destination_to_cloud) - Create connection in target workspace (supported via
create_connection_on_cloud) - Import catalog to target connection (handled by Ops MCP medic mode)
- Import state to target connection (handled by Ops MCP medic mode)
Steps 4-5 and 6-10 are already covered. Steps 1-3 have gaps documented below.
P0: Critical Gaps
1. Add source/destination config to describe responses (or new tool)
Problem: describe_cloud_source and describe_cloud_destination return name, ID, URL, and connector_definition_id — but not the connector configuration. Without the config, you cannot recreate the connector in a target workspace.
Affected code:
airbyte/mcp/cloud.py—CloudSourceDetailsmodel (line ~94-104) has no config fieldairbyte/mcp/cloud.py—CloudDestinationDetailsmodel (line ~107-117) has no config fieldairbyte/mcp/cloud.py—describe_cloud_source()function (line ~921-948) strips config from responseairbyte/mcp/cloud.py—describe_cloud_destination()function (line ~957-984) strips config from response
Proposed fix: Add a configuration field to CloudSourceDetails and CloudDestinationDetails, populated from the SourceResponse.configuration / DestinationResponse.configuration API data. The API already masks secrets, so sensitive values will appear redacted — the user would need to re-enter credentials for the target workspace.
Implementation Details
The underlying API calls (api_util.get_source() and api_util.get_destination()) already return SourceResponse / DestinationResponse objects that include configuration. The MCP tool just needs to pass it through. The _connector_info is already fetched and cached when source.name is accessed.
Key files:
airbyte/mcp/cloud.py:CloudSourceDetails,CloudDestinationDetails,describe_cloud_source(),describe_cloud_destination()airbyte/cloud/connectors.py:CloudSource._fetch_connector_info()(line ~194),CloudDestination._fetch_connector_info()(line ~279)
2. Add connector_name to describe source/dest responses
Problem: describe_cloud_source returns connector_definition_id (a UUID), but deploy_source_to_cloud requires source_connector_name (e.g., "source-postgres"). There is no tool that maps definition_id → connector name, making it impossible to programmatically recreate a connector.
Affected code:
airbyte/mcp/cloud.py—CloudSourceDetailsmodel hasconnector_definition_idbut noconnector_nameairbyte/mcp/cloud.py—CloudDestinationDetailsmodel same issue
Proposed fix: Add connector_name field (e.g., "source-postgres") to both response models. The SourceResponse object includes source_type which can be mapped to the connector name format.
P1: Important Gaps
3. Add config_api_root parameter to Coral MCP tools
Problem: The Ops MCP tools (get_connection_state, get_connection_catalog) accept a config_api_root parameter to target self-managed Airbyte instances. But all Coral MCP tools resolve the API root from env vars only — they have no equivalent override parameter. This prevents reading from or writing to self-managed instances without switching environment variables.
Affected code:
airbyte/mcp/cloud.py—_get_cloud_workspace()function (line ~238-270) hardcodes API root from env/config- All Coral MCP cloud tools that call
_get_cloud_workspace()
Proposed fix: Add an optional config_api_root parameter to _get_cloud_workspace() and propagate it to the relevant MCP tool signatures (at minimum: describe_cloud_source, describe_cloud_destination, describe_cloud_connection, get_connection_artifact, deploy_source_to_cloud, deploy_destination_to_cloud, create_connection_on_cloud).
Reference: Ops MCP approach
The Ops MCP already implements this pattern in connection_state.py:
config_api_root: Annotated[
str | None,
Field(
description="Optional API root URL override. "
"Defaults to Airbyte Cloud. "
"Use this to target local or self-hosted deployments.",
default=None,
),
] = None,And passes it through: api_root=config_api_root or constants.CLOUD_API_ROOT
4. Add schedule and enabled status to describe_cloud_connection
Problem: describe_cloud_connection returns source_id, dest_id, selected_streams, and table_prefix — but omits schedule type (cron/manual), cron expression, and enabled/disabled status. These are needed for a faithful migration.
Affected code:
airbyte/mcp/cloud.py—CloudConnectionDetailsmodel (line ~120-140) missing schedule and status fieldsairbyte/mcp/cloud.py—describe_cloud_connection()function (line ~993-1022)
Proposed fix: Add schedule_type, cron_expression, and status (active/inactive) fields to CloudConnectionDetails, populated from the ConnectionResponse data that is already fetched.
Checklist
- P0: Add config to
describe_cloud_source/describe_cloud_destinationresponses - P0: Add
connector_nameto source/dest describe responses - P1: Add
config_api_rootparameter to Coral MCP cloud tools - P1: Add schedule/status to
describe_cloud_connectionresponse