Improve link preview extraction in the Facebook marketing connector

[Diego Ruiz (diegoruizinga98-sys)]

Windsord.ai has HD links for each ad name. Although Airbyte also has that feature. Currently, ad_creative downloads the links in 64x644 but it should be at 1080x1080. I tried using this YAML to create that improvement, but without good results.

version: 7.0.4
type: DeclarativeSource

check:
type: CheckStream
stream_names:
- ads_parent

streams:

=========================================================

1. PARENT STREAM: Encuentra el Hash y empaca la "Mochila"

=========================================================

• type: DeclarativeStream
  name: ads_parent
  primary_key:
  • • id
      schema_loader:
      type: InlineSchemaLoader
      schema:
      type: object
      $schema: http://json-schema.org/schema#
      properties:
      id: {type: string}
      name: {type: string}
      safe_hash: {type: ["string", "null"]}
      parent_backpack: {type: ["string", "null"]}
      transformations:
  • type: AddFields
    fields:
    • type: AddedFieldDefinition
      path: ["safe_hash"]
      value: >-
      {%- set c = record.get('creative', {}) -%}
      {%- set afs = c.get('asset_feed_spec', {}) -%}
      {%- set imgs = afs.get('images', []) -%}
      {%- set vids = afs.get('videos', []) -%}
      {%- if c.get('image_hash') -%}
      {{- c.get('image_hash') -}}
      {%- elif imgs|length > 0 and imgs[0].get('hash') -%}
      {{- imgs[0].get('hash') -}}
      {%- elif vids|length > 0 and vids[0].get('thumbnail_hash') -%}
      {{- vids[0].get('thumbnail_hash') -}}
      {%- else -%}
      {{- '' -}}
      {%- endif -%}
    • type: AddedFieldDefinition
      path: ["parent_backpack"]
      value: "{{ record.id }}{{ record.name }}{{ record.safe_hash }}"
      retriever:
      type: SimpleRetriever
      partition_router:
      type: ListPartitionRouter
      values: "{{ config.account_ids }}"
      cursor_field: current_account_id
      paginator:
      type: DefaultPaginator
      pagination_strategy:
      type: CursorPagination
      cursor_value: "{{ response.get('paging', {}).get('cursors', {}).get('after') }}"
      stop_condition: >-
      {{ 'paging' not in response or 'next' not in response.get('paging', {}) }}
      page_token_option:
      type: RequestOption
      field_name: after
      inject_into: request_parameter
      requester:
      type: HttpRequester
      http_method: GET
      url_base: https://graph.facebook.com/v25.0/
      path: act_{{ stream_slice.current_account_id }}/ads
      authenticator:
      type: BearerAuthenticator
      api_token: "{{ config.access_token }}"
      request_parameters:
      fields: "id,name,creative{id,image_hash,asset_feed_spec}"
      limit: "100"

    ✅ AÑADIDO: El Padre también tiene el escudo contra el Rate Limiting de Facebook

    error_handler:
    type: DefaultErrorHandler
    max_retries: 15
    response_filters:
    - type: HttpResponseFilter
    action: RETRY
    http_codes: [400, 403, 429]
    error_message_contains: "too many calls"
    - type: HttpResponseFilter
    action: RETRY
    http_codes: [400, 403, 429]
    error_message_contains: "Application request limit reached"
    backoff_strategies:
    - type: ConstantBackoffStrategy
    backoff_time_in_seconds: 300
    record_selector:
    type: RecordSelector
    extractor:
    type: DpathExtractor
    field_path: [data]
    transform_before_filtering: true
    record_filter:
    type: RecordFilter
    condition: "{{ record.get('safe_hash', '') != '' }}"

=========================================================

2. CHILD STREAM: Desempaca y consulta la imagen HD

=========================================================

• type: DeclarativeStream
  name: ad_images_child
  primary_key:
  • • hash
      schema_loader:
      type: InlineSchemaLoader
      schema:
      type: object
      $schema: http://json-schema.org/schema#
      properties:
      url: {type: string}
      hash: {type: string}
      ad_id: {type: string}
      ad_name: {type: string}
      width: {type: integer}
      height: {type: integer}
      transformations:
  • type: AddFields
    fields:
    • type: AddedFieldDefinition
      path: ["ad_id"]
      value: "{{ stream_slice.backpack.split('~~')[0] }}"
    • type: AddedFieldDefinition
      path: ["ad_name"]
      value: "{{ stream_slice.backpack.split('~~')[1] }}"
      retriever:
      type: SimpleRetriever
      partition_router:
      type: SubstreamPartitionRouter
      parent_stream_configs:
    • type: ParentStreamConfig
      stream: "#/streams/0"
      parent_key: parent_backpack
      partition_field: backpack
      requester:
      type: HttpRequester
      http_method: GET
      url_base: https://graph.facebook.com/v25.0/
      path: 'act_{{ stream_slice.parent_slice.current_account_id }}/adimages?hashes=["{{ stream_slice.backpack.split("~~")[2] }}"]'
      authenticator:
      type: BearerAuthenticator
      api_token: "{{ config.access_token }}"
      request_parameters:
      fields: "hash,url,width,height"

    ✅ El hijo mantiene su escudo con 15 reintentos maximos

    error_handler:
    type: DefaultErrorHandler
    max_retries: 15
    response_filters:
    - type: HttpResponseFilter
    action: RETRY
    http_codes: [400, 403, 429]
    error_message_contains: "too many calls"
    - type: HttpResponseFilter
    action: RETRY
    http_codes: [400, 403, 429]
    error_message_contains: "Application request limit reached"
    backoff_strategies:
    - type: ConstantBackoffStrategy
    backoff_time_in_seconds: 300
    record_selector:
    type: RecordSelector
    extractor:
    type: DpathExtractor
    field_path: [data]

spec:
type: Spec
connection_specification:
type: object
$schema: http://json-schema.org/draft-07/schema#
required: [account_ids, access_token]
properties:
account_ids:
type: array
items: {type: string}
title: Facebook Ad Account IDs
access_token:
type: string
title: Access Token
airbyte_secret: true

Algo así debería descargar y si ya lo lograron indíquenme como.