Skip to content
Draft
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
59 changes: 58 additions & 1 deletion docs/integrations/destinations/bigquery.md
Original file line number Diff line number Diff line change
Expand Up @@ -178,7 +178,9 @@ namespace with `n` for converted namespaces.
| OBJECT | JSON |
| ARRAY | JSON |

## Troubleshooting permission issues
## Troubleshooting

### Permission errors

The service account does not have the proper permissions.

Expand All @@ -193,6 +195,61 @@ The HMAC key is wrong.
- Make sure the HMAC key is created for the BigQuery service account, and the service account has
permission to access the GCS bucket and path.

### "Socket file not created" timeout

_Applies to connector version 3.0.7 and later (socket data-channel mode)._

If your sync fails with an error containing `Socket file ... not created after 900000 ms`:

- This indicates the Airbyte platform sidecar did not create the inter-process communication
channel in time. The connector itself is not at fault.
- **Self-hosted deployments:** Verify the platform version supports socket-mode destinations.
Check worker pod logs for sidecar startup failures or OOM events.
- **Running multiple connections to the same destination simultaneously** can cause resource
contention. Try staggering connection schedules.
- If the error persists, contact [Airbyte Support](https://support.airbyte.com) with the full
destination logs.
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I really don't like this answer. Reasons:

  • It's very technical, which in itself isn't bad, but it assumes the reader knows a lot about an Airbyte deployment
  • It only talks about self-hosted.
  • Many self-hosted people are community users, who can't call support.
  • No indication of whether Cloud users would be affected by this, or how to resolve it (probably phone support in that case).

Please provide a more streamlined explanation that addresses the needs of each audience and does not over-assume their background knowledge.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rewrote this section in 3fc4e97 to address your feedback:

  • Removed the technical jargon ("platform sidecar", "socket data-channel mode", version callout)
  • Leads with "retry" as the first action
  • Covers both Cloud and self-hosted audiences with separate bullets
  • Doesn't assume the reader knows about pod logs or OOM — just mentions checking logs for container restarts

Devin session


### HTTP 400 "Request had invalid euc header" during upload

If your sync fails with `BigQueryException: 400 Bad Request` and the message
`Request had invalid euc header`:

- This is a transient error from the Google BigQuery resumable-upload API. The upload session
was invalidated mid-stream.
- **Retry the sync.** In most cases the error resolves on the next attempt.
- If the error recurs on every sync:
- Check whether a proxy, VPN, or firewall is modifying HTTP headers on requests to
`bigquery.googleapis.com`.
- Verify the service account key has not been rotated or revoked since the connection was
configured.
- Try reducing the **Google BigQuery Client Chunk Size** from the default 15 MiB to a
smaller value (for example, 5 MiB). Smaller chunks reduce the window for session
corruption.

### Broken pipe errors

If your sync fails with a `Broken pipe` error on the destination side:

- This is a transport-level error caused by a network interruption between the Airbyte worker
and the BigQuery API.
- **Reduce chunk size:** In the connector settings, decrease the **Google BigQuery Client Chunk
Size** to 5 MiB or less. Smaller uploads complete faster and are less susceptible to
connection timeouts.
- **Check network infrastructure:** Firewalls, NAT gateways, and cloud load balancers may
terminate idle or long-lived connections. Ensure outbound HTTPS to `*.googleapis.com` is not
subject to aggressive idle timeouts.
- **Retry the sync.** Broken pipe errors are typically transient.

### Load job timeouts

If your sync fails with `Fail to complete a load job in big query`:

- BigQuery load jobs have a 30-minute wait timeout. Very large batches or high BigQuery queue
contention can exceed this limit.
- Try reducing the volume per sync by using incremental sync mode or reducing the number of
streams per connection.

## Tutorials

Now that you have set up the BigQuery destination connector, check out the following BigQuery
Expand Down
Loading