Skip to content

Commit 737b46a

Browse files
docs(destination-bigquery): expand troubleshooting section for write failures
Co-Authored-By: ian.alton@airbyte.io <ian.alton@airbyte.io>
1 parent 0a34061 commit 737b46a

File tree

1 file changed

+58
-1
lines changed

1 file changed

+58
-1
lines changed

docs/integrations/destinations/bigquery.md

Lines changed: 58 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -178,7 +178,9 @@ namespace with `n` for converted namespaces.
178178
| OBJECT | JSON |
179179
| ARRAY | JSON |
180180

181-
## Troubleshooting permission issues
181+
## Troubleshooting
182+
183+
### Permission errors
182184

183185
The service account does not have the proper permissions.
184186

@@ -193,6 +195,61 @@ The HMAC key is wrong.
193195
- Make sure the HMAC key is created for the BigQuery service account, and the service account has
194196
permission to access the GCS bucket and path.
195197

198+
### "Socket file not created" timeout
199+
200+
_Applies to connector version 3.0.7 and later (socket data-channel mode)._
201+
202+
If your sync fails with an error containing `Socket file ... not created after 900000 ms`:
203+
204+
- This indicates the Airbyte platform sidecar did not create the inter-process communication
205+
channel in time. The connector itself is not at fault.
206+
- **Self-hosted deployments:** Verify the platform version supports socket-mode destinations.
207+
Check worker pod logs for sidecar startup failures or OOM events.
208+
- **Running multiple connections to the same destination simultaneously** can cause resource
209+
contention. Try staggering connection schedules.
210+
- If the error persists, contact [Airbyte Support](https://support.airbyte.com) with the full
211+
destination logs.
212+
213+
### HTTP 400 "Request had invalid euc header" during upload
214+
215+
If your sync fails with `BigQueryException: 400 Bad Request` and the message
216+
`Request had invalid euc header`:
217+
218+
- This is a transient error from the Google BigQuery resumable-upload API. The upload session
219+
was invalidated mid-stream.
220+
- **Retry the sync.** In most cases the error resolves on the next attempt.
221+
- If the error recurs on every sync:
222+
- Check whether a proxy, VPN, or firewall is modifying HTTP headers on requests to
223+
`bigquery.googleapis.com`.
224+
- Verify the service account key has not been rotated or revoked since the connection was
225+
configured.
226+
- Try reducing the **Google BigQuery Client Chunk Size** from the default 15 MiB to a
227+
smaller value (for example, 5 MiB). Smaller chunks reduce the window for session
228+
corruption.
229+
230+
### Broken pipe errors
231+
232+
If your sync fails with a `Broken pipe` error on the destination side:
233+
234+
- This is a transport-level error caused by a network interruption between the Airbyte worker
235+
and the BigQuery API.
236+
- **Reduce chunk size:** In the connector settings, decrease the **Google BigQuery Client Chunk
237+
Size** to 5 MiB or less. Smaller uploads complete faster and are less susceptible to
238+
connection timeouts.
239+
- **Check network infrastructure:** Firewalls, NAT gateways, and cloud load balancers may
240+
terminate idle or long-lived connections. Ensure outbound HTTPS to `*.googleapis.com` is not
241+
subject to aggressive idle timeouts.
242+
- **Retry the sync.** Broken pipe errors are typically transient.
243+
244+
### Load job timeouts
245+
246+
If your sync fails with `Fail to complete a load job in big query`:
247+
248+
- BigQuery load jobs have a 30-minute wait timeout. Very large batches or high BigQuery queue
249+
contention can exceed this limit.
250+
- Try reducing the volume per sync by using incremental sync mode or reducing the number of
251+
streams per connection.
252+
196253
## Tutorials
197254

198255
Now that you have set up the BigQuery destination connector, check out the following BigQuery

0 commit comments

Comments
 (0)