Airbyte Azure Blob destination fails when stream has zero records (“Empty non-final part received”) #73233
Rishabh1627rawat
started this conversation in
General
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
I am using Airbyte Custom Connector Builder in airbyte hosted we have a api for which we devleoped the custom builder there are different fileds in it like Patients,Assessments,Incidents,etc and and i can see thet data in my custom builder when i click on test
Some patients do not have any assessments so there assessments is empty
This is valid and expected data
In the connector:
I configured Error Handler to IGNORE 404 for the Assessments stream
I also added record filters so that patients with no assessments should be filter out
However, when the Assessments stream ends up with zero records, the sync fails at the destination with this error:
java.lang.IllegalStateException: Empty non-final part received: this should not happen
From the logs, this error comes from the Azure Blob destination while writing data.
Ii dont understand f I am already skipping records for patients with no assessments, why does Airbyte still try to write something to Azure Blob and fail?
Is there a way to
Prevent Airbyte from writing empty files/parts to Azure Blob?
Or configure Azure Blob destination to handle streams with 0 records?
Or is this a known limitation/bug with Azure Blob destination?
Environment
Airbyte version: 1.x
Source: Custom Connector (Builder / Declarative)
Destination: Azure Blob Storage
Stream: Assessments (can legitimately return zero records)
Any guidance or recommended workaround would be appreciated.
Beta Was this translation helpful? Give feedback.
All reactions