Delivery Method: Copy Raw files - How to use it? #69727
Unanswered
Ivan (ivan-opmed)
asked this question in
Platform General Questions
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
Hey guys,
I am trying to use the feature of Copy raw files
https://docs.airbyte.com/platform/using-airbyte/delivery-methods#metadata-sample
As I see when I apply the copy-raw-files flag for S3 source or SFTP bulk source it downloads files to the local /tmp folder
But on the destination it will store it as csv output file which include metadata of it - only 1 row which says that the file was downloaded and the size of it, name and path.
Whats the proper way to do it? Our use-case is to transfer raw(binary) files from S3 or SFTP directly to S3 (csv file was used just as test)
I was trying also to mount disk under the helm chart deployment for source/destination to enable access for both containers but it was unsuccessfully.
Who else used this feature and whats the real purpose of it if it's not working for us as described?
Specs:
Beta Was this translation helpful? Give feedback.
All reactions