Wanted to check with you guys that is there any service/google cloud api support to invoke file transfer from AWS S3 to google storage, as soon as new file get created in AWS S3 bucket?
I know that google cloud has storage transfer service, but as per my knowledge it cannot invoked for single file or trigger immediately if new file get added in AWS S3 bucket. We want to transfer as soon as there is new file in AWS S3.
We need this solution as we have implemented angularJS application to upload videos to AWS S3, but view videos from S3 is very bad experience, performance is very bad. So we wanted to write backend move files to google storage as soon as it get added into S3, this is short term solution we thought till the time we don’t have provision to upload videos directly to the google storage.
You can publish S3 events to AWS Lambda and run your transfer inside the Lambda function. Running Lambda functions in response to the objects created in S3 is a common use case. See Using AWS Lambda with Amazon S3.
Amazon S3 can publish events (for example, when an object is created in a bucket) to AWS Lambda and invoke your Lambda function by passing the event data as a parameter. This integration enables you to write Lambda functions that process Amazon S3 events. In Amazon S3, you add bucket notification configuration that identifies the type of event that you want Amazon S3 to publish and the Lambda function that you want to invoke.