I found this https://cloud.google.com/storage-transfer/docs/create-url-list but it is no way helpful to me as i dont have the size and have the md5 of the files and i cant even download them as they are too huge like 2 gb 3gb . If anyone has knowledge of making a script that makes a tsv file for a public url of a file for Google cloud storage transfer job without downloading the file . Thanks in advance (i am not a developer so plz explain answer step by step)
Unfortunately, there is no such solution. You must download and upload the files manually!
You can go for a cloud-based solution, if you don't want to download the files on your pc and upload them manually.
Like, your cloud server will pull the files from your predefined URLs, download them locally on that virtual machine, finally upload them on your Google Cloud Storage and remove the local file from the virtual machine.
All these can be done through a script which you have to write and run on that virtual machine.
You may launch a ubuntu virtual server from Digitalocean, AWS, GCP or any other cloud providers.
So, the steps are:-