I want to download files from a web page that offers public downloads.
I have little experience with the world of web pages, so initially I wrote a simple script using selenium that solves the problem, however I find this way of getting the data too twisted. Therefore, I have tried to download the file via POST requests, investigating a little the "Network" panel of tools I managed to find the request that apparently orders the download.
The button that appears surrounded in the image launches the three requests that appear on the right, except the first time the file is downloaded, one more request appears (this can be seen above the highlighted requests).
However, the response of this request, far from being the content that would allow me to write the desired file, is more like a fragment of html code that corresponds to the content displayed when in the network panel I double click on "downloadDir":
What could I be missing? Does anyone know how to solve the problem or it is not possible by this way?
You should use requests package instead of executing curl or other processes:
import requests
response = requests.post('https://localhost:4000/bananas/12345.png', data = '[ 1, 2, 3, 4 ]')
data = response.content
data contains the downloaded content after that, which you can store to disk for instance:
with open(path, 'wb') as s:
s.write(data)