I am trying to upload a CSV file of 60 mb to my application. But when i save the file to server , it only contains partial(25 mb) of total records from the original. This issue is occurring only randomly.
@fileupload.route('/loadfile',methods = ['POST'])
def store_to_db():
for _file in request.files.getlist('filelist'):
name, ext = FileUtil.get_filename_without_ext(_file.filename)#fetching file name and extension
fd, path = tempfile.mkstemp(suffix=ext, prefix=name)#saving file as temp
os.write(fd, _file.read())
try:
file_upload_service.savefiletodb(path)
except Exception as e:
logger.info(str(e))
os.close(fd)
Uploader side code:
data= {}
url_path = 'localhost:5000/loadfile'
data['filelist']=open('employe.txt', 'rb')
resp = requests.post(url=url_path, files=data)
Any idea why its behaving like this ?
Note:
When i upload through postman, i am not facing any issue.
If I upload through code , sometimes I get partial file.
file object in log , when upload via postman
'employee.TXT' ('text/plain')>
file object in log , when upload via code
'employee.TXT' ('None')>
If the issue is with the file size which is exceeding the file size limit, Flask will raise a RequestEntityTooLarge exception.
If this is the case, you can simply configure the MAX_CONTENT_LENGTH config key.
For example,
from flask import Flask, Request
app = Flask(__name__)
app.config['MAX_CONTENT_LENGTH'] = 20 * 1024 * 1024
The above code will set the File upload limit to 20 megabytes. If the user tries to upload a file larger than the set limit (in this case 20 MB), Then Flask will raise a RequestEntityTooLarge exception.
And then if required, Allowed file extensions can also be specified by,
ALLOWED_EXTENSIONS = {'txt', 'docx','pdf', 'png', 'jpg', 'jpeg', 'gif'}
For more help, Refer the Flask Docs for Uploading files.