Afternoon,
I recently came across AWS Lambda and Azure Functions. AWS imposes a limit on the size of zipped as well as unzipped files, which for python scripts need to include all of the dependent modules. I have been using lambda-uploader to package my script and it's module dependencies, but the pandas package is too big.
I have seen examples of people completing machine learning and using pandas on AWS Lambda (a little outdated though) but I can't see how they're doing it. Any suggestions?
The package that you upload to lambda should not contain anything but the code and support modules required for Lambda to run your code. The Lambda console UI limits the file size to 10MB but you can upload zip files up to 50MB if you place them in an S3 bucket and then request that Lambda load them from S3.
Any other assets that you require for execution such as machine learning models should be uploaded separately to S3 and then downloaded from within your Lambda function at execution time. The Lambda function can write to a /tmp folder but keep in mind it only has access to 512MB of disk space. Also keep in mind that the Lambda function has a maximum runtime of 300 seconds so downloading really large files will take time away from your function doing real work with the data you're downloading.
If you're using Python libraries, you can get rid of botocore, boto3 as they are already present in AWS's lambdas functions.