I am trying to string together lambda functions. The first lambda function is on a 30 minute timer to scrape and put the data in a S3 bucket, the next lambda function retrieves and parses that data and puts it in a seperate S3 bucket, and the last function performs analysis on that data and sends the user (in this case myself) an email of the results via pythons smtplib module.
Instead of having the last two lambda functions running on timers, I want the second function to be triggered when the first function is done, and the last function to be triggered when the second function is done. As well as, deleting the two folders in the first S3 bucket and the contents in the second S3 bucket to save on memory and processing time.
Is there a way to do this totally in the AWS web interface rather than rewrite the python code I already have?
This sounds like you have a use case that can fulfilled by step functions.
You would add a flow which has the first Lambda called, then if successful call the second Lambda function passing input/output between.
You would update your scheduled action to execute the step function instead of the first Lambda.
It can be done by using S3 Put Event as a Trigger to Lambda.
Second and third lambda can have S3 Put Event Trigger.
Timer Put Event Put Event
-------Lambda_1 -> S3 ------------> Lambda_2 --> S3 ------------> Lambda_3 ---> Email
Check Below link for more info :
Another solution which would involve rewriting your python code, but only slightly, is:
import boto3
lambda_client = boto3.client('lambda')
payload = b'{"a": 1, "b": 2, "c": 3}'
response = lambda_client.invoke(
FunctionName="my-second-function",
InvocationType='Event',
Payload=payload
)
By contrast, I find Step Functions to be way more complicated.