I have about a dozen aws lambda functions written in python. Each does its thing and logs what happens to CloudWatch Logs. that works fine, however it is tedious to parse through the logs for each of the lambdas as i make them very detailed for debugging.
What i would like to do is have a 'master log'. One log file for all of the lambdas (in addition to the cloudwatch one for each). this master log would aggregate all of the critical messages from all of the lambdas and leave out the mundane fluff.
Where and how are (is?) the best way to get this done?
I can use boto3 but it looks like S3 files are not really made to be appended to - they are meant to be replaced. that could be messy with 12 lambdas logging to it at once.
Perhaps could i use boto with cloudwatch logs?
AWS Lambda configures the logging for you automatically and it doesn't look like there is a way to customize which log group or log stream is used.
I haven't tried it but it seems like you could pass in your own log group and log stream to your handler, either as parameters to the handler or as environment variables. Then you could use something like watchtower to configure your own logging handler so all of your functions write to the same place.