Business
Jobs
  • About Us
  • Solutions
    • Job Postings
      Post your job and receive qualified candidates in 48h.
    • Candidate Assessments
      500+ technical and psychological tests, plus anti-fraud.
    • Headhunting
      Tailor-made executive search from start to finish.
    • Payroll + EOR
      Payroll dispersal and EOR across 15+ LATAM countries.
  • Pricing
  • Jobs

0

80
Views
Send CloudWatch logs to database

I am attempting to save our Cloudwatch logs in an on-premise Postgres database. I'm currently exporting logs to S3 and save in DynamoDB. My requirement now is to persist it in our DB, using node and AWS js-SDK. I'm not very strong on node and js-SDK, so I'll greatly appreciate any idea.

I tried a simple implementation.

const pools = require('../src/common/db'),
const AWS = require('aws-sdk');

// set the cwl
let cwl = new AWS.CloudWatchLogs({
  region: 'us-east-1',
  accessKeyId: 'ABCD1234',
  secretAccessKey: 'MNBV76543',
  Bucket: 'My_bucket'
});

// Get the events
cwl.getLogEvents({
  logGroupName: 'OurLogGroupname',
  logStreamName: 'specifiedLogstream'
}, (error, success ) =>{
  if(error){
    console.log(error)
  }
  console.log(success)
})

// Try saving to db
let sql = ''
pools.query_db('abc', 'INSERT INTO logging.aws_logs(request_id, duration, billed_duration) VALUES (?,?,?)', function(err, res){
  if(err) return callback(err);
  callback();
})
about 4 years ago · Juan Pablo Isaza
1 answers
Answer question

0

I would prefer the following way, if you really want to store all messages from Cloudwatch into a database:

  • Add a subscription to your Cloudwatch LogGroup

  • This subscription can be configured to trigger a Lambda

  • The Lambda will have the following logic:

    1. extract the message from the event variable
    2. prepare your SQL statement
    3. connect to database (retry if not possible)
    4. execute the SQL statement (retry if not possible)
    5. done

One good example on how to extract the message of a Cloudwatch Subscription invocation would the one for sending those logs to Opensearch (search the blueprints)

about 4 years ago · Juan Pablo Isaza Report
Answer question
Find remote jobs

Discover the new way to find a job!

Top jobs
Top job categories
Business
Post vacancy Pricing Sales
Legal
Terms and conditions Privacy policy
© 2026 PeakU Inc. All Rights Reserved.
Andres GPT
Show me some job opportunities
There's an error!