Thank you to everyone for your suggestion. I managed to get this to work.

The server doesn't need to be in the same VPC as the db.

I used this package to connect - data-api-client

Here's how I connected:

const dataApi = require('data-api-client');

const RDS = dataApi({
  secretArn: process.env.SECRET_ARN,
  resourceArn: process.env.RDS_ARN,
  database: process.env.DB_NAME,
  options: {
    region: process.env.REGION,
    accessKeyId: process.env.ACCESS_KEY_ID,
    secretAccessKey: process.env.SECRET_ACCESS_KEY,
  },
});

After turning on the data api on the database, I then had to create a secret in the secrets manager that is connected to the database. Then I created a new I AM role that gave full access to the database and secrets manager.

After that everything worked as expected.

This is likely a security group issue. You need to add the security group that your next app is on to the RDS inbound security group.