I am having an issue using Terraform (v0.9.2) adding services to an ELB (I'm using: https://github.com/segmentio/stack/blob/master/s3-logs/main.tf).
When I run terraform apply I get this error:
* module.solr.module.elb.aws_elb.main: 1 error(s) occurred:
* aws_elb.main: Failure configuring ELB attributes:
InvalidConfigurationRequest: Access Denied for bucket: my-service-
logs. Please check S3bucket permission
status code: 409, request id: xxxxxxxxxx-xxxx-xxxx-xxxxxxxxx
My service looks like this:
module "solr" {
source = "github.com/segmentio/stack/service"
name = "${var.prefix}-${terraform.env}-solr"
environment = "${terraform.env}"
image = "123456789876.dkr.ecr.eu-west-2.amazonaws.com/my-docker-image"
subnet_ids = "${element(split(",", module.vpc_subnets.private_subnets_id), 3)}"
security_groups = "${module.security.apache_solr_group}"
port = "8983"
cluster = "${module.ecs-cluster.name}"
log_bucket = "${module.s3_logs.id}"
iam_role = "${aws_iam_instance_profile.ecs.id}"
dns_name = ""
zone_id = "${var.route53_zone_id}"
}
My s3-logs bucket looks like this:
module "s3_logs" {
source = "github.com/segmentio/stack/s3-logs"
name = "${var.prefix}"
environment = "${terraform.env}"
account_id = "123456789876"
}
I checked in S3 and the bucket policy looks like this:
{
"Version": "2012-10-17",
"Id": "log-bucket-policy",
"Statement": [
{
"Sid": "log-bucket-policy",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::123456789876:root"
},
"Action": "s3:PutObject",
"Resource": "arn:aws:s3:::my-service-logs/*"
}
]
}
As far as I can see ELB should have access to the S3 bucket to store the logs (it's running in the same AWS account).
The bucket and the ELB are all in eu-west-2.
Any ideas on what the problem could be would be much appreciated.
The docs for ELB access logs say that you want to allow a specific Amazon account to be able to write to S3, not your account.
As such you want something like:
{
"Id": "Policy1429136655940",
"Version": "2012-10-17",
"Statement": [
{
"Sid": "Stmt1429136633762",
"Action": [
"s3:PutObject"
],
"Effect": "Allow",
"Resource": "arn:aws:s3:::my-loadbalancer-logs/my-app/AWSLogs/123456789012/*",
"Principal": {
"AWS": [
"652711504416"
]
}
}
]
}
In Terraform you can use the aws_elb_service_account data source to automatically fetch the account ID used for writing logs as can be seen in the example in the docs:
data "aws_elb_service_account" "main" {}
resource "aws_s3_bucket" "elb_logs" {
bucket = "my-elb-tf-test-bucket"
acl = "private"
policy = <<POLICY
{
"Id": "Policy",
"Version": "2012-10-17",
"Statement": [
{
"Action": [
"s3:PutObject"
],
"Effect": "Allow",
"Resource": "arn:aws:s3:::my-elb-tf-test-bucket/AWSLogs/*",
"Principal": {
"AWS": [
"${data.aws_elb_service_account.main.arn}"
]
}
}
]
}
POLICY
}
resource "aws_elb" "bar" {
name = "my-foobar-terraform-elb"
availability_zones = ["us-west-2a"]
access_logs {
bucket = "${aws_s3_bucket.elb_logs.bucket}"
interval = 5
}
listener {
instance_port = 8000
instance_protocol = "http"
lb_port = 80
lb_protocol = "http"
}
}
Even when having everything by the docs, I kept getting the "Access Denied for bucket" error. Removing the encryption from the bucket worked for me.
In the bucket policy, the account number must be NOT yours. Instead it belongs to AWS, and for each region, the account numbers you should use in your bucket policy are listed at: https://docs.aws.amazon.com/elasticloadbalancing/latest/classic/enable-access-logs.html#attach-bucket-policy
For instance, for us-east-1 region the account number is 127311923021.
Although the question is about Terraform, I post CloudFormation snippet created a bucket for ELB's access logs its Bucket policy:
MyAccessLogsBucket:
Type: AWS::S3::Bucket
DeletionPolicy: Retain
MyAllowELBAccessBucketPolicy:
Type: AWS::S3::BucketPolicy
Properties:
Bucket: !Ref MyAccessLogsBucket
PolicyDocument:
Version: "2012-10-17"
Statement:
- Effect: "Allow"
Principal:
AWS: "arn:aws:iam::127311923021:root"
Action:
- "s3:PutObject"
Resource: !Sub "arn:aws:s3:::${MyAccessLogsBucket}/AWSLogs/*"
In the principle, 127311923021 is used as this is AWS account number which should be used for account number in us-east-1.