Subscribe to the Blog

Get articles sent directly to your inbox.

TL;DR

In this post we will cover how we go about implementing AWS’s S3 security best practices using IaC security – essentially making sure that the S3 buckets are secure from the get-go, without needing to fix them after-the-fact.

What do we need to do to secure our S3 buckets?

AWS has a detailed document covering their suggestions for securing S3 buckets. If you haven’t read it, suggest doing so before continuing: https://docs.aws.amazon.com/AmazonS3/latest/userguide/security-best-practices.html

They break it into “preventative” measures (essentially making sure no one can access data they shouldn’t) and “monitoring and auditing” measures (to discover if someone accessed something they shouldn’t have).

Under preventative measures, we largely see:

  • Make sure it’s not public, unless you meant for it to be.
  • Make sure that those who can access the bucket, are limited by what they can do to only what they must (least privilege concept).
  • Encrypt the data at rest (when it’s “resting” on AWS’s hardware).
  • Encrypt the data in transit (as it’s crossing the Internet).
  • Version your objects so you can roll back, and lock objects from being modified.
  • Use VPC endpoints for accessing S3 bucket data, to avoid traffic going over the public Internet.

Under monitoring and auditing measures, we largely see:

  • Make sure you didn’t miss any buckets.
  • Monitor and log – CloudWatch, CloudTracker and access logs.
  • Use AWS Config and other AWS security services to identify deviations from the suggested measures.
  • Use Amazon Macie to identify sensitive content in your S3 buckets.

These are great suggestions. In this blog post, we’ll look at how we can enforce these measures while still building the S3 buckets, so we don’t need to fix things later. As we all know, fixing a setting after your S3 bucket is used in production can be costly, time consuming, and downright depressing.

We’ll assume you use Terraform for building your S3 buckets, and have some sort of CI/CD process around it (using a “generic” CI/CD platform, or a Terraform-specific one).

How do I check if my S3 buckets are following AWS’s best practices?

First of all, as said earlier, we’re assuming you use Terraform and have a CI/CD process built around it. What this means is that you have a job/step that executes terraform plan -out=myfile and a separate job/step that runs terraform apply myfile.

In between those two steps, you insert a Terraform security analysis tool. You can use open source options like checkov, tfsec, OPA, terraform-compliance and terrascan. Or, you can choose to use our very own Cloudrail. You need to make sure to configure your CI/CD pipeline to actually look at the exit code of your selected tool, and if it’s not zero, stop the pipeline. Most tools, Cloudrail included, support common output formats like JUnit and SARIF, so that the CI/CD platform can visually display any issues easily.

Related Article 

Introducing Cloudrail’s Static Analysis Mode

Let’s review a few of AWS’s suggested best practices and how they’re handled with a Terraform security analysis tool.

Ensure the buckets are not public by their ACLs

Your Terraform code should have buckets set to private by default, with specific buckets approved to be public if they’re a must. All of the mentioned tools will tell you if you have a bucket using any of the public ACLs (public-read, public-read-write, authenticated-read).

One difference with Cloudrail is that it can actually see if you have a public access block set at the bucket level or the account level (even if set outside of Terraform!). If such a block is set, then a public ACL on a bucket will not trigger a violation. This is an opinionated approach, but avoids creating noise for the development team and stopping the CI pipeline unnecessarily.

Ensure the buckets are not public by their policy

You can have a bucket exposed publicly by setting a policy with a principal of “*”. Some of the tools mentioned above are capable of identifying this, including Cloudrail.

For example, take a look at an S3 bucket policy allowing public read:

{
  "Version":"2012-10-17",
  "Statement":[
    {
      "Sid":"PublicRead",
      "Effect":"Allow",
      "Principal": {"AWS": "*"},
      "Action":["s3:GetObject","s3:GetObjectVersion"],
      "Resource":["arn:aws:s3:::bucket-with-public-policy-2/*"]
    }
  ]
}

In this specific example, the use of "Principal": {"AWS": "*"}, is the problematic part of the policy. Different IaC security tools have varying ways of alerting about this, as can be seen here.

Ensure you follow least privilege concepts in your policies

The S3 bucket policy shouldn’t grant any principal wide actions, like “*” and “s3:*”. In addition, IAM policies attached to users, groups and roles, shouldn’t use wildcard actions. Some of the tools mentioned above are capable of identifying these issues, including Cloudrail. Take a look at an example S3 bucket without specific actions in the policy and how the different tools react to it.

Encrypt data at rest, and in-transit

Encrypting the bucket at rest is covered by all of the IaC security tools. Cloudrail takes this a step further and actually checks to see if a bucket is public (either directly, or through a proxy like CloudFront). If a bucket is public, Cloudrail will not require it to be encrypted. Again, opinionated, but saves developers time while maintaining security.

Related Article  Indeni Cloudrail Case Study: Eating Dogfood and Enjoying it

Some of the tools, Cloudrail included, will also look at your bucket policy and make sure you have a condition in your policy requiring HTTPS access to the bucket (and not allowing HTTP).

And here’s a similar example where encryption is not used, at rest or in transit, and what the different tools say about it.

Use VPC endpoints

This is commonly overlooked. Thankfully, many Terraform modules make it easy to set up an S3 VPC endpoint. For example, Anton Babenko’s VPC module will do that by default.

Most of the IaC security tools actually miss the point of the VPC endpoint entirely. Cloudrail is unique in that it not only is aware of the need for the S3 endpoints, it will also check that your S3 bucket policy is requiring traffic accessing the objects to come from your defined VPC endpoints. If it’s not, you’ll get a message about it from Cloudrail.

If you’re interested in seeing how Cloudrail’s looks like for this, take a look here.

Summarizing this in a table

S3 Security Best PracticeStatic Analysis Tools Without Context (kics, Snyk, tfsec, terrascan)Static Analysis Tools With Context (checkov)Dynamic Analysis Tools With Context (Cloudrail)
Ensure the buckets are not public by their ACLsChecks bucket ACLsChecks bucket ACLsChecks bucket ACLs, as well as public access blocks configured in the account itself, or within the IaC
Ensure the buckets are not public by their policyChecks bucket policiesChecks bucket policiesChecks bucket policies
Ensure you follow least privilege concepts in your policiesChecks bucket policies, and some IAM policiesChecks bucket policies, and some IAM policiesCalculates how IAM and bucket policies impact every pair of IAM Entity and Bucket to determine where there are exposures
Encrypt data at rest, and in-transitChecks if bucket is not encrypted, or traffic is not encryptedChecks if bucket is not encrypted, or traffic is not encryptedChecks only buckets that are private – that is no public ACLs, no public access via S3 policy, and not public access via CloudFront.
Use VPC endpointsNot supportedNot supportedChecks which VPCs have entities accessing known services, then verifies VPC endpoints are configured, routing tables have assignments for those VPC endpoints and that the S3 bucket policy uses SourceVpce.

So, how does a secure S3 bucket look like in Terraform?

As an example, here is our own S3 bucket module that we use for S3 buckets used by the Cloudrail service:

resource "aws_s3_bucket" "bucket" {
  bucket = var.name
  acl    = var.acl # default: private

  server_side_encryption_configuration {
    rule {
      apply_server_side_encryption_by_default {
        sse_algorithm = "AES256"
      }
    }
  }

  lifecycle_rule {
    enabled = var.expiration_days > 0 ? true : false

    expiration {
      days = var.expiration_days
    }
  }

  dynamic "cors_rule" {
    for_each = var.cors_rule

    content {
      allowed_methods = cors_rule.value.allowed_methods
      allowed_origins = cors_rule.value.allowed_origins
      allowed_headers = lookup(cors_rule.value, "allowed_headers", null)
      expose_headers  = lookup(cors_rule.value, "expose_headers", null)
      max_age_seconds = lookup(cors_rule.value, "max_age_seconds", null)
    }
  }

  tags = merge(var.tags, {
    Name = "${var.name}"
  })
}

# Additional policies for bucket
resource "aws_s3_bucket_policy" "s3-bucket-policy" {
  count  = length(var.additional_policies)
  bucket = aws_s3_bucket.bucket.id

  policy = var.additional_policies[count.index]
}

# Ensure no public access is possible
resource "aws_s3_bucket_public_access_block" "bucket" {
  bucket = aws_s3_bucket.bucket.id

  block_public_acls       = var.block_public_acls # default: true
  block_public_policy     = var.block_public_policy # default: true
  ignore_public_acls      = var.ignore_public_acls # default: true
  restrict_public_buckets = var.restrict_public_buckets # default: true
}

And here’s how it’s used:

locals {
  vpce_policy = {
    "Version" : "2012-10-17",
    "Id" : "Policy1415115909152",
    "Statement" : [
      {
        "Sid" : "Deny-object-actions-non-from-VPCE",
        "Principal" : "*",
        "Action" : "s3:*Object",
        "Effect" : "Deny",
        "Resource" : ["arn:aws:s3:::%[1]s/*"]
        "Condition" : {
          "NotIpAddress" : {
            "aws:SourceIp" : "${var.s3_allowed_ip_addresses}"
          },
         "StringNotEquals" : {
           ### We added the hardcoded ID of VPCE to S3 (from Production Management VPC)
           "aws:SourceVpce" : ["${module.s3-endpoint.id}", "vpce-05d218.......b2f0"]
         }
       }
     }
   ]
  }
}
### Process S3
module "process" {
  source = "../s3"

  name = "${lower(local.basename)}-process-s3"
  tags = local.base_tags

  expiration_days = 1

  additional_policies = [format(jsonencode(local.vpce_policy), "${lower(local.basename)}-process-s3")]
}

The “process” S3 bucket is one of the buckets we have in our setup.

Conclusion

There have been many security incidents in recent years caused by misconfigured S3 buckets. To avoid that from happening to your organization, follow AWS best practices, and enforce them at the build stage, by using a Terraform security tool.

Take a look at Cloudrail today, by signing up and beginning to use it within minutes.