InsightIDR

AWS CloudTrail

Amazon Web Services, or AWS, is a cloud service integration that allows you to track how your corporate cloud services are being used.

Before You Begin

Before you can set up this event source, you'll need to complete several tasks first:

Enable Access to S3 Regions

Depending on the region your CloudTrail logs are stored in, the collector will need to be able to reach the following URL to collect the logs:

Enable CloudTrail in all regions

To get maximum coverage of CloudTrail monitoring, you should enable CloudTrail in all regions, even if you don't have any EC2 instances or other AWS resources running in all regions. This helps to ensure that, going forward, if an attacker compromises a resource in your AWS account that allows them to create/modify resources in other regions, you'll be able to monitor and alert on that behavior.

To enable CloudTrail:

  1. In the AWS Console, go to CloudTrail → Trails → Create new trail.
  2. Add a name for your trail in the "Trail name" field.
  3. For the "Apply trail to all regions" option, select Yes .
  4. For the "Create a new S3 bucket" option, select Yes.
  5. Add a name for your S3 bucket. Record this for future steps.
  6. Click Create.

Create IAM Policy

Next, you must create the IAM policy to control privileges and access.

To create the IAM policy:

  1. In the AWS Console, go to IAM → Policies → Create Policy → Create Your Own Policy.
  2. Add a Name and Description for your Policy. Keep note of these for later use.
  3. Enter a policy using the following Policy template, which is based on the principal of least privilege and only allows access to the specific S3 bucket you created for your CloudTrail logs:
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "s3:Get*",
                "s3:List*"
            ],
            "Resource": [
                "arn:aws:s3:::CloudTrailsS3BucketNameGoesHere",
                "arn:aws:s3:::CloudTrailsS3BucketNameGoesHere/*"
            ]
        }
    ]
}

Create IAM Group

Now you must create the IAM group, which controls user access to a group with specific privileges.

  1. In the AWS Console, go to IAM → Groups → Create New Group.
  2. Create a Group Name and select Next Step.
  3. Select the IAM Policy you created earlier and select Next Step.

Create and configure IAM User

Now create the user that can access the group and will inherit the privileges.

To create the user:

  1. In the AWS Console, go to IAM → User → Add user.
  2. Add a User name and select Programmatic Access under the "Access Type" section and select Next: Permissions.
  3. Select the Group you created earlier and select Next: Review.
  4. On the "Complete" page, select Show on the Secret Access Key.
  5. Copy and save this User's Access Key and Secret Key in a secure location for later use.

Setup S3 bucket policy

Finally, create a policy for the S3 bucket that dictates that the user and S3 bucket data are associated.

To create the bucket policy:

  1. Find the ARN for the user associated with the access key configured in the collector:
  1. Find the bucket configured for the CloudTrail logs.
  2. Go to the bucket properties in "S3" and click Edit Bucket Policy.**
  3. Add List* and GetObject rights to the bucket that match the ARN of the user.
  4. Click Save.
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "AWSCloudTrailAclCheck20150319",
            "Effect": "Allow",
            "Principal": {
                "Service": "cloudtrail.amazonaws.com"
            },
            "Action": "s3:GetBucketAcl",
            "Resource": "arn:aws:s3:::CLOUDTRAILS S3 BUCKET NAME"
        },
        {
            "Sid": "AWSCloudTrailWrite20150319",
            "Effect": "Allow",
            "Principal": {
                "Service": "cloudtrail.amazonaws.com"
            },
            "Action": "s3:PutObject",
            "Resource": "arn:aws:s3:::CLOUDTRAILS S3 BUCKET NAME/AWSLogs/AWS ACCOUNT NUMBER/*",
            "Condition": {
                "StringEquals": {
                    "s3:x-amz-acl": "bucket-owner-full-control"
                }
            }
        },
        {
            "Sid": "",
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::AWS ACCOUNT NUMBER:user/IAM USER NAME"
            },
            "Action": "s3:List*",
            "Resource": [
                "arn:aws:s3:::CLOUDTRAILS S3 BUCKET NAME",
                "arn:aws:s3:::CLOUDTRAILS S3 BUCKET NAME/*"
            ]
        },
        {
            "Sid": "",
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::AWS ACCOUNT NUMBER:user/IAM USER NAME"
            },
            "Action": "s3:GetObject",
            "Resource": "arn:aws:s3:::CLOUDTRAILS S3 BUCKET NAME/*"
        }
    ]
}

How to Configure This Event Source

  1. From your dashboard, select Data Collection on the left hand menu.

  2. When the Data Collection page appears, click the Setup Event Source dropdown and choose Add Event Source.

  3. From the “Security Data” section, click the Cloud Services icon. The “Add Event Source” panel appears. 

  4. Choose your collector and event source. You can also name your event source if you want.

  5. Choose a time zone and optionally display only US time zones.
  6. Optionally choose to send unfiltered logs.
  7. Select your existing credentials or use EC2 IAM Roles.
  8. Enter the Secret Key created in previous steps.
  9. Enter the S3 Bucket Name created in previous steps.
  10. Enter the S3 Key Prefix created in previous steps.
  11. Select the Bucket Region Name.
  12. Enter the refresh rate in minutes.
  13. Configure your default domain and any Advanced Event Source Settings
  14. Click Save.

Troubleshooting

InsightIDR Not Ingesting Logs

If you find that InsightIDR is not ingesting logs and data is not appearing, please do the following:

  1. Check that your IAM policy is correct.
  2. Check that you've used the right region.
  3. Ensure there are actually logs in the S3 bucket.
  4. Ensure that the S3 region of your event sources matches the S3 region used by your CloudTrail.

301 Error

If you encounter this error, this means that the S3 region in the event source does not match the region of the CloudTrail logs. Make sure that both the event source and your CloudTrail use the same S3 region.

Difficulty with S3 Key Prefix

Note that key prefixes are only necessary in the event source configuration if you configured one in AWS CloudTrail.

A normal structure without a key prefix is as follows: bucket_name/AWSLogs/Account ID/CloudTrail/region/YYYY/MM/DD/file_name.json.gz

Structures with a key prefix would look like this: bucket_name/prefix_name/AWSLogs/Account ID/CloudTrail/region/YYYY/MM/DD/file_name.json.gz

Adding an S3 Bucket Folder

To add an S3 bucket folder name, simply add / at the end of the bucket name.

AWS CloudTrail


Suggested Edits are limited on API Reference Pages

You can only suggest edits to Markdown body content, but not to the API spec.