Are Your AWS CloudTrail Costs Out of Control? Here’s Why

Are you happy with your CloudTrail bill? I asked that same question in my previous article about CloudWatch, and now, it is time to reflect on AWS CloudTrail.

In this article, I will explore possible reasons why you are overspending on CloudTrail, and discuss ways to keep the costs of CloudTrail well controlled.

Introduction

AWS CloudTrail is one of those services that delivers huge value to the AWS ecosystem while not spending tons of money. It enables auditing, security monitoring, and operational troubleshooting by tracking your user activity and API calls.

However, it is also one of those services that can get very expensive quite easily - especially because it tracks all activities in an AWS account/organization and generates audit logs.

Understanding how AWS charges you is a crucial step to avoid misconfiguration and keep compliance at the lowest price point.

The Serverless Paradigm

Yes, AWS CloudTrail is a serverless service. And that is because it is a fully managed service that automatically records AWS API activity and stores it in Amazon S3 or CloudWatch Logs. You don’t need to provision or manage servers - AWS handles everything behind the scenes.

Serverless is not just about the literal meaning of "Server-less" but also the principles behind it, for example, no infrastructure management and pay-per-use pricing. However, the trick part lies in the last one: the first copy of trails is free, but the second copy onwoards inccur charges, and this is where you might be spending quite some money.

CloudTrail pricing

Let's understand how AWS customers pay for CloudTrail. I want to focus on Trails, which is the subject of this article. On the CloudTrail pricing page, there are also details about Lake and Insights, which I will not cover here for now.

Do you want me to investigate and share insights about Lake or Insights for a future article? Let me know in the comments!

• For management events delivered to S3, you pay $2.00 per 100,000 events delivered (after the first free copy)
• For data events delivered to S3, you pay $0.10 per 100,000 events delivered
• For network activity events delivered to S3, you pay $0.10 per 100,000 events delivered

NOTE: Amazon S3 charges apply and are not included in this analysis.

If you take a close look at the pricing and compare it to the trails in our AWS account, you can immediately see where the eventual high costs come from, so now, it is important to understand why.

Duplicated trails

The sentence I want you to reflect on from the pricing is "after the first free copy". What does that mean?

Let's imagine you set up organization trails in all member accounts. These trails go to an S3 bucket in a centralized logging account, for example. To deliver the same events to other destinations to allow different groups (for instance, developers, security, auditors, etc) to get their copy of these audit logs, you also created trails in the individual accounts. Although this is a valid use case, this can be costly - the first copy of these events is free of charge and you pay for the other ones - and that is why I call this duplicated trails. They generate a metric called PaidEventsRecorded, and this is how AWS charges you.

Still on the hypothetical AWS account, let's say this environment generates 5 million management events delivered to S3 per month:

• The first trail, delivered to an S3 bucket that developers can access, is free of charge.
• The second trail, delivered to an S3 bucket that the security team can access, will cost $100 (5,000,000 / 100,000 * $2.00 = $100)
• The third trail, delivered to an S3 bucket that the auditors team can access, will cost $100 (5,000,000 / 100,000 * $2.00 = $100)

In total, these duplicated trails cost you $200. But is there something we can do in this scenario while still keeping compliance and the least privileges?

To avoid unnecessary costs from the PaidEventsRecorded metric, you can opt to remove the trails created in the specific accounts and keep the organization trail. By doing that, all the management events trail logs are still delivered to a centralized S3 bucket, but now you can control access per account with IAM roles.

For example, to give access to developers on their specific accounts within the organization, you can create an IAM role to be assumed by the developers from a specific account but only allow them to access the bucket prefix of the trails from their account. Your policy would look more or less like this:

{
    "Effect": "Allow"
    "Resource": "arn:aws:s3:::organization-trail-bucket/AWSLogs/OU_ID/ACCOUNT_ID/*",
    "Action": [
        "s3:Get*",
        "s3:HeadObject",
        "s3:List*",
        "s3:RestoreObject"
    ],
},
{
    "Effect": "Allow"
    "Resource": "arn:aws:s3:::organization-trail-bucket",
    "Action": "s3:ListBucket",
    "Condition": {
        "StringLike": {
            "s3:prefix": [
                "AWSLogs/OU_ID/ACCOUNT_ID/*"
            ]
        }
    },
}

By keeping the organization trail delivering logs to a centralized S3 bucket and controlling the access via IAM roles, you can eradicate the costs of having duplicated trails.

Data events

Slightly more complicated than the Management Events, you also pay for Data Events delivered to S3. Because there are no free copies (they always incur charges from the first copy) and the data events can generate a lot more audit logs, it's more difficult to reduce costs. They generate a metric called DataEventsRecorded, and this is how AWS charges you.

Nevertheless, data events are important for audit and compliance purposes. Of course, you can always negotiate with the audit team or the team responsible for those logs if they are necessary according to the policies defined by the organization.

AWS's advice is to "filter out AWS KMS or Amazon RDS Data API events by choosing Exclude AWS KMS events or Exclude Amazon RDS Data API events on the Create trail or Update trail pages". This can help you reduce the number of logs generated by data events.

Monitoring CloudTrail costs

Using the Cost Explorer console, you can get an overview of the PaidEventsRecorded and DataEventsRecorded metrics and how their costs increase/decrease over time.

You can select the metrics under the "Usage Type" filter:

Conclusion

In conclusion, managing AWS CloudTrail costs effectively requires understanding the pricing structure and identifying areas where expenses can be reduced.

By avoiding duplicated trails and utilizing IAM roles to control access to a centralized S3 bucket, you can eliminate unnecessary charges from multiple copies of management events.

For data events, consider filtering out less critical logs to reduce costs.

Regularly monitoring these expenses using tools like Cost Explorer can help you track and manage your CloudTrail spending more efficiently, ensuring compliance without overspending.

I would love to hear your thoughts! Let me know in the comments how your experience was with CloudTrail trails costs.