The cloud has made it easier than ever to adhere to compliance standards such as HIPAA, HITRUST and PCI-DSS. Requirements to enable encryption of data at rest/in transit, maintain secure networks and perform frequent server patching are features that are baked in to many cloud services.
One critical compliance requirement is to capture and securely retain log data. Aside from being mandatory, it is critical for a variety of reasons. Reports suggest that it frequently takes companies more than 200 days before realizing that they have suffered a breach. Being able to review historical log data is often necessary to assess the full impact of the breach and to find out how it happened.
AWS makes it easy to capture necessary log files for your cloud environment. With just a few mouse clicks you can log and retain:
- all information about the IP traffic going to and from network interfaces using VPC Flow Logs
- detailed information about requests sent to your load balancer using ELB Access Logs
- event history of your AWS account activity, including actions taken through the AWS Management Console, AWS SDKs, command line tools, and other AWS services using AWS CloudTrail
- details about access requests to your S3 buckets using S3 Server Access Logging
- information to evaluate the configurations of your AWS resources using AWS Config
These logs files can be pushed to CloudWatch for immediate viewing and to S3 for long term archival. However, this is just the tip of the iceberg as you can also push any of your application log files to CloudWatch and S3.
If the worst should happen and you suffer a breach, having to trawl through all your logs to discover the cause can be a soul destroying and reactive process. Instead, putting a solution in place to easily review, monitor and alert in real-time is a far more proactive approach.
Let’s take a look at some of the solutions available today that can make your log management much simpler.
This newly released service makes it easy to analyze, investigate, and quickly identify the root cause of potential security issues or suspicious activities. The service automatically analyzes all VPC Flow Logs, AWS CloudTrail, and Amazon GuardDuty logs and creates a centralized, interactive view.
Unfortunately, this service doesn’t yet analyze Amazon Config, S3 Access Logs, ELB Access logs or any other CloudWatch logs that you may have. However, I would expect these to added at some point in the future.
Splunk can do it all and is the granddaddy of log analysis solutions. With more features than you’ll probably ever need, Splunk is capable of ingesting all your AWS logs in one place for graphs, reports, alerts, dashboards, and visualizations.
The downside to this solution is the cost. It is very, very expensive and is more of an enterprise solution than most smaller businesses can afford.
This new kid on the block is a lightweight version of Splunk, it doesn’t anything close to the same number of features, but it allows you to ingest AWS log files in an easy-to-use interface for analysis, graphs and alerts.
LogDNA has plenty of integrations to ingest your AWS log files from CloudWatch and other AWS services, but the S3 implementation is still work in progress. However, that feature may be available by the time you read this blog.
If you are looking for a solution to solve your log file issues, databasable has masses of experience with AWS, compliance and log management. Please reach out to learn more.