Retrieve and Analyze VPC Flow Logs with the AWS CLI

How to Retrieve and Analyze VPC Flow Logs Using the AWS CLI

Amazon VPC Flow Logs are an essential tool for monitoring and troubleshooting your AWS network. They capture IP traffic information going to and from your network interfaces, allowing you to diagnose security group rules, track connectivity, and audit network activity.

While the AWS Management Console is great for visual checks, the AWS Command Line Interface (CLI) offers a powerful, scriptable, and efficient way to retrieve and analyze these logs directly from your terminal. This is especially useful for automation, integrating with other scripts, or quickly searching through large volumes of log data.

This guide will show you the exact AWS CLI commands to retrieve your VPC flow logs, whether you’re sending them to Amazon CloudWatch Logs or an Amazon S3 bucket.

Prerequisites: Before you begin, ensure you have the AWS CLI installed and configured with the necessary IAM permissions to access CloudWatch Logs (logs:*) and S3 (s3:*).

Your Log Destination Matters: The commands you’ll use depend entirely on where you configured your flow logs to be published. There are two destinations:

• Amazon CloudWatch Logs: Ideal for real-time analysis, alarming, and structured querying with Logs Insights.
• Amazon S3: Best for long-term archival, compliance, and large-scale offline analysis with tools like Amazon Athena or third-party log processing systems.

We’ll cover both methods.

Method 1: Retrieving Flow Logs from Amazon CloudWatch Logs

When your logs are in CloudWatch, you’re interacting with log groups and log streams.

Step 1: Find Your Log Group

First, you need to identify the log group. If you don’t know the exact name, you can list all log groups, filtering for common VPC flow log prefixes.

# List all log groups
aws logs describe-log-groups

# Or, query for log groups related to flow logs
aws logs describe-log-groups --log-group-name-prefix "/aws/vpc/flowlogs" --query 'logGroups[*].logGroupName'

Step 2: List the Log Streams

Each network interface (ENI) typically sends its logs to a different log stream within that group. You need to list the streams to find the one you want to inspect.

# Replace YOUR_LOG_GROUP_NAME with the name from Step 1
aws logs describe-log-streams --log-group-name YOUR_LOG_GROUP_NAME --query 'logStreams[*].logStreamName' --output text

Step 3: Get the Log Events

Once you have both the log group and log stream names, you can fetch the actual log data.

# Replace with your specific group and stream names
aws logs get-log-events \
    --log-group-name YOUR_LOG_GROUP_NAME \
    --log-stream-name YOUR_LOG_STREAM_NAME \
    --query 'events[*].message' --output text

This command will output the raw log messages. You can add --start-time and --end-time (using Unix timestamps in milliseconds) to narrow your search to a specific timeframe.

Pro-Tip: Use CloudWatch Logs Insights for Filtering

A much more powerful CLI method is to use Logs Insights, which lets you query and filter before retrieving data.

For example, to find all REJECTED traffic from the last 3 hours:

# Get a timestamp for 3 hours ago (syntax varies by OS)
# macOS/BSD:
START_TIME=$(date -v-3H +%s000)
# GNU/Linux:
# START_TIME=$(date -d '3 hours ago' +%s000)

aws logs start-query \
    --log-group-name YOUR_LOG_GROUP_NAME \
    --start-time $START_TIME \
    --end-time $(date +%s000) \
    --query-string "fields @timestamp, srcAddr, dstAddr, action | filter action = 'REJECT' | sort @timestamp desc | limit 20"

This returns a queryId. You then use that ID to get the results:

# Wait a few seconds for the query to run, then:
aws logs get-query-results --query-id "YOUR_QUERY_ID_FROM_PREVIOUS_COMMAND"

Method 2: Retrieving Flow Logs from Amazon S3

When your logs are in S3, you’re dealing with Gzip-compressed text files organized in a specific folder structure.

Step 1: List the Log Files in Your Bucket

The S3 path for flow logs follows a standard format: s3://<bucket-name>/<prefix>/AWSLogs/<account-id>/vpcflowlogs/<region>/<YYYY>/<MM>/<DD>/.

You can use the aws s3 ls command to navigate this structure.

# List the contents of your bucket (replace with your bucket name)
aws s3 ls s3://your-log-bucket-name/prefix/AWSLogs/

# List logs for a specific day (e.g., October 18, 2025 in us-east-1)
# Replace with your Account ID, Region, and Date
aws s3 ls s3://your-log-bucket-name/prefix/AWSLogs/123456789012/vpcflowlogs/us-east-1/2025/10/18/ --recursive

This will show you all the .log.gz files for that day.

Step 2: Download the Log File

Once you’ve identified a file, use aws s3 cp to download it to your local machine.

# Replace with the full S3 path to the log file
aws s3 cp s3://your-log-bucket-name/prefix/AWSLogs/123456789012/vpcflowlogs/us-east-1/2025/10/18/123456789012_vpcflowlogs_us-east-1_fl-12345abc_20251018T1500Z_a1b2c3d4.log.gz .

# The "." at the end copies it to your current directory

Step 3: Decompress and Analyze the File

The downloaded file is compressed. On Linux or macOS, you can decompress it and search it in one go.

# Decompress and search for all "REJECT" entries
gunzip -c 123456789012_vpcflowlogs_us-east-1_fl-12345abc_20251018T1500Z_a1b2c3d4.log.gz | grep "REJECT"

# Decompress and view the file page by page
gunzip -c 123456789012_vpcflowlogs_us-east-1_fl-12345abc_20251018T1500Z_a1b2c3d4.log.gz | less

Using gunzip -c (which decompresses to standard output) piped to tools like grep, awk, or less allows you to analyze the log data without saving the large, uncompressed file to your disk.

Conclusion

The AWS CLI provides a fast and flexible way to access your VPC flow logs. For quick, real-time checks and powerful filtering, the aws logs commands for CloudWatch are ideal. For bulk retrieval and deep, historical analysis, the aws s3 commands give you direct access to the raw log files. Mastering both methods will significantly speed up your network troubleshooting and security analysis workflows.