Nahuel Hernandez

Nahuel Hernandez

Another personal blog about IT, Automation, Cloud, DevOps and Stuff.

Automated AWS Resource Cleanup with aws-nuke

Learn how to automate AWS resource cleanup using aws-nuke in an AWS Lambda function. This comprehensive guide covers everything from setting up resource filters to prevent accidental deletions, configuring AWS profiles, and implementing weekly cleanup schedules. Discover how to protect critical resources while efficiently managing development and testing environments. Includes detailed examples for filtering EC2 instances, S3 buckets, CloudFront distributions, and more, along with cost-effective implementation strategies and safety measures to prevent self-destruction.

5-Minute Read

aws_nuke

A comprehensive guide on how to automate AWS resource cleanup using aws-nuke in an AWS Lambda function, ensuring your AWS accounts stay clean while preventing the cleanup task from deleting itself.

Table of contents

Introduction

Managing AWS resources can become overwhelming, especially in development and testing environments. aws-nuke is a powerful tool that helps you clean up AWS accounts by removing all resources. In this guide, we’ll show you how to automate this process using AWS Lambda, which is more cost-effective than running it in ECS Fargate.

What is aws-nuke?

aws-nuke is a tool that deletes all resources in an AWS account. It’s particularly useful for:

  • Development environments
  • Testing accounts
  • Sandbox environments
  • CI/CD cleanup

Setup Process

1. Create aws-nuke Configuration

First, create a configuration file (nuke-config.yml) with comprehensive resource filtering:

regions:
  - "us-west-2"  # Add your regions
  - "us-east-1"  # For CloudFront distributions

account-blocklist:
  - "1234567890"  # Add accounts to never clean

accounts:
  0987654321:  # Your account ID
    filters:
      # Prevent self-deletion
      LambdaFunction:
        - type: glob
          value: "aws-nuke-*"
      IAMRole:
        - type: glob
          value: "aws-nuke-*"
      IAMUser:
        - type: glob
          value: "aws-nuke-*"
      IAMPolicy:
        - type: glob
          value: "aws-nuke-*"
      
      # Protect specific resources
      EC2Instance:
        - type: glob
          value: "prod-*"  # Protect production instances
        - type: glob
          value: "bastion-*"  # Protect bastion hosts
      
      EC2KeyPair:
        - type: glob
          value: "prod-*"  # Protect production key pairs
      
      EC2SecurityGroup:
        - type: glob
          value: "default"  # Protect default security group
        - type: glob
          value: "prod-*"  # Protect production security groups
      
      EC2VPC:
        - type: glob
          value: "vpc-*prod*"  # Protect production VPCs
      
      S3Bucket:
        - type: glob
          value: "prod-*"  # Protect production buckets
        - type: glob
          value: "*-backup"  # Protect backup buckets
        - type: glob
          value: "*-logs"  # Protect log buckets
      
      CloudFrontDistribution:
        - type: glob
          value: "prod-*"  # Protect production distributions
      
      RDSInstance:
        - type: glob
          value: "prod-*"  # Protect production databases
      
      ElastiCacheCluster:
        - type: glob
          value: "prod-*"  # Protect production cache clusters
      
      Route53HostedZone:
        - type: glob
          value: "*.prod.*"  # Protect production domains
      
      # Protect specific IAM roles and users
      IAMRole:
        - type: glob
          value: "aws-nuke-*"
        - type: glob
          value: "prod-*"
        - type: glob
          value: "admin-*"
      
      IAMUser:
        - type: glob
          value: "aws-nuke-*"
        - type: glob
          value: "prod-*"
        - type: glob
          value: "admin-*"
      
      # Protect specific Lambda functions
      LambdaFunction:
        - type: glob
          value: "aws-nuke-*"
        - type: glob
          value: "prod-*"
        - type: glob
          value: "critical-*"

2. Create Lambda Function

Create a Lambda function using a custom container:

FROM public.ecr.aws/lambda/provided:al2

# Download aws-nuke
RUN curl -L https://github.com/rebuy-de/aws-nuke/releases/download/v2.25.0/aws-nuke-v2.25.0-linux-amd64.tar.gz | tar -xz

# Copy configuration
COPY nuke-config.yml /var/task/

# Set the CMD to your handler
CMD [ "aws-nuke-v2.25.0-linux-amd64/aws-nuke-v2.25.0-linux-amd64", "--config", "/var/task/nuke-config.yml", "--no-dry-run", "--quiet" ]

3. Configure AWS Profiles

Create AWS profiles for the Lambda function:

# Create AWS credentials file
mkdir -p ~/.aws
cat > ~/.aws/credentials << EOF
[aws-nuke-profile]
aws_access_key_id = YOUR_ACCESS_KEY
aws_secret_access_key = YOUR_SECRET_KEY
region = us-west-2
EOF

# Create AWS config file
cat > ~/.aws/config << EOF
[profile aws-nuke-profile]
region = us-west-2
output = json
EOF

4. Deploy Lambda Function

# Build and push container
aws ecr create-repository --repository-name aws-nuke
docker build -t aws-nuke .
docker tag aws-nuke:latest $AWS_ACCOUNT_ID.dkr.ecr.$AWS_REGION.amazonaws.com/aws-nuke:latest
docker push $AWS_ACCOUNT_ID.dkr.ecr.$AWS_REGION.amazonaws.com/aws-nuke:latest

# Create Lambda function
aws lambda create-function \
  --function-name aws-nuke \
  --package-type Image \
  --code ImageUri=$AWS_ACCOUNT_ID.dkr.ecr.$AWS_REGION.amazonaws.com/aws-nuke:latest \
  --role arn:aws:iam::$AWS_ACCOUNT_ID:role/aws-nuke-lambda-role \
  --timeout 900 \
  --memory-size 512 \
  --environment "Variables={AWS_PROFILE=aws-nuke-profile}" \
  --publish

5. Set Up EventBridge Rule

Create an EventBridge rule to trigger the Lambda weekly:

{
  "Name": "aws-nuke-weekly",
  "ScheduleExpression": "cron(0 0 ? * SUN *)",  // Run every Sunday at midnight
  "State": "ENABLED",
  "Targets": [
    {
      "Id": "aws-nuke-lambda",
      "Arn": "arn:aws:lambda:us-west-2:YOUR_ACCOUNT_ID:function:aws-nuke"
    }
  ]
}

Important Considerations

Lambda Timeout

  • Lambda functions have a maximum execution time of 15 minutes (900 seconds)
  • Configure aws-nuke to handle large accounts efficiently: 
    # Add to nuke-config.yml
resource-types:
  - "EC2Instance"
  - "EC2SecurityGroup"
  - "S3Bucket"
  # Add other resource types as needed

AWS Profiles

  • Use specific AWS profiles for the Lambda function
  • Exclude these profiles in the nuke configuration
  • Store credentials securely in AWS Secrets Manager

Resource Deletion Order

  • Some resources depend on others
  • aws-nuke handles dependencies automatically
  • Consider adding custom deletion order if needed

Safety Measures

  1. Resource Filtering

    • Use filters to protect critical resources
    • Exclude Lambda function and related resources
    • Add specific resources to the blocklist
    • Use glob patterns for flexible matching

  2. IAM Permissions

    • Use least privilege principle
    • Create specific role for Lambda
    • Implement resource-based policies
    • Regular permission audits

  3. Monitoring

    • Set up CloudWatch alarms
    • Monitor function execution
    • Review logs regularly
    • Set up notifications for failures

  4. Backup Configuration

    • Store configuration in Lambda package
    • Version control configuration files
    • Regular configuration reviews
    • Document all exclusions

Cost Analysis

Lambda Costs

  • Memory: 512MB
  • Duration: ~5 minutes (300 seconds)
  • Weekly execution: 4 times per month
  • Cost per execution: ~$0.0001 (300 seconds * $0.0000166667 per second)
  • Monthly cost: ~$0.0004

ECS Fargate Costs (for comparison)

  • CPU: 0.25 vCPU
  • Memory: 0.5GB
  • Duration: 24/7 (even when not running)
  • Monthly cost: ~$30-40

Conclusion

Automating AWS resource cleanup with aws-nuke and Lambda provides a cost-effective way to maintain clean AWS accounts. By implementing proper safety measures and monitoring, you can ensure the cleanup process runs safely and reliably.

Remember to:

  • Regularly review and update filters
  • Monitor function execution
  • Keep configurations in version control
  • Test in non-production environments first
  • Document all protected resources
  • Review costs regularly

This setup provides a robust and cost-effective solution for automated AWS resource cleanup while ensuring the cleanup process doesn’t delete itself.

Reference

Categories

Recent Posts

About

Over 15-year experience in the IT industry. Working in SysOps, DevOps and Architecture roles with mission-critical systems across a wide range of industries. Wide experience with AWS, Terraform, Kubernetes, Containers, CI/CD pipelines, and Linux. Always keeping up with the latest technologies. Passionate about automating the run of the mill. Big focus on problem-solving.