Tech Stack

  • Lambda function with Python and Boto3 library
  • DynamoDB to store key metadata about instances and AMIs
  • DynamoDB Streams to capture event changes on tha table
  • Cloudwatch to trigger the automation at X times
  • Terraform to deploy the whole infrastructure (install if you don’t have it)

Goal

How many times have you lost valuable data without an AMI?

In the past, I used to struggled with getting the latest back up data from an EC2 instance since I kept forgetting to take AMIs.

It is often recommended to back up your instance data as often as possible in case of any failure.


As for many others, 2020 was a year of many personal and professional changes. The beginning of last year, my H1B work visa stamping was denied. Soon after I got laid off for the same immigration issues. This led to saying good bye to many great relationships. Then, when I thought everything was starting to work out, I was denied a student visa to continue my Masters.

It’s been a very challenging time where I’ve had to overcome many obstacles because of my origins, Venezuela🇻🇪. Although there might be few benefits to come from such a poor South American country…


Before Reading

Before diving deeper into how this script is structured, we need to understand the fundamental steps in this process:

1) Taking a snapshot

2) Creating an encrypted volume from the snapshot

3) Removing the unencrypted volumes

These will help us look at the big picture as we progress through the article. So even though it might feel complex at some point, you have to remember these fundamental steps!

All the resources are provisioned using Terraform. If you don’t have it, don’t worry. I’ll leave some references below to installed it. …


Before Reading

  • This function will only run successfully only if an EBS volume is attached right after created.
  • At the moment, this is just working for the general purpose instances where their block device is XVD, not NVME block devices.
  • As always, there needs to be a set of IAM permissions for services to interact among each other.

Tech Stack

Lambda: Python — Boto3

SSM: Shell Scripting

CloudWatch: Events

Goal

One of the biggest motivation behind developing this function was to automate the process for which a volume is made available in an EC2 instance.

Usually, this is somewhat a process that requires some Linux…


Before Reading

  • CloudWatch agents for monitoring disk and memory need to be installed
  • At the moment, this automation only works with XFS file system volumes
  • This automation only works up to the Partition level — not Logical yet
  • /etc/fstab needs to have a specific format for the script to run successfully
  • IAM permissions to execute service interactions are really important

Tech Stack

Scripting languages: Shell Script, Python

Libraries and tools: Jq (shell), Boto3 (python)

Infrastructure used: Lambda Functions, SSM Documents (more details below)

Goal

The main purpose of this automation is to enable EBS volume expansion without worrying about the underlaying tasks that need to…


Before Reading

— SES — Simple Email Service: Email service
— DynamoDB: NoSQL database service

Tech Stack

This automation tool is a series of two blogs. This is the second part which will use the following services:

— DynamoDB — Terraform
— SES
— Lambda — Boto3
— CloudWatch Events

I am not going to go into much detail of how DynamoDB/Terraform works since it is out of the scope of this post. However, I am going to provide some links in the reference section where you can be instructed more about these if you’d like. I will leave a subdirectory with all the…


Before Reading

— IAM — Identity Access Management: User management service
— DynamoDB: NoSQL database service

Tech Stack

This automation tool is a series of two blogs. This is the first part which will use the following services:

— IAM
— DynamoDB — Terraform
— Lambda — Boto3
— CloudWatch Events

I am not going to go into much detail of how DynamoDB/Terraform works since it is out of the scope of this post. However, I am going to provide some links in the reference section where you can be instructed more about these if you’d like. I will leave a subdirectory with all…


Before reading

  • Lambda functions need IAM permissions to interact with other services.
  • Best practice is to allow only the permissions necessary for functions to work with other services.

Tech Stack

Similar to the previous automation articles, the tech stack to accomplish this tool is the following:

  • AWS Lambda
  • Python using Boto3
  • AWS CloudWatch

Once again, the purpose of this article is not to give a thorough explanation from these services, but rather to provide their applied use cases. The main functions that will be used from Boto3 are:

ec2.describe_instances(),
ec2.start_instances(),
ec2.stop_instances()

Goal

One of the best ways to minimize cost in AWS is by turning…


Before Reading

Best practice to create security groups

  • Reference security group IDs instead of individual IPs (State of the art)
  • If there’s a need for IPs, be as specific as possible, e.g. IPs with /32 range

Tech Stacks

Similar to the previous automation articles, the tech stack to accomplish this tool is the following:

  • AWS Lambda
  • Python using Boto3
  • AWS S3

Again, I will try to steer away from the grainy details since it is not the scope of the article.There are two major resources which need to be handled in this automation piece, EC2 and S3. …


Before reading

  • EBS — Elastic Block Storage
  • DR — Disaster Recovery
  • This is not considering encrypted snapshots

Tech Stack

Following this thread of automation articles, the tech stack to accomplish this tool is the following:

  • AWS Lambda
  • Python using Boto3
  • AWS CloudWatch Events

I will not go into a lot of grainy details about each of these services since it is not the scope of the article. …

Ed Reinoso

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store