5. December 2017 21:38
by Aaron Medacco
0 Comments

AWS re:Invent 2017 - Day 3 Experience

5. December 2017 21:38 by Aaron Medacco | 0 Comments

The following is my Day 3 re:Invent 2017 experience. Missed Day 2? Check it out here.

Out and early at 8:30 (holy ****!) and was famished. Decided to check out the Buffet at the Bellagio. Maybe it was just me, but I was expecting a little bit more from this. Most things in Las Vegas are extravagant and contribute to an overall spectacle, but the Bellagio Buffet made me think of Golden Corral a bit. Maybe it gets better after breakfast, I don't know. 

AWS re:Invent 2017

AWS re:Invent 2017

The plate of an uncultured white bread American.

Food was pretty good, but I didn't grab anything hard to mess up. Second trip back, grabbed some watermelon that tasted like apple crisp and ice cream. Not sure what that was about. Maybe the staff used the same knife for desserts and fruit slicing. From what I could tell, half the patrons were re:Invent attendees, either wearing their badge or the hoodie.

Walked back to my room to watch the Keynote by Andy Jassy, but only caught the last bit of it. After some difficulty getting the live stream to work on my laptop, watched him announce the machine learning and internet of things services. Those aren't really my wheelhouse (yet?), but seemed interesting nontheless. Succumbed to a food coma afterwards for a short nap.

Headed over to the Venetian to go back to the Expo for a new hoodie and for my next breakout session. AWS was holding the merchandise hostage if you didn't fill out evaluations for breakout sessions, so I couldn't get the hoodie until after I came back post-session. Good to know for next year. Back where the session halls were, got in the Reserved line for the Optimizing EC2 for Fun and Profit #bigsavings #newfeatures (CMP202) session. Talked with a gentleman while in line about the new announcements, specifically the S3 Select and Glacier Select features. I wasn't clear what the difference was between S3 Select and Athena and neither was he. I'll have to go try it out for myself.

AWS re:Invent 2017

Awaiting new feature announcements.

AWS re:Invent 2017

Great speaker as always. AWS always has good speakers.

AWS re:Invent 2017

More talk about Reserved and Spot Instances.

Best thing about this session was the announcements of new features. The first one was a really helpful feature AWS added to the Cost Explorer within the management console that gives instance recommendations based on your account's historical usage. Having a tool like this that does cost analysis and recommendations is great, means I don't have to. I pulled up the SDK and AWS CLI reference while he was demonstrating it, but couldn't find any methods where I could pull those recommendations using Lambda or a batch script. I figured it'd be useful to automate a monthly email or something that tells you that month's instance billing recommendations. Ended up talking to the speaker afterwards who said it's not available, but will be in the months to come. Nice!

Second announcement was regarding Spot Instances and being able to hibernate instances when they're going to be terminated. The way this was described was that hibernation acts the same way as when you "open and close your laptop". So if you are using a Spot Instance set to hibernate, when that instance gets terminated in the event another customer bids higher or AWS adds it back to the On-Demand instance pool, it will save state to EBS and when you receive it back, it will pick up where it left off instead of needing to completely re-initialize before doing whatever work you wanted. 

T2 Unlimited was also covered, which essentially allows you to not worry so much about requiring the credits for burst capacity of your T2 series of EC2 instances. The rest of the session covered a lot of cost optimization techniques that have been labored to death. Use Reserved Instances, use Spot Instances, choose the correct instance type for your workload, periodically check in to make sure you actually need the capacity you've provisioned, take advantage of serverless for cases where an always-on machine isn't necessary, and other tips of the "don't be an idiot" variety. Again, I must be biased since most of this information seems elementary. I think next year I need to stick to the 400-level courses to get the most value. That said, the presentation was excellent like always. I won't knock it just because I knew information coming in.

Found the shuttle a short walk from the hall, and decided to be lazy (smart) for once. Got back to the Bellagio for some poker before dinner, and came out plus $105. During all the walks back from the Aria to the Bellagio, I kept eyeballing the Gordon Ramsay Burger across the street at the Planet Hollywood, so I stopped in for dinner. 

AWS re:Invent 2017

Pretty flashy for a burger place.

AWS re:Invent 2017

I ate it all...No, I didn't. But wanted to try out the dogs and the burgers.

For a burger and hot dog place, I'd give it a 7/10. Probably would be a bit higher if they had dill pickles / relish, and honestly, better service. You can imagine this was pretty messy to eat, especially the hot dog, so I asked one of the girls upfront where the bathroom was to go wash my hands. The one across the hall was out of order (go figure), so I had to go to the one out thru some of the casino and next to P.F. Changs. I think the tables next to me thought I just walked out without paying. Heard them say "There he is." when I returned. Really? Do I look like a criminal? Yeah, I came to Vegas for a full week to rip off a burger joint.

Cheers!

3. December 2017 18:42
by Aaron Medacco
0 Comments

AWS re:Invent 2017 - Day 2 Experience

3. December 2017 18:42 by Aaron Medacco | 0 Comments

The following is my Day 2 re:Invent 2017 experience. Missed Day 1? Check it out here.

Day 2, I started off the day waking up around 11:30am. Needed the rest from the all-nighter drive Monday morning. Cleaning lady actually woke me up when she was making her rounds on the 4th floor. This meant that I missed the breakout session I reserved, Deploying Business Analytics at Enterprise Scale with Amazon QuickSight (ABD311). I'm currently in the process of recording a Pluralsight course on Amazon QuickSight, so I felt this information could be helpful as I wrap up that course. Guess I'll have to check it out later once the sessions are uploaded to YouTube. Just another reason to never drive through the night before re:Invent again.

After getting ready, I exposed my nerd skin to sunlight and walked over to the Venetian. This is where I'd be the majority of the day. I kind of lucked out because all of my sessions for the day besides the one I slept over were in the same hotel, and pretty back-to-back so I didn't have to get creative with my downtime. 

First session of the day was Deep Dive into the New Network Load Balancer (NET304). I was curious about this since the network load balancer's announcement recently, but never had a use case or a reason to go and implement one myself. 

AWS re:Invent 2017

I have to admit, I didn't know it could route to IP addresses.

AWS re:Invent 2017

Should have picked a better seat.

AWS re:Invent 2017

Putting it all together.

The takeaways I got was that the NLB is essentially your go-to option for TCP traffic at scale, but for web applications you'd still be mostly using the Application Load Balancer or the Classic Load Balancer. The 25% cheaper than ALB fact seems significant and it uses the same kinds of components used by ALB like targets, target groups, and listeners. Additionally, it supports routing to ECS, EC2, and external IPs, as well as allowing for static IPs per availability zone.

As I was walking out of the session, there was a massive line hugging the wall of the hall and around the corner for the next session which I had a reservation seat for (thank god). That session was Running Lean Architectures: How to Optimize for Cost Efficiency (ARC303). Nerds would have to squeeze in and cuddle for this one, this session was full. 

AWS re:Invent 2017

Before the madness.

AWS re:Invent 2017

Wasn't totally full, but pretty full.

AWS re:Invent 2017

People still filing in.

AWS re:Invent 2017

Obvious, but relevant slide.

I had some mixed feelings about this session, but thought it was overall solid. On one hand, much of the information was definitely important for AWS users to save money on their monthly bill, but at the same time, I felt a lot of it was fairly obvious to anyone using AWS. For instance, I have to imagine everybody knows they should be using Reserved Instances. I feel like any potential or current user of AWS would have read about pricing thoroughly before even considering moving to Amazon Web Services as a platform, but perhaps I'm biased. There were a fair number of managers in the session and at re:Invent in general, so maybe they're not aware of obvious ways to save money. 

Aside from covering Spot and Reserved Instance use cases, there was some time covering Convertable Reserved Instances, which is still fairly new. I did enjoy the tips and tricks given on reducing Lambda costs by looking for ways to cut down on function idle time, using Step Functions instead of manual sleep calls, and migrating smaller applications into ECS instead of running each application on their own instance. The Lambda example highlighted that many customer functions involve several calls to APIs which occur sequentially where each call waits on the prior to finish. This can rack up billed run time even though Lambda isn't actually performing work during those waiting periods. The trick they suggested was essentially to shotgun the requests all at once, instead of one-by-one, but as I thought about it, that would only work if those API calls were such that they didn't depend on the result of the prior. When they brought up Step Functions it was kind of obvious you could just use that service if that was the case, though.

The presenters did a great job, and kept highlighting the need to move to a "cattle" mentality instead of "pet" mentality when thinking about your cloud infrastructure. Essentially, they encouraged moving away from manual pushing, RDPing, naming instances thinks like "Smeagle" and the like. Honestly, a lot of no-brainers and elementary information but still a  good session.

Had some downtime after to go get something to eat. Grabbed the Baked Rigatoni from the Grand Lux Cafe in the Venetian. The woman who directed me to it probably thought I was crazy. I was in a bit of a rush, and basically attacked her with, "OMG, where is food!? Help me!"

AWS re:Invent 2017

Overall, 7/10. Wasn't very expensive and now that I think about it, my first actual meal (some cold pizza slices don't count) since getting to Vegas. 

Next up was ElastiCache Deep Dive: Best Practices and Usage Patterns (DAT305) inside the Venetian Theatre. I was excited about this session since I haven't done much of anything with ElastiCache in practice, but I know some projects running on AWS that leverage it heavily. 

AWS re:Invent 2017

About 10 minutes before getting mind-pwned.

AWS re:Invent 2017

Random guy playing some Hearthstone while waiting for the session to begin.

AWS re:Invent 2017

Sick seat.

Definitely felt a little bit out of my depth on this one. I'm not someone who is familiar with Redis, outside of just knowing what it does so I was clueless during some of the session. I didn't take notes, but I recall a lot of talk about re-sharding clusters, the old backup and recovery method vs. the new online managed re-sharding available, pros of enabling cluster mode (was clueless, made sense at the time, but couldn't explain it to someone else), security, and best practices. My favorite part of the session involved use cases and how ElastiCache can benefit: IoT, gaming apps, chat apps, rate limiting services, big data, geolocation or recommendation apps, and Ad Tech. Again, I was out of my depth, but I'll be taking a closer look at this service during 2018 to fix that.

After some more Frogger in the hallways, headed over to the Expo, grabbed some swag and walked around all the vendor booths. Food and drinks were provided by AWS and the place was a lot bigger than I expected. There's a similar event occurring at the Aria (the Quad), which I'll check out later in the week. 

AWS re:Invent 2017

Wall in front of the Expo by Registration.

There were AWS experts and team members involved with just about everything AWS scattered around to answer attendee questions which I though was freaking awesome. Valuable face-time with the actual people that work and know about the stuff being developed.

AWS re:Invent 2017

Favorite section of the Venetian Expo.

AWS re:Invent 2017

More madness.

Talked to the guys at the new PrivateLink booth to ask if QuickSight was getting an "endpoint" or a method for connecting private AWS databases soon. Ended up getting the de facto answer from the Analytics booth who had Quinton Alsbury at it. Once I saw him there, I'm like "Oh here we go, this guy is the guy!" Apparently, the feature's been recently put in public preview, which I somehow missed. Visited a few other AWS booths like Trusted Advisor and the Partner Network one, and then walked around all the vendor booths for a bit.

AWS re:Invent 2017

Unfortunately, I didn't have much time to chit chat with a lot of them since the Expo was closing soon. I'll have to do so more at the Quad. Walked over to the interactive search map they had towards the sides of the room to look for a certain company I thought might be there. Sure enough, found something familiar:

AWS re:Invent 2017

A wild technology learning company appears.

Spoke with Dan Anderegg, whose the Curriculum Manager for AWS within Pluralsight. After some talk about Pluralsight path development, I finished my beer and got out, only to find I actually stayed too long and was already super late to my final session for the day, Deep Dive on Amazon Elastic Block Store (Amazon EBS) (STG306). Did I mention how it's hard to try to do everything you want to at re:Invent? 

Ended up walking home and wanting to just chill out, which is how this post is getting done. 

Cheers!

1. December 2017 17:26
by Aaron Medacco
0 Comments

AWS re:Invent 2017 - Day 1 Experience

1. December 2017 17:26 by Aaron Medacco | 0 Comments

Welcome to the first in a series of blog posts detailing my experience at AWS re:Invent 2017. If you're someone who is considering going to an AWS re:Invent conference, hopefully what follows will give you a flavor for what you can expect should you choose to fork over the cash for a ticket. The following content contains my personal impressions and experience, and may not (probably doesn't?) reflect the typical experience. Also, there will be some non-AWS fluff as well as I have not been to Las Vegas before.

AWS re:Invent 2017

My adventure starts at about Midnight. Yes, midnight. Living in Scottsdale, AZ, I figured, "Why not just drive instead of fly? After all, it's only a 6 hour drive and there won't be any traffic in the middle of the night." While that was true, what a mistake in retrospect. Arriving in Las Vegas with hardly any sleep after the road trip left me in pretty ragged shape for Monday's events. Next year, I'll definitely be flying and will get there on Sunday so I get can settle prior to Monday. I actually arrived so early, I couldn't check into my room and needed to burn some time. What better activity to do when exhausted than sit down at poker tables. Lost a quick $900 in short order. Hahaha! Truth be told, I got "coolered" back to back, but I probably played bad, too.

Once I got checked into my room at the Bellagio around 9:00am, I headed back to the Aria to get registered and pick up my re:Invent hoodie. Unfortunately, they didn't have my size, only had up to a Small. I couldn't help but smile about that. I ended up going to the Venetian later to exchange my Small for a Medium. Anyways, got my badge, ready to go! Or was I?

By the way, kudos to the Bellagio for putting these in every room. Forgot my phone charger. Well, the correct phone charger at least...

 AWS re:Invent 2017

...except it didn't have a charger compatible with my Samsung Galaxy S8. Kind of funny, but I wasn't laughing. Alright, maybe a little. Would end up getting one at a Phone store among one of the malls at the Strip. Oh yeah, and I also forgot to buy a memory card for my video recorder prior to leaving. Picked up one of those from a Best Buy Express vending machine. Vegas knows.

By this time I was crashing. Came back to my room, fell asleep, and missed 2 breakout sessions I was reserved for. Great job, Aaron! Off to a great start! 

Walked to the Aria to go check out the Certification Lounge. They had tables set up, food and drink, and some goodies available depending on what certifications you'd achieved. The registration badges have indicators on them that tell people if you're AWS certified or not, which they use to allow or deny access. I didn't end up staying too long, but there were a decent number of attendees with laptops open working and networking. Here's some of the things collected this year by walking around to the events: 

AWS re:Invent 2017

The re:Invent hoodie picked up at Registration (left) and the certification t-shirt inside the Certification Lounge (right).

AWS re:Invent 2017

Water bottle and AWS pins were given away at the Venetian Expo (top-left), badge and info packet at Registration (right), and the certification stickers at the Certification Lounge depending on which ones you've completed (bottom-left).

Headed over to the MGM Grand for my first breakout session, GPS: Anti Patterns: Learning From Failure (GPSTEC302). Before I discuss the session, I have to talk about something I severely underestimated about re:Invent. Walking! My body was definitely NOT ready. And I'm not an out-of-shape or big guy, either. The walking is legit! I remember tweeting about what I imagined would be my schedule weeks before re:Invent and Eric Hammond telling me I was being pretty optimistic about what I would actually be able to attend. No joke. Okay, enough of my complaining.

AWS re:Invent 2017

Waiting for things to get started.

AWS re:Invent 2017

Session about half-full. Plenty of room to get comfortable.

AWS re:Invent 2017

Presenter's shirt says, "got root?". Explaining methods for ensuring account resource compliance and using AWS account best practices when it comes to logging, backups, and fast reaction to nefarious changes.

This was an excellent session. The presenters were fantastic and poked fun at mistakes they themselves have made or those of customers they've talked to have made regarding automation (or lack thereof), compliance, and just overall bone-headedness (is that a word?). The big takeaways I found were to consider using services like CloudWatch, CloudTrail and Config to monitor and log activity in your AWS accounts to become aware when stupid raises it's ugly head. They threw out questions like, "What would happen if the root account's credentials were compromised and you didn't know about it until it was too late?", and "You have an automated process for creating backups, but do you actually test those backups?". From this came suggestions to regularly store and test backups to another account in case an account gets compromised and using things like MFA, especially for root and privileged users.

Additionally, the presenters made a good argument for not using the management console for activities once you become more familiar with AWS, particularly if you're leveraging the automation tools AWS provides like OpsWorks and CloudFormation as that kind of manual mucking around via the console can leave you in funny states for stacks deployed with those services. Along those lines, they also suggested dividing up the different tiers of your application infrastructure into their own stacks so that when you need to make changes to something or scale, you don't end up changing the whole system. Instead, you only modify or scale the relevant stack. Overall good session. If they have it again next year, I would recommend it. You'll get some laughs, if nothing else. The guys were pretty funny.

Once out, I had a meeting scheduled to talk with a company (presumably about upcoming Pluralsight work) at the Global Partner Summit Welcome Reception. Now, I'll admit I got a little frustrated trying to find where the **** this was taking place! AWS did a great job sending lots of guides with re:Invent flags everywhere to answer questions and direct attendees to their events, and these guys were godsends every time except when it came to finding this event. I think I just got unlucky with a few that were misinformed.

AWS re:Invent 2017

These guys were scattered all over the strip and inside the hotels. Very helpful!

First, I was told to go to the one of the ballrooms. Found what appeared to be some kind of Presenter's Registration there. Then, found another guide who said to go to the Garden Grand Arena. Walked over there, total graveyard, and ironically, a random dude there who wasn't even one of the re:Invent guides told me where it actually was. He also said, "Oh yeah, and unless you want to be standing in line all night, you might want to reconsider." It was late enough at this point, I figured I'd just head back to the Bellagio for a much needed poker session, so that's what I did. However, on the way back, holy ****, he was right. I've never seen a line as long as the one to get into the GPS Welcome Reception in my life. It went from the food court, through the entire casino, out of the casino, and further back to I couldn't tell where. Apparently, I was the only one who missed the memo, since everyone else knew where to go, but still, that line. 

Long hike back to the Bellagio, played poker for about 3 hours, lost $200 (man, I suck), and on my way back to my room discovered I didn't eat anything all day. LOL! Picked up a couple pizza slices and crashed for the night. A good night's sleep? Yes, please. Tomorrow would be better.

Cheers!

14. July 2017 03:22
by Aaron Medacco
0 Comments

Copying Multiple AMIs From One AWS Region to Another

14. July 2017 03:22 by Aaron Medacco | 0 Comments

It's common to want to move AMIs between regions to provision copies of instances to a new region. You can copy an image from one region to another in the web console but only if you select one at a time. At least this was the case for me. Therefore, I wrote the following batch file which will allow you to copy any amount of AMIs to another region.

This post assumes you have installed and configured the AWS CLI appropriately and that you have permission to invoke the copy-image action for EC2.

Creating a batch script and text file:

  1. Create a file named amis.txt and write the IDs of each AMI you want to copy on a new line and save it in your working directory. For example:
    ami-xxxxxxxx
    ami-xxxxxxxx
  2. Create a file named copy-amis.bat with the following contents and substitute the appropriate values for the source region your copying AMIs from and the destination region your copying AMIs to:
    SET source-region=us-east-1
    SET dest-region=us-east-2
    FOR /F "tokens=*" %%A IN (amis.txt) DO (
    	ECHO "Copying AMI %%A from %source-region% to %dest-region%
    	SET ami-name=%%A-copy-from-%source-region%
    	aws ec2 copy-image --source-image-id %%A --source-region %source-region% --region %dest-region% --name %ami-name%
    )
    ECHO "Finished copying AMIs from %source-region% to %dest-region%
  3. Run the batch file.

That's it. You should see copies of your images in the destination region shortly after.

Cheers!

3. July 2017 22:25
by Aaron Medacco
15 Comments

Scheduling Automated AMI Backups of Your EC2 Instances

3. July 2017 22:25 by Aaron Medacco | 15 Comments

When considering disaster recovery options for systems or applications running on Amazon Web Services, a frequent solution is to use AMIs to restore instances to a known acceptable state in the event of failure or catastrophe. If you're team has decided on this approach, you'll want to automate the creation and maintenance of these AMIs to prevent mistakes or somebody "forgetting" to do the task. In this post, I'll walk through how to set this up in AWS within a matter of minutes using Amazon's serverless compute offering, Lambda

Automated AMI Backups

In departure from many of my other posts involving Lambda, the following steps will make use of the AWS CLI so I will assume that you've already installed and configured it on your machine. This is to grant longevity to these posts and to protect their relevance from slipping due to the browser-based console's rate of change.

I have also added some flexibility in the maintenance of these backups I encourage the reader to configure to their liking. These include a variable number of backups the reader would like to maintain for each EC2 instance they wish to have AMIs taken, a customizable tag the reader can assign to EC2 instances they'd like to backup, and the option to also delete snapshots of the AMIs being de-registered once they exit the backup window. I have included default values, but I still encourage you to read the options before implementing this solution.

Let's get started.

Creating an IAM policy for access permissions:

  1. Create a file named iam-policy.json with the following contents and save it in your working directory:
    {
        "Version": "2012-10-17",
        "Statement": [
            {
                "Sid": "Stmt1499061014000",
                "Effect": "Allow",
                "Action": [
                    "ec2:CreateImage",
                    "ec2:CreateTags",
                    "ec2:DeleteSnapshot",
                    "ec2:DeregisterImage",
                    "ec2:DescribeImages",
                    "ec2:DescribeInstances"
                ],
                "Resource": [
                    "*"
                ]
            }
        ]
    }
  2. In your command prompt or terminal window, invoke the following command:
    aws iam create-policy --policy-name ami-backup-policy --policy-document file://iam-policy.json
  3. You'll receive output with details of the policy you've just created. Write down the ARN value as you will need it later.

Creating the IAM role for the Lambda function:

  1. Create a file named role-trust-policy.json with the following contents and save it in your working directory:
    {
      "Version": "2012-10-17",
      "Statement": [
        {
          "Effect": "Allow",
          "Principal": {
            "Service": "lambda.amazonaws.com"
          },
          "Action": "sts:AssumeRole"
        }
      ]
    }
  2. In your command prompt or terminal window, invoke the following command:
    aws iam create-role --role-name ami-backup-role --assume-role-policy-document file://role-trust-policy.json
  3. You'll receive output with details of the role you've just created. Be sure to write down the role ARN value provided. You'll need it later.
  4. Run the following command to attach the policy to the role. You must substitute ARN for the policy ARN you wrote down from the prior step: 
    aws iam attach-role-policy --policy-arn ARN --role-name ami-backup-role

Creating the Lambda function:

  1.  Create a file named index.js with the following contents and save it in your working directory:
    Note: The following file is the code managing your AMI backups. There are a number of configurable options to be aware of and I have commented descriptions of each in the code. 
    var AWS = require("aws-sdk");
    var ec2 = new AWS.EC2();
    
    var numBackupsToRetain = 2; // The Number Of AMI Backups You Wish To Retain For Each EC2 Instance.
    var instancesToBackupTagName = "BackupAMI"; // Tag Key Attached To Instances You Want AMI Backups Of. Tag Value Should Be Set To "Yes".
    var imageBackupTagName = "ScheduledAMIBackup"; // Tag Key Attached To AMIs Created By This Process. This Process Will Set Tag Value To "True".
    var imageBackupInstanceIdentifierTagName = "ScheduledAMIInstanceId"; // Tag Key Attached To AMIs Created By This Process. This Process Will Set Tag Value To The Instance ID.
    var deleteSnaphots = true; // True if you want to delete snapshots during cleanup. False if you want to only delete AMI, and leave snapshots intact.
    
    exports.handler = function(event, context) {
        var describeInstancesParams = {
            DryRun: false,
            Filters: [{
                Name: "tag:" + instancesToBackupTagName,
                Values: ["Yes"]
            }]
        };
        ec2.describeInstances(describeInstancesParams, function(err, data) {
            if (err) {
                console.log("Failure retrieving instances.");
                console.log(err, err.stack); 
            }
            else {
                for (var i = 0; i < data.Reservations.length; i++) {
                    for (var j = 0; j < data.Reservations[i].Instances.length; j++) {
                        var instanceId = data.Reservations[i].Instances[j].InstanceId;
                        createImage(instanceId);
                    }
                }
            }
        });
        cleanupOldBackups();
    };
    
    var createImage = function(instanceId) {
        console.log("Found Instance: " + instanceId);
        var createImageParams = {
            InstanceId: instanceId,
            Name: "AMI Scheduled Backup I(" + instanceId + ") T(" + new Date().getTime() + ")",
            Description: "AMI Scheduled Backup for Instance (" + instanceId + ")",
            NoReboot: true,
            DryRun: false
        };
        ec2.createImage(createImageParams, function(err, data) {
            if (err) {
                console.log("Failure creating image request for Instance: " + instanceId);
                console.log(err, err.stack);
            }
            else {
                var imageId = data.ImageId;
                console.log("Success creating image request for Instance: " + instanceId + ". Image: " + imageId);
                var createTagsParams = {
                    Resources: [imageId],
                    Tags: [{
                        Key: "Name",
                        Value: "AMI Backup I(" + instanceId + ")"
                    },
                    {
                        Key: imageBackupTagName,
                        Value: "True"
                    },
                    {
                        Key: imageBackupInstanceIdentifierTagName,
                        Value: instanceId
                    }]
                };
                ec2.createTags(createTagsParams, function(err, data) {
                    if (err) {
                        console.log("Failure tagging Image: " + imageId);
                        console.log(err, err.stack);
                    }
                    else {
                        console.log("Success tagging Image: " + imageId);
                    }
                });
            }
        });
    };
    
    var cleanupOldBackups = function() {
        var describeImagesParams = {
            DryRun: false,
            Filters: [{
                Name: "tag:" + imageBackupTagName,
                Values: ["True"]
            }]
        };
        ec2.describeImages(describeImagesParams, function(err, data) {
            if (err) {
                console.log("Failure retrieving images for deletion.");
                console.log(err, err.stack); 
            }
            else {
                var images = data.Images;
                var instanceDictionary = {};
                var instances = [];
                for (var i = 0; i < images.length; i++) {
                    var currentImage = images[i];
                    for (var j = 0; j < currentImage.Tags.length; j++) {
                        var currentTag = currentImage.Tags[j];
                        if (currentTag.Key === imageBackupInstanceIdentifierTagName) {
                            var instanceId = currentTag.Value;
                            if (instanceDictionary[instanceId] === null || instanceDictionary[instanceId] === undefined) {
                                instanceDictionary[instanceId] = [];
                                instances.push(instanceId);
                            }
                            instanceDictionary[instanceId].push({
                                ImageId: currentImage.ImageId,
                                CreationDate: currentImage.CreationDate,
                                BlockDeviceMappings: currentImage.BlockDeviceMappings
                            });
                            break;
                        }
                    }
                }
                for (var t = 0; t < instances.length; t++) {
                    var imageInstanceId = instances[t];
                    var instanceImages = instanceDictionary[imageInstanceId];
                    if (instanceImages.length > numBackupsToRetain) {
                        instanceImages.sort(function (a, b) {
                           return new Date(b.CreationDate) - new Date(a.CreationDate); 
                        });
                        for (var k = numBackupsToRetain; k < instanceImages.length; k++) {
                            var imageId = instanceImages[k].ImageId;
                            var creationDate = instanceImages[k].CreationDate;
                            var blockDeviceMappings = instanceImages[k].BlockDeviceMappings;
                            deregisterImage(imageId, creationDate, blockDeviceMappings);
                        }   
                    }
                    else {
                        console.log("AMI Backup Cleanup not required for Instance: " + imageInstanceId + ". Not enough backups in window yet.");
                    }
                }
            }
        });
    };
    
    var deregisterImage = function(imageId, creationDate, blockDeviceMappings) {
        console.log("Found Image: " + imageId + ". Creation Date: " + creationDate);
        var deregisterImageParams = {
            DryRun: false,
            ImageId: imageId
        };
        console.log("Deregistering Image: " + imageId + ". Creation Date: " + creationDate);
        ec2.deregisterImage(deregisterImageParams, function(err, data) {
           if (err) {
               console.log("Failure deregistering image.");
               console.log(err, err.stack);
           } 
           else {
               console.log("Success deregistering image.");
               if (deleteSnaphots) {
                    for (var p = 0; p < blockDeviceMappings.length; p++) {
                       var snapshotId = blockDeviceMappings[p].Ebs.SnapshotId;
                       if (snapshotId) {
                           deleteSnapshot(snapshotId);
                       }
                   }    
               }
           }
        });
    };
    
    var deleteSnapshot = function(snapshotId) {
        var deleteSnapshotParams = {
            DryRun: false,
            SnapshotId: snapshotId
        };
        ec2.deleteSnapshot(deleteSnapshotParams, function(err, data) {
            if (err) {
                console.log("Failure deleting snapshot. Snapshot: " + snapshotId + ".");
                console.log(err, err.stack);
            }
            else {
                console.log("Success deleting snapshot. Snapshot: " + snapshotId + ".");
            }
        })
    };
  2. Zip this file to a zip called index.zip.
  3. In your command prompt or terminal window, invoke the following command. You must substitute ARN for the role ARN you wrote down from the prior step: 
    aws lambda create-function --function-name ami-backup-function --runtime nodejs6.10 --handler index.handler --role ARN --zip-file fileb://index.zip --timeout 30
  4. You'll receive output details about the Lambda function you've just created. Write down the Function ARN value for later use.

Scheduling the Lambda function:

  1. In your command prompt or terminal window, invoke the following command:
    Note: Feel free to adjust the schedule expression for your own use.
    aws events put-rule --name ami-backup-event-rule --schedule-expression "rate(1 day)"
  2. You'll get the Rule ARN value back as output. Write this down for later.
  3. Run the following command. Substitute ARN for the Rule ARN you just wrote down:
    aws lambda add-permission --function-name ami-backup-function --statement-id LambdaPermission --action "lambda:InvokeFunction" --principal events.amazonaws.com --source-arn ARN
  4. Run the following command. Substitute ARN for the Function ARN of the Lambda function you wrote down:
    aws events put-targets --rule ami-backup-event-rule --targets "Id"="1","Arn"="ARN"

Remember, you must assign the appropriate tag to each EC2 instance you want windowed AMI backups for. Leave a comment if you run into any issue using this solution.

Cheers!

Copyright © 2016-2017 Aaron Medacco