7. December 2017 02:49
by Aaron Medacco
0 Comments

AWS re:Invent 2017 - Day 4 Experience

7. December 2017 02:49 by Aaron Medacco | 0 Comments

The following is my Day 4 re:Invent 2017 experience. Missed Day 3? Check it out here.

The common theme of not waking up early continues. Missed the entire Werner Vogels Keynote, which is fine since from what others were saying it was a storm to get in. I didn't have it reserved anyways. AWS added it to the event catalog after I had already decided my schedule. Not sure why I thought you were just supposed to show up. 

First session of the day, AWS Database and Analytics State of the Union - 2017 (DAT201). This took place in the Venetian Theatre.

AWS re:Invent 2017

Love this venue for sessions.

I wasn't sure what to expect from a "State of the Union" session. For the most part, this was a combination of sales pitch, history lesson into the origin of some of the AWS database services, and explanation of miscellaneous features. After the explanation of what the RDS Multi-AZ feature does (really? who doesn't know what this is?), the session moved on to highlight the motivations for building Aurora and DynamoDB. Essentially, AWS wanted to combine the benefits of commercial-grade performance provided by products like Oracle and SQL Server with the low-cost of MySQL, PostgreSQL and MariaDB. The product of these efforts became the Aurora product. 

AWS re:Invent 2017

Horrible quality pic. I don't know why.

After sharing some history, a few of the newest Aurora features came up. Specifically, Amazon Aurora Multi-Master and Amazon Aurora Serverless. Amazon Aurora Multi-Master allows you bring the disaster recovery of a failed master instance to almost nothing, 100 ms. The single-region version of this feature is available in preview now, with the multi-region version available later in 2018. Amazon Aurora Serverless allows you to essentially have an on-demand database that scales for you and is for applications that have unpredictable workloads. Being serverless, the customer manages hardly anything.

The origins for DynamoDB came from a disaster affecting Amazon.com. I didn't write down the exact year, but basically Amazon.com was leveraging Oracle to power the online retail site, however the site became unavailable during Christmas. The cause of this was traced back to limitations in Oracle or Amazon.com's implementation of it, of this I wasn't clear. In response, Amazon built the NoSQL database service, DynamoDB, to handle massive volumes of transactions it was experiencing.

AWS re:Invent 2017

DynamoDB. But does it work at scale? Haha.

The remainder of the session focused on general overviews of the rest of the data service catalog. Therefore, a lot of the session was review for anyone who regularly spends time learning Amazon Web Services. Several attendees started leaving during this time. There's a lot going on during re:Invent so while I understand time is precious during the week, I always stay for the whole session. Maybe it's just out of respect for the presenter. Either way, I did learn a few tidbits about Aurora and some origin history of some services I didn't know before.

AWS re:Invent 2017

Big Amazon Echo.

Grabbed a shuttle to the Aria to check out what was going on at the Quad. I was expecting it to be as large as the Venetian Expo. Boy, was that off the mark! The Quad was very underwhelming, especially after visiting the Expo. There was a small handful of booths, but nothing like the Expo which was aisles and aisles full of them. I smiled when I saw the Certification Lounge there, looked like a large-sized cubicle. At this point, it became clear to me that the Venetian was definitely the primary hub for re:Invent. Good to know for next year's hotel room reservations.

They did have a cool Lego section, though.

During my video filming of the re:Invent areas of the Aria, I got yelled at by some woman telling me I couldn't record. I waited until she stopped looking, and turned it back on after. What a joke! Now, I actually was told not to video record prior to this while in the Venetian casino, which while I'd argue is unenforceable, makes more sense to me. However, back in the Aria, what is there to film that can cause harm!? It's a bunch of nerds with laptops walking around banners, session rooms, and re:Invent help desks a quarter mile from the casino floor! Ridiculous. It's 2017, and there's a tech conference going on here. Are you going to watch every attendee's laptop, tablet and smartphone, too, because guess what, those devices can do video recording as well. Unenforceable and moronic. Anyways...

AWS re:Invent 2017

In case the re:Invent app fails you.

Returned to my room after grabbing a Starbucks and did some blogging until my next session at the Venetian, Taking DevOps Closer to the AWS Edge (CTD401). 

AWS re:Invent 2017

Emptier session. Was actually really good.

This session was possible my favorite of the conference. And I certainly wasn't expecting that. The session title, in my opinion, is misleading, however. Now, the presenter did say that the same session took place last year and included a demo involving saving CloudFormation templates to CodeCommit and managing a delivery pipeline with CodePipeline to push modifications to a development CloudFront distribution, perform some testing, and then do the same to a production CloudFront distribution. That seems more DevOps to me. What we got instead was an in-depth overview of how to incorporate the edge services of AWS into application design and development and how to use CloudFront

AWS re:Invent 2017

More terrible quality pics.

AWS re:Invent 2017

Logic determining the TTL CloudFront uses.

Most of the session was a deep dive and explanation of CloudFront and how it works between your origin (EC2, S3, etc.) and your clients (users). The presenter explained how the TCP connections, both from the user to the edge location, and from the edge location to the origin function as well as provided some tips for keeping your cache-hit ratio high using headers like cloudfront-is-mobile-viewer to reduce variability. Plus, there were some cool examples given of Lambda@Edge taking headers and custom modifying them in the in-between. 

AWS re:Invent 2017

Lambda@Edge examples.

I've not used CloudFront a lot, but I'm more confident about it after this session. A lot of people walked out throughout this session, probably hoping for something different. Can't say I wouldn't have wanted to do the same thing if I knew CloudFront inside and out already. Being a 400-level course, it was surprisingly easy to grasp, perhaps due to the presenter.

AWS re:Invent 2017

Lego blocks.

Back at the Bellagio, stopped in for some grub at the FIX Restaurant & Bar. Snatched a cocktail, salmon, and mashed potatoes.

AWS re:Invent 2017

9/10. Would have been 10/10 if the waitress had brought the mac n cheese I ordered. Maybe she didn't hear me? Don't know. The food coma was instant, so I took a power nap before going out to check out the re:Play party.

Which brings us to the re:Play party! AWS goes all out for this. You could only enter in through the Venetian Hall A even though it was behind the LINQ. 

AWS re:Invent 2017

Oomce, oomce, oomce.

Food and drinks were included and the place was packed. It took place under a set of large tents, one being totally dedicated to games like glow-in-the-dark pool, glow-in-the-dark ping pong, adult-sized ball container, putt-putt pool, batting cages, dodge-ball and more.

AWS re:Invent 2017

Guy got lost in the balls. They couldn't find him.

AWS re:Invent 2017

Another tent is where the rave was going on with DJ Snake.

AWS re:Invent 2017

Oomce, oomce, oomce.

AWS re:Invent 2017

Not sure what these were about.

And then a final tent was packed full of arcade style games. There were some other areas I didn't explore since the line was ridiculous or I wasn't clear how to get in. 

AWS re:Invent 2017

Ancient video games.

I didn't end up staying too long since everything had huge lines and I'm not one for live music anyways.

AWS re:Invent 2017

Walked back to the Venetian casino and played poker with other attendees from re:Invent. Lost another $300. What is going on, man! I'm just the worst, although I turned Aces up when a guy got a set. Understandable, but annoying. Came home with some Taco Bell (I know, classy) and turned in for the night.

Cheers!

1. December 2017 17:26
by Aaron Medacco
0 Comments

AWS re:Invent 2017 - Day 1 Experience

1. December 2017 17:26 by Aaron Medacco | 0 Comments

Welcome to the first in a series of blog posts detailing my experience at AWS re:Invent 2017. If you're someone who is considering going to an AWS re:Invent conference, hopefully what follows will give you a flavor for what you can expect should you choose to fork over the cash for a ticket. The following content contains my personal impressions and experience, and may not (probably doesn't?) reflect the typical experience. Also, there will be some non-AWS fluff as well as I have not been to Las Vegas before.

AWS re:Invent 2017

My adventure starts at about Midnight. Yes, midnight. Living in Scottsdale, AZ, I figured, "Why not just drive instead of fly? After all, it's only a 6 hour drive and there won't be any traffic in the middle of the night." While that was true, what a mistake in retrospect. Arriving in Las Vegas with hardly any sleep after the road trip left me in pretty ragged shape for Monday's events. Next year, I'll definitely be flying and will get there on Sunday so I get can settle prior to Monday. I actually arrived so early, I couldn't check into my room and needed to burn some time. What better activity to do when exhausted than sit down at poker tables. Lost a quick $900 in short order. Hahaha! Truth be told, I got "coolered" back to back, but I probably played bad, too.

Once I got checked into my room at the Bellagio around 9:00am, I headed back to the Aria to get registered and pick up my re:Invent hoodie. Unfortunately, they didn't have my size, only had up to a Small. I couldn't help but smile about that. I ended up going to the Venetian later to exchange my Small for a Medium. Anyways, got my badge, ready to go! Or was I?

By the way, kudos to the Bellagio for putting these in every room. Forgot my phone charger. Well, the correct phone charger at least...

 AWS re:Invent 2017

...except it didn't have a charger compatible with my Samsung Galaxy S8. Kind of funny, but I wasn't laughing. Alright, maybe a little. Would end up getting one at a Phone store among one of the malls at the Strip. Oh yeah, and I also forgot to buy a memory card for my video recorder prior to leaving. Picked up one of those from a Best Buy Express vending machine. Vegas knows.

By this time I was crashing. Came back to my room, fell asleep, and missed 2 breakout sessions I was reserved for. Great job, Aaron! Off to a great start! 

Walked to the Aria to go check out the Certification Lounge. They had tables set up, food and drink, and some goodies available depending on what certifications you'd achieved. The registration badges have indicators on them that tell people if you're AWS certified or not, which they use to allow or deny access. I didn't end up staying too long, but there were a decent number of attendees with laptops open working and networking. Here's some of the things collected this year by walking around to the events: 

AWS re:Invent 2017

The re:Invent hoodie picked up at Registration (left) and the certification t-shirt inside the Certification Lounge (right).

AWS re:Invent 2017

Water bottle and AWS pins were given away at the Venetian Expo (top-left), badge and info packet at Registration (right), and the certification stickers at the Certification Lounge depending on which ones you've completed (bottom-left).

Headed over to the MGM Grand for my first breakout session, GPS: Anti Patterns: Learning From Failure (GPSTEC302). Before I discuss the session, I have to talk about something I severely underestimated about re:Invent. Walking! My body was definitely NOT ready. And I'm not an out-of-shape or big guy, either. The walking is legit! I remember tweeting about what I imagined would be my schedule weeks before re:Invent and Eric Hammond telling me I was being pretty optimistic about what I would actually be able to attend. No joke. Okay, enough of my complaining.

AWS re:Invent 2017

Waiting for things to get started.

AWS re:Invent 2017

Session about half-full. Plenty of room to get comfortable.

AWS re:Invent 2017

Presenter's shirt says, "got root?". Explaining methods for ensuring account resource compliance and using AWS account best practices when it comes to logging, backups, and fast reaction to nefarious changes.

This was an excellent session. The presenters were fantastic and poked fun at mistakes they themselves have made or those of customers they've talked to have made regarding automation (or lack thereof), compliance, and just overall bone-headedness (is that a word?). The big takeaways I found were to consider using services like CloudWatch, CloudTrail and Config to monitor and log activity in your AWS accounts to become aware when stupid raises it's ugly head. They threw out questions like, "What would happen if the root account's credentials were compromised and you didn't know about it until it was too late?", and "You have an automated process for creating backups, but do you actually test those backups?". From this came suggestions to regularly store and test backups to another account in case an account gets compromised and using things like MFA, especially for root and privileged users.

Additionally, the presenters made a good argument for not using the management console for activities once you become more familiar with AWS, particularly if you're leveraging the automation tools AWS provides like OpsWorks and CloudFormation as that kind of manual mucking around via the console can leave you in funny states for stacks deployed with those services. Along those lines, they also suggested dividing up the different tiers of your application infrastructure into their own stacks so that when you need to make changes to something or scale, you don't end up changing the whole system. Instead, you only modify or scale the relevant stack. Overall good session. If they have it again next year, I would recommend it. You'll get some laughs, if nothing else. The guys were pretty funny.

Once out, I had a meeting scheduled to talk with a company (presumably about upcoming Pluralsight work) at the Global Partner Summit Welcome Reception. Now, I'll admit I got a little frustrated trying to find where the **** this was taking place! AWS did a great job sending lots of guides with re:Invent flags everywhere to answer questions and direct attendees to their events, and these guys were godsends every time except when it came to finding this event. I think I just got unlucky with a few that were misinformed.

AWS re:Invent 2017

These guys were scattered all over the strip and inside the hotels. Very helpful!

First, I was told to go to the one of the ballrooms. Found what appeared to be some kind of Presenter's Registration there. Then, found another guide who said to go to the Garden Grand Arena. Walked over there, total graveyard, and ironically, a random dude there who wasn't even one of the re:Invent guides told me where it actually was. He also said, "Oh yeah, and unless you want to be standing in line all night, you might want to reconsider." It was late enough at this point, I figured I'd just head back to the Bellagio for a much needed poker session, so that's what I did. However, on the way back, holy ****, he was right. I've never seen a line as long as the one to get into the GPS Welcome Reception in my life. It went from the food court, through the entire casino, out of the casino, and further back to I couldn't tell where. Apparently, I was the only one who missed the memo, since everyone else knew where to go, but still, that line. 

Long hike back to the Bellagio, played poker for about 3 hours, lost $200 (man, I suck), and on my way back to my room discovered I didn't eat anything all day. LOL! Picked up a couple pizza slices and crashed for the night. A good night's sleep? Yes, please. Tomorrow would be better.

Cheers!

22. April 2017 23:50
by Aaron Medacco
0 Comments

Using Nested Stacks w/ AWS CloudFormation

22. April 2017 23:50 by Aaron Medacco | 0 Comments

When describing your cloud infrastructure using AWS CloudFormation, your templates can become large and difficult to manage as your desired stacks grow significant. CloudFormation allows you to nest templates giving you the ability to piecemeal different chunks of your infrastructure into smaller modules. For instance, suppose you have several templates that involve creating an elastic load balancer. Rather than copying / pasting the same JSON between each template, you can write one template for provisioning the load balancer and then reference that template in "parent" templates that require it. 

This has several advantages. Code concerning the load balancer is consolidated in one place. This means when changes need to be made to it's configuration, you don't need to revisit each template where you copied the code at one point in time. This saves you both time and grief, removing the chance for human error to cause one template whose ELB is different than another template when they should be identical. It also enhances your ability to develop and test your CloudFormation templates. When writing templates, it's common to make incremental changes to JSON, create a stack from the template to validate structure and behavior, tear down the stack, and rinse / repeat. For large templates, provisioning and deleting stacks will slow you down as you wait for feedback. Templates that are more modular / smaller generate more specific testing providing feedback in faster iterations.

AWS CloudFormation 

In this post, I'll reveal two CloudFormation templates, one that provisions an application load balancer, and one that creates a hosted zone with a record set pointing to the load balancer being provisioned. I'll do this by nesting the template for the load balancer inside the template that creates a Route 53 hosted zone. Keep in mind, I won't be setting up listeners or target groups for the load balancer. I'm only demonstrating how to nest CloudFormation templates.

Here is my template for provisioning an application load balancer without any configuration:

Note: You can download this template here.

{
    "AWSTemplateFormatVersion": "2010-09-09",
	"Description": "Simple Load Balancer",
	"Parameters": {
		"VPC": {
			"Type": "AWS::EC2::VPC::Id",
			"Description": "VPC for the load balancer."
		},
		"PublicSubnet1": {
			"Type": "AWS::EC2::Subnet::Id",
			"Description": "First public subnet."
		},
		"PublicSubnet2": {
			"Type": "AWS::EC2::Subnet::Id",
			"Description": "Second public subnet."
		}
	},
	"Resources": {
		"ElasticLoadBalancer": {
			"Type": "AWS::ElasticLoadBalancingV2::LoadBalancer",
			"Properties" : {
				"Name": "Load-Balancer",
				"Scheme": "internet-facing",
				"Subnets": [ {"Ref": "PublicSubnet1"}, {"Ref": "PublicSubnet2"} ],
				"Tags": [ { "Key": "Name", "Value": "CloudFormation Load Balancer" } ]
			}
		}
	},
	"Outputs": {
		"LoadBalancerDNS": {
			"Description": "Public DNS For Load Balancer",
			"Value": { "Fn::GetAtt": [ "ElasticLoadBalancer", "DNSName" ] }
		},
		"LoadBalancerHostedZoneID": {
			"Description": "Canonical Hosted Zone ID of load balancer.",
			"Value": { "Fn::GetAtt": [ "ElasticLoadBalancer", "CanonicalHostedZoneID" ] } 
		}
	}
}

Notice that there are some parameters related to networking being asked for. In this case, I'm requesting two public subnets for the ELB since I intend to have it exposed to the internet. I've also defined some outputs, one is the public DNS for the load balancer I'm creating, and the other is it's hosted zone. These values will come in handy when my parent template sets up an A record set pointing to the newly made ELB.

The following is the template for provisioning a hosted zone given a domain name:

Note: You can download this template here.

{
    "AWSTemplateFormatVersion": "2010-09-09",
	"Description": "Generate internet-facing load balancer.",
	"Parameters": {
		"Domain": {
			"Type": "String",
			"Description": "Domain serviced by load balancer."
		},
		"VPC": {
			"Type": "AWS::EC2::VPC::Id",
			"Description": "VPC for the load balancer."
		},
		"PublicSubnet1": {
			"Type": "AWS::EC2::Subnet::Id",
			"Description": "First public subnet."
		},
		"PublicSubnet2": {
			"Type": "AWS::EC2::Subnet::Id",
			"Description": "Second public subnet."
		}
	},
	"Resources": {
		"HostedZone": {
			"Type": "AWS::Route53::HostedZone",
			"Properties": {
				"Name": { "Ref": "Domain" }
			}
		},
		"HostedZoneRecords": {
			"Type": "AWS::Route53::RecordSetGroup",
			"Properties": {
				"HostedZoneId": { "Ref": "HostedZone" },
				"RecordSets": [{
					"Name": { "Ref": "Domain" },
					"Type": "A",
					"AliasTarget": {
						"DNSName": { "Fn::GetAtt": [ "LoadBalancerStack", "Outputs.LoadBalancerDNS" ]},
						"HostedZoneId": { "Fn::GetAtt": [ "LoadBalancerStack", "Outputs.LoadBalancerHostedZoneID" ]}
					}
				}]
			}
		},
		"LoadBalancerStack": {
			"Type": "AWS::CloudFormation::Stack",
			"Properties": {
				"Parameters": {
					"VPC": { "Ref": "VPC" },
					"PublicSubnet1": { "Ref": "PublicSubnet1" },
					"PublicSubnet2": { "Ref": "PublicSubnet2" }
				},
				"TemplateURL": "https://s3.amazonaws.com/cf-templates-1bc7bmahm5ald-us-east-1/loadbalancer.json"
			}
		}
	}
}

You can see I've added a parameter asking for a domain in addition to the values required for the previous template. Then, in the resources section of the template, I'm creating an AWS::CloudFormation::Stack referencing the location my nested template is stored and passing the parameters required in order to invoke it. In the section where I define my hosted zone records, I need to know the DNS name and hosted zone of the application load balancer. These are retrieved by referencing the outputs returned by the nested template. 

Creating a CloudFormation stack using the parent template, I now have a Route 53 hosted zone for the input domain pointing to the newly created load balancer. From here, I could reference the load balancer template in any number of templates requiring it without bloating each of them with pasted JSON. The next step would be to create listeners and target groups with EC2 instance targets, but that is a separate exercise.

Cheers!

7. January 2017 20:00
by Aaron Medacco
0 Comments

Third Party Tools for Diagramming Your AWS System Architecture

7. January 2017 20:00 by Aaron Medacco | 0 Comments

As AWS professionals, there are times when we're asked to provide diagrams or high-level blueprints of whatever system we are designing. Maybe you need to create a proof of concept of your solution before getting the buy-in you need to start. Perhaps you're working on a team, and need an agreed upon blueprint so team members stay on the same page and can refer back as they build their cloud solution. Also, for those participating or looking to participate in the AWS Partner Network as either a Technical or Consulting Partner, diagrams for your customers' system architecture are required for reference submissions. These references are necessary in order to meet partnership requirements.

The following third-party tools can be helpful in creating such materials. Admittedly, I used to create these using Photoshop, but for larger architectures or for diagrams that require a lot of detail, it'd be more efficient using tools built specifically for this task.

Cloudcraft

Cloudcraft provides an easy-to-use solution for creating AWS architecture diagrams. It features a tilted tiled grid where you can drag and drop resources relevant to your solution. The controls are smart and provide cost estimation. For instance, when placing an EC2 instance, you can choose what kind of instance it is and see it reflected both on the diagram and in the cost estimate. Once your done editing your diagram, you can export it as an image or generate a link to share with others.

CloudCraft Diagram

However, the feature that excited me most was being able to sync real-time information of your resources to Cloudcraft directly. By creating a user with read-only access, you can allow Cloudcraft to pull information from your AWS account, and generate your diagram for you. These diagrams are persistent and will update as you change your architecture over time. Certainly better than using Photoshop.

Cloudcraft offers one free and three paid plans. For simple architectures that don't require large diagrams, you should be able to accomplish what you need without spending any money or providing credit card information. Those with larger architectures who will need larger grids should look at their Pro Solo plan ($49/mo). In addition to the infinite grid size, you'll receive the awesome ability of syncing with your AWS account and gain basic support from Cloudcraft. All of this is limited to one user until you reach the Pro Team plan ($245/mo) which includes priority support from Cloudcraft and allows for team collaboration. For additional customization and larger team sizes, Cloudcraft asks that you contact them for enterprise arrangements. Check them out here.

Lucidchart

Lucidchart also features a custom drag and drop interface for diagramming your AWS resources. With a rich library of AWS shapes and service icons, you can create easily understood top-down blueprints of your system architecture. These are strictly 2D and do not have the 3D feel of the Cloudcraft diagrams. Real-time collaboration with your team members is supported as is team chat. For me, Lucidchart felt like Photoshop Lite for AWS blueprints. There are a lot of options available, and if it's easier, you can select one of their pre-made templates to get a head start.

LucidChart Diagram

Lucidchart also has the ability to import resources from your AWS account using the least access required, something I find very cool (if you haven't noticed). Also, for those who have other diagram needs, Lucidchart is not exclusive to only AWS usage and so it can provide even more value over other cloud-only diagram creation services.

A free trial exists that does not require credit card information. Paid plans start at ($5/mo) for the Basic which supports an unlimited amount of shapes and documents within 100 MB of storage for a single user. Pro plans are ($9/mo) and expand storage limits to 1 GB with access to all shape libraries and support for Visio import/export. Team plans start at ($20/mo) and increase with the number of users. It works out to ($6/mo) per user on your team as you move beyond ($20/mo). Team plans come with management tools and the ability to integrate with third party tools like JIRA and Google Apps. Enterprise customers should contact Lucidchart directly. Check them out here.

Hava

While Hava does not allow for heavy customization of your architectural diagrams, it has additional use cases which I though were exciting. Hava also allows you to sync your AWS resources from your account using read-only access in order to create a rough diagram of your architecture. Unfortunately, this is where the customization stops. You're pretty much stuck with the diagram they generate, which you can export as an image or PDF. Those who aren't talented artists that want a quick way to get a passing diagram will find this kind of automation useful.

Hava Diagram

Hava has other features outside of generating diagrams that are worth mentioning. Anyone interested in efficiency? AWS professionals writing their own CloudFormation templates can generate diagrams from the raw template JSON, allowing you to see a blueprint of what your stack will look like (and cost) without actually provisioning it. Additionally, Hava will generate a CloudFormation template from diagrams you import. What!? This means you can setup an environment in AWS that you are comfortable with, then import it into Hava. From there, you can generate a CloudFormation template from the diagram that you can use to replicate the environment going forward. Mmm...time.

Hava offers a two week free trial with limited features. Paid plans are tiered by the number of resources being visualized. A resource counts as any EC2 instance, RDS instance, Load Balancer or Elasticache. Every plan allows for an unlimited amount of users. Tiers are XS ($39/mo, 20 resources), S ($199/mo, 100 resources), M ($499/mo, 250 resources), L ($999/mo, 500 resources), and XL ($1,999/mo, 1000 resources). Again, not strictly a customized diagramming tool, but a huge value to the right customers. Check them out here.

Cheers!

Copyright © 2016-2017 Aaron Medacco