Tuesday, August 07, 2012

A dozen random things I've learned while using Amazon Web Services that you may find useful

I ran a demonstration project using Amazon Web Services (AWS) about a year and a half ago, and we continue to use AWS for one-off projects and sites. It's a useful tool.  That being said, I've learned (usually the hard way) a few non-obvious tricks and techniques that may be of interest to my fellow web geeks. If you've been using AWS for any length of time, you have probably learned all of these already. If not, allow me to offer them for your consideration now. Disclaimer - I've not used all of the services available from AWS, so I'll only mention the ones I've actually used myself.

First - a short glossary of terms (this will make the discussion a little easier):
  • AWS - Amazon Web Services. This is the umbrella under which all of the services described below live.

  • EC2 - Elastic Cloud Computing; this is the virtual cloud based servers. Use this for webhosting. Once you kill an instance, everything vanishes in a puff of smoke (unless you take some additional steps).

  • EBS - Elastic Block Storage. Persistent storage for the EC2 service - i.e. it doesn't vanish after you stop it. Think of it as a virtual external hard drive. Can only be used as part of an EC2 instance.

  • S3 - Simple Storage Service. Sort of a virtual fileserver; in my experience you would use this for bulk/utility storage, not for http requests (like for images used on a website). You can use it outside of a EC2 instance.

  • AMI - Amazon Machine Image: a pre-configured virtualized operating system image used to boot to an instance. These come in a variety of flavors, and you can create your own.

And now, a dozen random hints and bits of advice:
  1. There are a lot of pre-built AMI's out there but unless you have a specific reason to do so, most of the time, when choosing an Amazon Machine Image (AMI), you should find one that is as bare-bones as possible and add only the applications and services you need. Your environment will be more secure and not be bogged down with things you don't need. This is one good reason to use Linux, rather than Windows.

  2. When choosing an AMI, use an EBS-based image, not an S3-based one. Once you've initially spun up an instance and made all the changes you want (adding services, installing software), you'll want to create a custom image to save those changes. Believe me - it's a whole lot easier to do this with EBS-based images. It is possible to convert an S3-based image to an EBS-based image, but believe me, it's a bitch to do - not for the faint of heart. Also: EBS-based instances can be rebooted and stopped (not  just terminated).

  3. Security groups are a little like firewalls; they can control who has access to your instance. You can assign more than one security group to an instance. If you have several instances, use an instance-specific security group and include in it a shared security group.

  4. If you are connecting from one instance to another - say for a DB server - you can only connect using the private IP if you are in the same Availability Zone. If you are in different Availability Zones, you'll need to connect using external IPs.

  5. Amazon offers a "Free Tier" service, which is good for a year's worth of service to new accounts. It gives you access to a "micro" instance of either Linux or Windows service, 30 GB of EBS storage, and some additional services. Good for getting your feet wet and seeing if AWS is something that appeals to you. Note you can still incur costs using this free tier if you aren't mindful.

  6. Set up your architecture so that you are storing data and as much configuration information as possible on a separate EBS volume; this way, should your instance go down (and believe me, it will, sooner or later), you won't loose it. If you make anything more than trivial changes to your operating environment (updates/patches/new software), you should create an updated image (see #2, above), so you won't lose those either.

  7. When your instance is created, it will be assigned a dynamic (random) external IP address. Request an "Elastic (static) IP" and use that instead. You are not billed for these as long as they are attached to an instance (this is to discourage requesting IPs that aren't actually in service). This makes managing your DNS settings much easier since you can re-assign the Elastic IP to a different instance if necessary, say if your instance has to be restarted.  

  8. You will probably need to use Elastic IPs if you have to go through a corporate firewall to get to AWS. So that you can get them in your firewall rules, you may find that you need to reserve IPs,  Be aware if you have requested IPs but don't actually have them assigned, you'll be hit with a small charge for each unused IP. If you are using AWS for testing, you can swap IPs around, which can reduce your need for IPs.

  9. The default billing method is by credit card. This can be a pain if you are doing this for a larger business. While not specifically for AWS, Amazon does appear to offer corporate billing by purchase order from the following URL:  http://www.amazon.com/gp/cobrandcard/marketing.html?pr=ibprox&inc=def&place=url  - Please note I've not actually managed to move to to this method, but it does appear to be possible. 

  10. A fairly large 3rd party ecology has sprung up to assist you in managing your AWS stuff, including stand-alone tools, browser add-ons, and web services with free and paid tiers. Depending on what you are doing you may find one or more of these useful. There are command-line tools available from Amazon for alphageeks (I avoid these);  I find Amazon's web-based management console is all I really need.

  11. Watch your costs - while prices seem low, they can creep up so that what started out as a cheap solution can turn into one that's not so cheap. The cost of Windows instances and Redhat instances (as opposed to other Linux distros) can add up. Running instances rack up costs - if you are using this for development purposes, halt or stop it when you aren't using it (you can do this with EBS-based images).

  12. Keep copious notes. You'll be glad you did.
Let me know if you found any of these useful, and put any of your own favorite tricks or hints in the comment section.


Anoushka Sakthi said...

It is interesting to know how vast amount of data is stored and retrieved from the cloud. The whole process was explained clearly with the suitable example.
cloud computing training in chennai
cloud computing training

mari A said...

Needed to compose you a very little word to thank you yet again regarding the nice suggestions you’ve contributed here. aws training in chennai

alanna jules said...

Good details on AWS web service. It is important to have a good AWS DR plan. Thanks for sharing

Dhileepan K said...

Good things about AWS, I really enjoy it, keep your good work

AWS Training in Chennai | AWS Training Institute in Chennai