Blog

Amazon Cloud Hosting

Amazon is a huge player in the cloud hosting space. Cloud hosting is basically where a company fills a server farm with racks upon racks of physical computers, hard drives and routers. The company then uses software to combine the individual computers into a super computer which is then partitioned off into a series of virtual servers of varying sizes and types. The company then resells usage of these virtual servers to their clients. Amazon Web Services (the division which provides the service) offers a variety of different types of virtual servers, but the basic, and most flexible, is called Elastic Cloud Compute (EC2).

Amazon EC2

Instances

Instances can be thought of as the virtual processor, motherboard and RAM of the virtual server. Amazon offers three different types of Instances (On-Demand, Reserved, and Spot) and of varying different sizes.

On-Demand Instances

On-demand Instances are those you intend on using on a temporary basis. You are paying only for the amount of time you actually use the instance, so they are excellent for short-term projects and to get settings worked out.

Reserved Instances

Reserved Instances are instances which are dedicated to your account. They do not go away if you stop, or terminate them. Well, that is not quite correct. You are actually reserving usage of a particular type of instance, rather than a particular instance. The different levels of Reserved Instances are basically usage structures. You prepay to reserve an instance and in exchange get a discount on the hourly rate. Reserved Instances are ideal for long-term server applications, like website host, email servers, etc.

Reserved Instances Utilization Rates
  • Heavy Utilization - These instances are used 80%+ of the month.The core website and email servers.
  • Medium Utilization - These instances are used for 40-79% utilization rates. If you run a few heavy traffic websites, then these instances would be the load-balanced servers to support demand during peak times like the evenings and weekends.
  • Light Utilization - These instances are used for 17-30% utilization rates. This time frame corresponds really well with development servers that are started in the morning, run for 7-8 hours then turned off in the evening.

Spot Instances

Spot Instances are similar to on-demand instances, but are designed for special project type circumstances. Amazon obviously wants to keep all of their servers running all the time (ie. 100% utilization), however with the on-demand type structure, there are times when some servers are not being used. During these low slow times, Amazon would rather sell time on them temporarily for a discount rather than let them run empty. These temporary discounted servers are the spot instances. Spot instances work really well for periodically maintenance activities. To use a spot instance, you indicate the size of instance and the maximum price you bid for usage of that instance. Once the price for that size of instance goes below the bid price, the instance starts up and you get it until the prices goes back over your max bid price. Note you are only charged the actual price, not your bid price, so you can often pay less per hour than your bid price for spot instances.

EC2 Resources

Elastic Block Store Volumes

Elastic Block Storage (EBS) volumes are the virtual hard drives of the virtual server. There are two types of EBS Volumes, Standard and Provisioned  IOPS (Input/output Operations Per Second).

Standard EBS Volumes

Standard EBS volumes correspond the best to physical media hard disks. You can read and write to them at average rates and deliver about 100 IOPS. Unless you need high writing/ reading capabilities, a standard EBS is what you'd use.

Provisioned IOPS Volumes

Provisioned IOPS are for high read/write type situations. The most common examples is a database server. These volumes are very powerful, but also very expensive (relatively). There are other AWS Services offered, like S3, SES and RDP, but I currently don't use them some will avoid going into detail on those services until I use them.

Setting up a Free Tier Amazon EC2 Instance

Amazon AWS is currently offering a 'free tier' for 1 year. Simply you get a micro instance to get your server up and running, play with different settings and such. It is the standard free trial offer, but with a virtual server. If you've never used AWS before, I recommend using the free tier server to get acquainted with the capabilities of AWS then move to a real server later. Also, once you have all the settings working on the free tier instance, you can transfer to paid instance in 15 minutes. AWS Free Tier

  • 750  hrs/month Micro instance (613 MB of RAM, Linux or Windows)
  • 750 hrs/month Elastic Load Balancer (15 GB of data processing)
  • 30GB of EBS space
  • 5 GB of Amazon S3 standard storage

Setting up a New Instance

  1. Go to and login to your .
  2. Click on "EC2" under the Compute & Networking section. (Note you may have to choose your region at this point if it hasn't been setup yet.)
  3. At the "Amazon EC2 Console Dashboard" there should be a button in the middle of screen called "Launce Instance", Click it.
  4. The wizard will pop-up asking you to choose a type of wizard. Select "Class Wizard" and click "Continue" at the bottom right.
  5. Select the "Amazon Linux AMI ####.##"  AMI. It should be the top option under the "Quick Start" tab. Note, you can use any of the AMIs with a yellow star next to the select button for the free tier. The 32-bit version will be slightly easier to deal with later, but 64-bit version works just fine also.
  6. You will now need to determine the basic instance details. For the free tier, make sure the "Number of Instances" is set to 1 and the "Instance Type" is "Micro  (t1.micro, 613 MiB)". The "Availability Zone" doesn't matter right now so "No Preference" is fine. Click the "Continue" when the settings are correct.
  7. You now can determine some of the advanced options. The only thing you need to be concerned about is the "Shutdown Behavior" which should be set to "Stop". Click the "Continue" when the settings are correct.
  8. The next page is a the storage details. New instances default to a "Root Volume" which is effectively a new blank standard EBS volume.I recommend you uncheck the "Delete on Termination" checkbox to prevent you from accidentally erasing your data when the instance dies.Click the "Continue" when the settings are correct.
  9. Now you can set metadata you want to correspond with this instance. These key/value pairs will help with searching and administrating large clouds of multiple servers. In addition to the "Name" key, we generally always place a "admin" key with the value equaling the programmer who administrates the instance. You can place up to 10, and you can always change them later. Enter something for the "Name" key's value then press "Continue".
  10. The next step is absolutely essential to run a secure instance and have access via an SSH client. Instead of using usernames and passwords, AWS uses usernames and encryption keys, called "Key Pairs." This encryption level prevents brute force attacks against your instance. Enter a name (alphanumeric without spaces) then press the "Create and Download your Key Pair" button. You will be expected to save the key file somewhere on your locale computer; remember where because you will need this file later when setting up your SSH and SCP clients. Click "Continue" once you have created your Key Pair.
  11. The last setting you need to determine is the firewall. Amazon allows you to create an off-instance firewall to limit access to your instance. Click on the "Create a New Security Group" radio button then enter a "Group Name" and "Group Description". Leave the the "Inbound Rules" empty for now. Typically when you create an instance, you will you use a pre-created security group that you already setup for the purpose of the instance.  Click "Continue" once the new security group is created.
  12. This last page is just a review of the settings for your new instance. Look over them and make sure everything is correct then click "Launch." A few moments later your simple Free Tier instance will be up and running. The next step is getting access to it, then installing software and configuring everything. These will be discussed in future posts.

Installing the necessary software on an AWS Amazon Linux AMI server

There is a variety of software you will need to get your new AWS web server up and running. You probably already have the desktop clients if you every did any server work previously, the core server software however will need to be installed, depending on your purposes for the server. This page will be updated from time to time as new installation and configuration guides are added.

Desktop Clients

SoftwareDescriptionAvailable at:Documentation
  Free SSH client. Utilizes basic command line style interface   |
  Free SCP/SFTP/FTP client for Windows. Offers a graphical user interface to move and edit files.    

I am bias to Windows software. All of these programs run on Windows XP and Windows 7 (32-bit & 64-bit systems). If you are running a Linux or Mac system....well...they may work. The program's name link will go to instructions on configuring the software to access your AWS Instance.

Core Server Software

SoftwareUsageDescriptionDocumentation
Apache2 Website hosting The basic web server which deals with internet (http/https) traffic to the server.  
PHP Dynamic Websites (optional) Requires:Apache2 Scripting language for creating dynamic webpages. Used by most CMS, Wiki & Blog systems to manage content  
MySQL Database The basic free SQL database server. Used by many CMS, Wiki & Blog systems to store content.  
phpMyAdmin Database Administration (optional) Requires:Apache2, PHP & MySQL Graphical, HTML based admin tool for accessing and managing mySQL databases.  
Postfix Mail-Transfer-Agent (ie: email server) Accepts and sends email. Versatile and can be used with a variety of database structures.  
Courier Email Client Portal (optional) Requires:Postfix Offers a portal to access email via any client, including MS Outlook, Thunderbird & smart phones. Offers IMAP and POP3 systems.  
Spamassassin Email Spam filter (optional) Requires:Postfix Works with MTAs to prevent spam from arriving on server  
BIND9 DNS Server (optional) DNS server which allows you to create your own dns records.  

Note all of these programs are free, and most are open source. All of the installation instructions are specific to the Amazon Linux AMI. This stripped down version of Linux is a special Amazon derivative of Fedora. When I was originally setting up our servers, some of the differences between RedHat, Ubuntu, Debian and this version of Linux drove me crazy, therefore all of these instructions worked on the newest Amazon Linux AMI version (currently 2012.03).

Setting up PuTTY for AWS access

PuTTY is a free open source SSH client. You will need to install it (basically download the installer and run it) if you have not already done so. Make sure you have both PuTTY and PuTTYgen programs.

Configuration for AWS Instance access

These instructions assume you have already setup an AWS instance. If you haven't setup an AWS Instance, you can find help at Setting up a Free Tier Amazon EC2 Instance.

  1. Open your AWS Console (go to http://aws.amazon.com and login)
  2. Go to "EC2" under "Compute and Networking"
  3. Click on "Instances" under the "Instances" section of the Navigation pane. This will display all of the instances you currently have running. Clicking on the name of the instance will show the details of that instance below. Select the instance you want to configure PuTTY for then find the "Key Pair Name" and "Security Groups" values under the "Description" tab. You will need to edit the security group in order to allow PuTTY to access your instance then confirm the security key with the key pair name.
  4. Find the value for "Public DNS" under the "Description" tab then highlight it (shift+ left click while selecting the text) and press CTRL+C to copy the text. You will need this value when setting up PuTTY and I find copy & pasting a whole lot easier than retyping something.
  5. Click on "Security Groups" under the "Networking & Security" section of the Navigation pane. This will show your security groups for this region. Click on the instance's security group to see the details of that group.
  6. Click on the "Inbound" tab to edit the firewall associated with this security group.
  7. SSH clients use port 22 for access, so you will need to verify that TCP port 22 (SSH) is listed on the table to the right. If it is not listed, or there is no table, select "SSH" under for "Create a new rule" then add your computer's ip address to the source line followed by "/32". AWS security groups use CIDR notation for IP address ranges. Simply, "/32" limits the range to a single IP address. Click "Add Rule" then click "Apply Rule Changes"
  8. Click on "Key Pairs" under the "Networking & Security" section of the Navigation pane. The "Fingerprint" for the "Key Pair Name" will be needed later to confirm your connection to the AWS Instance.
  9. Open PuTTYgen. Click on "Load" then choose the Key Pair file for the "Key Pair Name" of the instance. If you just created the instance following the above instructions, the key file is the one you had to save after you generated the "Key Pair Name."
  10. Click "Generate" to create the PuTTY usable security key. Save the file somewhere you will remember and can control, since access to this file will allow access to the AWS instance. Close PuTTYgen.
  11. Open PuTTY. The default "Category " should be "Session." If "Session" is not selected, select it.
  12. Click on the "Host Name (or IP address)" input and press CTRL+C to past your "Public DNS" address as the host name. Make sure "Port" is set to "22" and "SSH" is selected as the "Connection type:".
  13. Expand the "Connection" Category and expand the "SSH" section and click on "Auth."
  14. Click on "Browse" and open the Putty key you just created with PuTTYgen.
  15. Click on the "Session" Category again and choose "Save." This way you won't have to repeat setting up PuTTY every time you want to use it. NOTE: This is security weakness, because anyone with access to your computer would then be able to access your AWS Instance, however most people have their own private computer which limits the security risk. I just find it a pain to redo everything every time I want to access the server.
  16. Click "Open" to open the SSH connection.
  17. Type "ec2-user" at the "login as:" prompt. The "ec2-user" is the default user for the Amazon Linux AMI. You cannot login as "root" as a security measure.
  18. To transfer to the "root" user, type:
    $ sudo su