AWS Important Money Saving Tips

Amazon’s virtual web server platform has grown to one of the largest server providers in the world.
Many businesses and individuals, from Netflix, the US Govt, down to little shops with one tiny server running.

If you are using AWS Amazon Web Services, you should be familiar with their costing structure. The greatest benefit of AWS’s cloud server business is that you only pay for what you use.
Each server, service, feature, etc… has its own associated cost per minute use as defined on their AWS Simple Monthly Calculator.

I’ve been using AWS for nearly 7 years, as an ‘early adopter’ and have learned over time, several easy ways to avoid spending unnecessary money with them (notice that I didn’t write ‘ways to save money’).

1. Disassociate Unused Elastic IPs.  See graphic below.

There is no additional cost to attach an Elastic IP to each server you have running on AWS.  However, if you stop the server, and leave the Elastic IP “running” without deleting it from your account, you will be paying for it indefinitely.  AWS doesn’t remind you that you have un-attached Elastic IPs.

Each non-attached Elastic IP is costing you $3.66 per month  (that’s almost $45/year per IP address).

For example, each dev server that you turn on and off as needed.  If you have an Elastic IP set for each server, and if you stop that server, you will be paying for the non-attached Elastic IP.

So next time you turn off a dev server, if you don’t need an elastic IP associated to it, best to remove it, or you will be paying just to keep the address available to you.

More money saving tips on the way, stay tuned…

AWS Linux Ubuntu Server Setup

Ubuntu Server Setup – Top 5 things to do on the First Day

You’ve just unwrapped your new Ubuntu server running on Ubuntu 12.04 LTS and now it’s time to roll up your sleeves and get started. What should you do first? Here’s a checklist of the top things I do for each new Ubuntu server that I install.

Let’s approach it from two ways:
– You’re working with a managed network provider like Rackspace, etc… or
– You’re doing it yourself on AWS as a new EC2 instance (Similar to managed, with some caveats).

You have your server credentials and have logged on to your server with a SSH connection. You are staring at your command prompt and are ready to begin…
1. Check your server to make sure you got what you wanted

Are the drives partitioned right? Do you have any space on /var to work?

df -h

What release are you running? The current (as of 2/13) latest LTS version of Ubuntu is 12.04.x

lsb_release -a

2. Update your copy of distribution files.

sudo apt-get update

This will make sure that as we get started installing new software, we’ll have the latest and greatest of everything.

3. Install The LAMP stack ([Linux], Apache, MySQL, PHP) We’ll want to throw in phpmyadmin here as well. In case you aren’t familiar, phpmyadmin is great for setting up database permissions and general DB management from within a browser-based GUI.

sudo apt-get install lamp-server^ phpmyadmin

Answer Yes to the prompts as it installs each of the requested programs in the LAMP stack. You’ll need to pick out a password for MySQL and then confirm it. Later you’ll be asked for a password for phpmyadmin. Use the same password, it will make things easier for later. (You can always change your password later from within MySQL or PhpMyAdmin.

4. Install htop
htop is essentially “top” on steroids. It gives you a good visual picture of your server’s load and what processes are running. It’s sort of like the baby monitor of servers.

sudo apt-get install htop

5. Setup a test web page
You’ve installed apache, so you can test that everything is set correctly. Go to your default web root directory /var/www (this can be changed later, and we’ll show you how to do that in our next blog in this series)

cd /var/www

Now, edit the file index.html and put in some text that identifies it as your new server.

sudo vi /var/www/index.html

(or .php, or .htm depending on your configuration)

Now start or reload apache to make sure everything is running right.

sudo service apache2 reload

After you’ve saved the file, you can go to a browser window and type in your server ip address. You should see your temporary index page that you just created/edited.

Configuring Apache, setting up cron jobs, rsync to synchronize with multiple servers, setting up MySQL database connectivity, and more…
Check out our other blog posts on these topics.

AWS Ubuntu Server Setup

Ubuntu Server AWS EC2 ELB Apache Log Fix

If you’ve launched AWS EC2 server instances behind a load balancer, then you’ve probably noticed that your Apache logs are showing the internal ip address of the load balancer for all entries.

I’ve seen a few blogs that explain that the reason for this is that the ELB load balancer is configured as a reverse proxy and the actual ip address of the origin is masked.  Aside from the cosmetics of seeing thousands of log entries all with the same ip address, this is actually bad for SEO as your sitemaps will also be showing all your visitors coming from one ip address.  Also, if you are using geoip the country filter won’t work as it will be looking at all visitors as coming from Amazon’s region that your EC2 instances are located.

Here’s the fix:
Very simple actually.  It worked for me on Ubuntu 12.04 LTS and I assume it will work on any Linux configuration.  Instead of logging the Host (which in our case is the internal ip), we want to log the X-Forwarded_for which is the origin ip for the entry.
Open your apache configuration file for edit.  On Ubuntu it’s at /etc/apache2/apache2.conf    You can also use locate to find it on your instance if it’s not on Ubuntu.  locate apache2.conf

sudo vi /etc/apache2/apache2.conf

Next within configuration file look for the section with LogFormat   There should be several lines beginning with LogFormat.  The top three should be for vhost_combined, combined, and common.   In all three lines where you see %h  (which is echoing to the apache log the host) change that to %{X-Forwarded-For}i


LogFormat “%v:%p %h %l %u %t \”%r\” %>s %O \”%{Referer}i\” \”%{User-Agent}i\”” vhost_combined
LogFormat “%h %l %u %t \”%r\” %>s %O \”%{Referer}i\” \”%{User-Agent}i\”” combined
LogFormat “%h %l %u %t \”%r\” %>s %O” common


LogFormat “%v:%p %{X-Forwarded-For}i %l %u %t \”%r\” %>s %O \”%{Referer}i\” \”%{User-Agent}i\”” vhost_combined
LogFormat “%{X-Forwarded-For}i %l %u %t \”%r\” %>s %O \”%{Referer}i\” \”%{User-Agent}i\”” combined
LogFormat “%{X-Forwarded-For}i %l %u %t \”%r\” %>s %O” common

Then remember to reload apache

sudo service apache2 reload

Note of Caution: It’s relatively easy for someone to spoof the header and change the X-Forwarded-For to whatever address they want. That would mask from you their actual ip. Therefore, this solution is not intended for applications that have security implications.

Another tip. Within your web application if you are using the ip for any purpose, like Maxmind’s geoip, you’ll need to echo the X-Forwarded-For instead of the Remote host as well. e.g. $ip=$_SERVER[‘HTTP_X_FORWARDED_FOR’];

Let us know if you have any suggestions on how to improve this method.

AWS Browser Tools Internet Tips & Tricks

How to Get USA Proxy Using Your AWS Server – Three Easy Steps

If you have an AWS Amazon Web Services Linux Web Server, you are in luck.  Creating a full USA Proxy Server has never been easier.

Here’s a quick guide that will show you how to set up a proxy server using your existing AWS server.

We’ll show you in just a few easy and completely free steps how to create a winproxy tunnel. Your local PC will take the IP address of your AWS server.

AWS Windows

How to Copy Files to AWS EC2 Server From Windows PC With Command Prompt

If you run AWS servers from a Windows PC, you need an easy way to transfer files up and back from your work desktop to the server and back.

Linux users have the ease of native SSH tools such as SCP built in to their command tool utilities, and Windows users want the same flexibility.

Follow these easy steps and you’ll be transferring files in a flash.

1.  Get PSCP
Download the Putty tool PSCP (Putty SCP) for secure file transfer on Windows.  This is a freeware tool.

AWS Ubuntu Server Setup

Ubuntu Server How to Configure for Sendmail Port 587

Avoid AWS Blocking your Server from sending emails!
I was using AWS Amazon’s Web Service and one of my servers was set to reply to users contacting with a generic response. I had configured Sendmail and tested everything was working fine.

A day later, I got a message from AWS:

You recently reached a limit on the volume of email you were able to send out of SMTP port 25 on your instance…In order to maintain the quality of EC2 addresses for sending email, we enforce default limits on the amount of email that can be sent from EC2 accounts. If you wish to send larger amounts of email from EC2, you can apply to have these limits removed from your account by filling out our online request form.

What’s happening here is that SMTP is automatically set to use port 25 for sending out emails.  Port 25 is an open port and ISPs, along with AWS limits the traffic on that port to avoid spammers compromising or commandeering their servers for their own pursuits.

Move SMTP to port 587

Move your SMTP outbound traffic to port 587.  Port 587 is an MSA (message submission agent) port that requires SMTP authentication.  Keep in mind that port 587 is not intended for spam either.  It is, however, a better port to use to send authorized  emails from your server to your legitimate users.

Here’s the Fix!

Here’s how you can easily start sending email traffic over port 587 and possibly get off your ISP’s blacklist.

AWS Web Servers

AWS Micro Servers Are Not Cheap But You Have Options

Amazon Web Services (AWS) has an amazing array of web servers and options for everyone (Netflix is their biggest customer).
For a new customer, the first year includes a ‘free-tier‘ which generously reduces prices, and some services are even free.

I signed up and moved to AWS a little over a year ago. You need familiarize yourself with their nomenclature and how to navigate their control panel. Once you get the hang of it, everything goes very smoothly. They even give you basic server monitoring tools for free during the first year. The free support is good too, if you’re willing to read through forums of how others solved your problems. You can pay for a higher level of support that gives you one-on-one responses.

In the back of my head I remembered that the first year was coming to a close. I thought the micro server cost (their smallest dedicated size) was under $10/month…
So, when I received an automated email from AWS reminding me that the year would end in July, I figured I’d let it continue until I had time to look into alternatives.

Well, today I had a look… They offer a very good online calculator for you to plan and know exactly what your monthly costs will be (actually, they bill in hourly segments. It actually gets that granular!). Bottom line, 1 micro (tiny) server at their Virginia location, with a dedicated ip address (“Elastic IP”) and one volume, currently costs about $18.50 / month with no long term commitments. You can even start up and terminate a server for a few hours to test something and only pay for the hours used.

There are ways to lower the annual cost for a dedicated server. For example, you can commit to a yearly contract, and pay a reduced upfront fee, and then a small monthly charge. Doing it that way drops the cost to less than half.

Here’s a link to the Amazon AWS Web Services Monthly Cost Calculator where you can calculate your own costs if you’re thinking of moving there.