Ubuntu How to Change Color Linux Directory Listings

Here’s another change I automatically make when installing a new Linux Ubuntu Server.
I’m using Bash prompt, and that’s the standard prompt for all Debian (Ubuntu, Mint, etc…) installations, so the chances are that you are using Bash too.

Problem:
The standard default color scheme for Linux directory listings (when you type in ls at the linux prompt) is hard to read. If you’re using putty on a Windows machine to connect to a linux server, the color scheme is dreadful. The deep blue is simply illegible for me.
Continue reading “Ubuntu How to Change Color Linux Directory Listings”

Ubuntu Server How to Configure for Sendmail Port 587

Avoid AWS Blocking your Server from sending emails!
I was using AWS Amazon’s Web Service and one of my servers was set to reply to users contacting with a generic response. I had configured Sendmail and tested everything was working fine.

A day later, I got a message from AWS:

You recently reached a limit on the volume of email you were able to send out of SMTP port 25 on your instance…In order to maintain the quality of EC2 addresses for sending email, we enforce default limits on the amount of email that can be sent from EC2 accounts. If you wish to send larger amounts of email from EC2, you can apply to have these limits removed from your account by filling out our online request form.

What’s happening here is that SMTP is automatically set to use port 25 for sending out emails.  Port 25 is an open port and ISPs, along with AWS limits the traffic on that port to avoid spammers compromising or commandeering their servers for their own pursuits.

Move SMTP to port 587

Move your SMTP outbound traffic to port 587.  Port 587 is an MSA (message submission agent) port that requires SMTP authentication.  Keep in mind that port 587 is not intended for spam either.  It is, however, a better port to use to send authorized  emails from your server to your legitimate users.

Here’s the Fix!

Here’s how you can easily start sending email traffic over port 587 and possibly get off your ISP’s blacklist.

Continue reading “Ubuntu Server How to Configure for Sendmail Port 587”

Ubuntu Server Setup – Top 5 things to do on the First Day

You’ve just unwrapped your new Ubuntu server running on Ubuntu 12.04 LTS and now it’s time to roll up your sleeves and get started. What should you do first? Here’s a checklist of the top things I do for each new Ubuntu server that I install.

Let’s approach it from two ways:
– You’re working with a managed network provider like Rackspace, etc… or
– You’re doing it yourself on AWS as a new EC2 instance (Similar to managed, with some caveats).

You have your server credentials and have logged on to your server with a SSH connection. You are staring at your command prompt and are ready to begin…
1. Check your server to make sure you got what you wanted

Are the drives partitioned right? Do you have any space on /var to work?

df -h

What release are you running? The current (as of 2/13) latest LTS version of Ubuntu is 12.04.x

lsb_release -a

2. Update your copy of distribution files.

sudo apt-get update

This will make sure that as we get started installing new software, we’ll have the latest and greatest of everything.

3. Install The LAMP stack ([Linux], Apache, MySQL, PHP) We’ll want to throw in phpmyadmin here as well. In case you aren’t familiar, phpmyadmin is great for setting up database permissions and general DB management from within a browser-based GUI.

sudo apt-get install lamp-server^ phpmyadmin

Answer Yes to the prompts as it installs each of the requested programs in the LAMP stack. You’ll need to pick out a password for MySQL and then confirm it. Later you’ll be asked for a password for phpmyadmin. Use the same password, it will make things easier for later. (You can always change your password later from within MySQL or PhpMyAdmin.

4. Install htop
htop is essentially “top” on steroids. It gives you a good visual picture of your server’s load and what processes are running. It’s sort of like the baby monitor of servers.

sudo apt-get install htop

5. Setup a test web page
You’ve installed apache, so you can test that everything is set correctly. Go to your default web root directory /var/www (this can be changed later, and we’ll show you how to do that in our next blog in this series)

cd /var/www

Now, edit the file index.html and put in some text that identifies it as your new server.

sudo vi /var/www/index.html

(or .php, or .htm depending on your configuration)

Now start or reload apache to make sure everything is running right.

sudo service apache2 reload

After you’ve saved the file, you can go to a browser window and type in your server ip address. You should see your temporary index page that you just created/edited.

Next…
Configuring Apache, setting up cron jobs, rsync to synchronize with multiple servers, setting up MySQL database connectivity, and more…
Check out our other blog posts on these topics.

Ubuntu Server AWS EC2 ELB Apache Log Fix

If you’ve launched AWS EC2 server instances behind a load balancer, then you’ve probably noticed that your Apache logs are showing the internal ip address of the load balancer for all entries.

I’ve seen a few blogs that explain that the reason for this is that the ELB load balancer is configured as a reverse proxy and the actual ip address of the origin is masked.  Aside from the cosmetics of seeing thousands of log entries all with the same ip address, this is actually bad for SEO as your sitemaps will also be showing all your visitors coming from one ip address.  Also, if you are using geoip the country filter won’t work as it will be looking at all visitors as coming from Amazon’s region that your EC2 instances are located.

Here’s the fix:
Very simple actually.  It worked for me on Ubuntu 12.04 LTS and I assume it will work on any Linux configuration.  Instead of logging the Host (which in our case is the internal ip), we want to log the X-Forwarded_for which is the origin ip for the entry.
Open your apache configuration file for edit.  On Ubuntu it’s at /etc/apache2/apache2.conf    You can also use locate to find it on your instance if it’s not on Ubuntu.  locate apache2.conf

sudo vi /etc/apache2/apache2.conf

Next within configuration file look for the section with LogFormat   There should be several lines beginning with LogFormat.  The top three should be for vhost_combined, combined, and common.   In all three lines where you see %h  (which is echoing to the apache log the host) change that to %{X-Forwarded-For}i

Before

LogFormat “%v:%p %h %l %u %t \”%r\” %>s %O \”%{Referer}i\” \”%{User-Agent}i\”” vhost_combined
LogFormat “%h %l %u %t \”%r\” %>s %O \”%{Referer}i\” \”%{User-Agent}i\”” combined
LogFormat “%h %l %u %t \”%r\” %>s %O” common

After

LogFormat “%v:%p %{X-Forwarded-For}i %l %u %t \”%r\” %>s %O \”%{Referer}i\” \”%{User-Agent}i\”” vhost_combined
LogFormat “%{X-Forwarded-For}i %l %u %t \”%r\” %>s %O \”%{Referer}i\” \”%{User-Agent}i\”” combined
LogFormat “%{X-Forwarded-For}i %l %u %t \”%r\” %>s %O” common

Then remember to reload apache

sudo service apache2 reload

Note of Caution: It’s relatively easy for someone to spoof the header and change the X-Forwarded-For to whatever address they want. That would mask from you their actual ip. Therefore, this solution is not intended for applications that have security implications.

Another tip. Within your web application if you are using the ip for any purpose, like Maxmind’s geoip, you’ll need to echo the X-Forwarded-For instead of the Remote host as well. e.g. $ip=$_SERVER[‘HTTP_X_FORWARDED_FOR’];

Let us know if you have any suggestions on how to improve this method.

Ubuntu Server – How to Sync Files Between Multiple Web Servers

Another installment in our How to Set up an Ubuntu Server Series:

Synchronizing multiple servers – You have more than one web server in your host configuration. What’s the easiest way to synch files among the multiple web servers?
Solution – rsync does the trick efficiently and easily.

This tutorial assumes you are comfortable on the linux command line.

Steps to Set up rsync for multiple servers synchronization:

Step One

First make sure rsync is installed on each server that you’ll be working with.  By typing rsync from a command prompt, you should get the rsync response asking you for parameters.  If rsync isn’t found, then install it.

sudo apt-get install rsync

Step Two

Test synchronizing files.  From a command prompt on your 2nd server, type in an rsync command to retrieve files from the 1st server.  Let’s assume you have a directory /var/www/ set up on both servers and you wish to synchronize the files from the first server to the 2nd. The first server acts as the master.  Any new files or changes to existing files on the master will overwrite existing files on the 2nd server.  Type in from the command prompt on server 2

sudo rsync -ra user@FirstServerIP:/var/www/ /var/www/

The first server in this example sends all the files in /var/www/ along with the (-r) recursive subdirectories. You can also sync only specific files, etc…
If this works, you are ready to move on to step 3. If not, check your connectivity with the FirstServer. Generally, if you can ssh to another server, you should be able to rsync with it as well. AWS EC2 Users If you are using AWS Amazon’s ec2 servers, which require a passkey, the rsync command will look like this:

rsync -rave “ssh -i /home/ubuntu/.ssh/passkey.pem” ubuntu@EC2ServerIP:/var/www/ /var/www/

Note that for AWS EC2, you will first need to copy your passkey over to your .ssh directory

Step Three

Next, let’s set this up to synchronize all the time automatically. We accomplish this with a cron job. Here again, we’re assuming you are familiar with how to set up and edit crons. From your root user (sudo su) edit your crontab

crontab -e

Then, add a line to the crontab that tells it to run the rsync command every minute (or any frequency that suits you).

* * * * * rsync -ar FirstServerIP:/var/www/* /var/www/

Test your configuration!  Go to your first server and make a change or add a file to the synchronized directory. Then go back to the 2nd server and see if the file is there. You can replicate this process to as many servers as you need, but keep in mind that frequency of sync and amount of data passing on your internal network can get costly and cause a hit on performance.

I hope this little tutorial was helpful.  Please drop me a note if you have any questions or comments.