Saturday, June 7, 2008

Kiteboarding at Harsens Island

Ray and I headed out to Harsens Island for Kiteboarding. Ray had been there a few times before, but it was my first time out there. I got a kick out of the car ferry. While on the island it is hard to imagine that you are just outside of Detroit... it felt more like a barrier island on the east coast. The wind was a bit to light for most of the day, but Ray was able to get up and go when it occasionally picked up.

View Larger Map

Wednesday, June 4, 2008

GeoDjango on Slicehost: Getting Started (1 of 4)


This series of posts describes a relatively inexpensive way of setting up GeoDjango on a server that is publicly accessible. Specifically, I will go over how to setup GeoDjango on a Slicehost VPS slice that is running Ubuntu 8.04 (Hardy). The posts assume you have a local computer running Ubuntu Linux.


The setup process was drawn from a variety of other sources. If you need more information on any of the topics, refer to the following pages:
This first post describes the preliminary steps of signing up for a slice and creating a SSH keypair. The subsequent posts will cover the initial configuration of Ubuntu 8.04 LTS, building/installing the necessary software, and creating a test GeoDjango project.

Several of the parameters described in this series of posts are just for demonstration purposes, and will vary for each user. For these posts the following values were used:
  • Slice IP: 11.222.333.444
  • Slice Password: secret
  • Slice Name: myslice
  • Admin user: demo

Getting a Slice

The first step is to purchase a slice from Slicehost. I signed up for the smallest size slice (256MB, 10GB storage, 100GB bandwidth, currently @ $20/month), but these instructions should work for larger slices too (although the free memory stats will differ). After signing up for a slice, you will receive an IP for your slice and a Slice Manager Password.

It's probably too early to tell, but if you find these instructions useful consider signing up using my referral link. If I get enough referral credits, I'll probably upgrade to a larger slice...

If you need to rebuild your slice (i.e. erase everything and start over) go to: and login to the SliceManager. Click on the name of your slice, then select Rebuild. Select the '"Ubuntu 8.04 LTS (hardy)" image, give it a name (such as myslice), and rebuild it.

Setting up an SSH Key Pair

The first step for setting up a secure connection between your local computer and the slice is generating a SSH Key Pair. If you end up rebuilding your slice, you can reuse the SSH key pair.

On your LOCAL COMPUTER, type in the following to generate an SSH key pair.
$ mkdir ~/.ssh
$ ssh-keygen -t rsa

This will create a public (~/.ssh/ and private (~/.ssh/id_rsa) key files. The public key will be later transferred to your slice.

Ok, that was all of the preliminaries. On to Part 2 - Ubuntu 8.04 Configuration

GeoDjango on Slicehost: Ubuntu 8.04 Configuration (2 of 4)

OS Configuration

This post goes over how to configure your Ubuntu 8.04 Slice on Slicehost.

(general reference:

Log On to your Slice as root

Note: you will need the ip and password for the slice which was provided to you when the slice was created.
ssh root@11.222.333.444

Change the root password for your slice by entering

and specifying a new password for the root user account. Logout and SSH to connect again to verify that the new password works. NOTE: If you rebuilt the slice, you may need to remove an entry from the local ~/.ssh/known_hosts file in order to connect.

Create an admin (non-root) user account

Add a user and give them a password...
adduser demo

Give the user admin privileges by opening the /etc/sudoers file
nano /etc/sudoers

and add the following to the end of the file:
demo   ALL=(ALL) ALL

SSH Configuration

(general reference: (SSH Copy section & SSH Config section))
Copy your public key from your local computer to your slice by running the following on your LOCAL COMPUTER...
scp ~/.ssh/ demo@11.222.333.444:/home/demo/

Note: you will need to provide the demo user's password that you created earlier.

Now on the SLICE computer, setup the SSH permissions...
mkdir /home/demo/.ssh
mv /home/demo/ /home/demo/.ssh/authorized_keys
chown -R demo:demo /home/demo/.ssh
chmod 700 /home/demo/.ssh
chmod 600 /home/demo/.ssh/authorized_keys

Make changes to the default SSH configuration...
nano /etc/ssh/sshd_config

Replace the content of /etc/ssh/sshd_config with the following...
# Package generated configuration file
# See the sshd(8) manpage for details

# What ports, IPs and protocols we listen for
Port 30000
# Use these options to restrict which interfaces/protocols sshd will bind to
#ListenAddress ::
Protocol 2
# HostKeys for protocol version 2
HostKey /etc/ssh/ssh_host_rsa_key
HostKey /etc/ssh/ssh_host_dsa_key
#Privilege Separation is turned on for security
UsePrivilegeSeparation yes

# Lifetime and size of ephemeral version 1 server key
KeyRegenerationInterval 3600
ServerKeyBits 768

# Logging
SyslogFacility AUTH
LogLevel INFO

# Authentication:
LoginGraceTime 120
PermitRootLogin no
StrictModes yes

RSAAuthentication yes
PubkeyAuthentication yes
#AuthorizedKeysFile     %h/.ssh/authorized_keys

# Don't read the user's ~/.rhosts and ~/.shosts files
IgnoreRhosts yes
# For this to work you will also need host keys in /etc/ssh_known_hosts
RhostsRSAAuthentication no
# similar for protocol version 2
HostbasedAuthentication no
# Uncomment if you don't trust ~/.ssh/known_hosts for RhostsRSAAuthentication
#IgnoreUserKnownHosts yes

# To enable empty passwords, change to yes (NOT RECOMMENDED)
PermitEmptyPasswords no

# Change to yes to enable challenge-response passwords (beware issues with
# some PAM modules and threads)
ChallengeResponseAuthentication no

# Change to no to disable tunnelled clear text passwords
PasswordAuthentication no

# Kerberos options
#KerberosAuthentication no
#KerberosGetAFSToken no
#KerberosOrLocalPasswd yes
#KerberosTicketCleanup yes

# GSSAPI options
#GSSAPIAuthentication no
#GSSAPICleanupCredentials yes

X11Forwarding no
X11DisplayOffset 10
PrintMotd no
PrintLastLog yes
TCPKeepAlive yes
#UseLogin no

#MaxStartups 10:30:60
#Banner /etc/

# Allow client to pass locale environment variables
AcceptEnv LANG LC_*

Subsystem sftp /usr/lib/openssh/sftp-server

UsePAM no
UseDNS no
AllowUsers demo

IP Table Configuration

Just follow the instructions on (iptables section))

Save any existing rules...
iptables-save > /etc/iptables.up.rules

Create a new list of iptable rules...
nano /etc/iptables.test.rules

Paste in the following content...

#  Allows all loopback (lo0) traffic and drop all traffic to 127/8 that doesn't use lo0
-A INPUT -i lo -j ACCEPT
-A INPUT -i ! lo -d -j REJECT

#  Accepts all established inbound connections

#  Allows all outbound traffic
#  You can modify this to only allow certain traffic

# Allows HTTP and HTTPS connections from anywhere (the normal ports for websites)
-A INPUT -p tcp --dport 80 -j ACCEPT
-A INPUT -p tcp --dport 443 -j ACCEPT

#  Allows SSH connections
-A INPUT -p tcp -m state --state NEW --dport 30000 -j ACCEPT

# Allow ping
-A INPUT -p icmp -m icmp --icmp-type 8 -j ACCEPT

# log iptables denied calls
-A INPUT -m limit --limit 5/min -j LOG --log-prefix "iptables denied: " --log-level 7

# Reject all other inbound - default deny unless explicitly allowed policy


Update the rules...
iptables-restore < /etc/iptables.test.rules

Check to see that the IP table rules were updated...
iptables -L

Save the rules...
iptables-save > /etc/iptables.up.rules

Make sure the rules are applied after a reboot...
nano /etc/network/interfaces

Add the single line after just after 'iface lo inet loopback'
auto lo
iface lo inet loopback
pre-up iptables-restore < /etc/iptables.up.rules

# The primary network interface

Reload the SSH configuration...
/etc/init.d/ssh reload

* Reloading OpenBSD Secure Shell server's configuration sshd [ OK ]

Test the login by opening up a new shell, and logging in as the admin users...
ssh -p 30000 demo@11.222.333.444
Note that if you are unable to connect, you may need to remove stored host keys in the ~/.ssh/known_hosts file. Once you verify that you can login as the admin user, you can close the connection that uses the root login.

Updates to the OS

To get a more useful bash shell that the default, edit the ~/.bashrc file...
nano ~/.bashrc
and add the following to the end
export PS1='\[\033[0;35m\]\h\[\033[0;33m\] \w\[\033[00m\]: '
alias free='free -m'
alias update="sudo aptitude update"
alias install="sudo aptitude install"
alias upgrade="sudo aptitude safe-upgrade"
alias remove="sudo aptitude remove"

Reload the configuration changes for the bash shell...
source ~/.bashrc

Update the OS with any released patches...
sudo aptitude update

Set the locale...
sudo locale-gen en_US.UTF-8
sudo /usr/sbin/update-locale LANG=en_US.UTF-8

Update and upgrade...
sudo aptitude safe-upgrade
sudo aptitude full-upgrade

Install the package of essential build programs...
sudo aptitude install build-essential

Ok, now the OS is configured and updated... you are now ready to proceed to Part 3 - installing the software applications.

GeoDjango on Slicehost: Installing Software (3 of 4)

This page goes through installing the necessary software applications for running GeoDjango.


Python 2.5 is installed by default with Ubuntu 8.04. Ok, that was easy, lets move on to something else...

Apache HTTP Server


Install the Apache packages...
sudo aptitude install apache2 apache2.2-common apache2-mpm-prefork apache2-utils libexpat1 ssl-cert python-central libapache2-mod-python

Edit the Apache configuration file...
sudo nano /etc/apache2/apache2.conf

Add this line to the end of apache2.conf (replace 'myslice' with whatever you named your slice)
ServerName myslice

Restart Apache...
sudo apache2ctl graceful
sudo /etc/init.d/apache2 restart

The Apache Web server should have started. Verify this by browsing to http://11.222.333.444
You should see the following message...
It Works!


Install the subversion application from the repository...
sudo apt-get install subversion


Checkout the GeoDjango code...
svn co ~/django_gis

Setup a symbolic links so that Python knows how to find the GeoDjango code and the shell knows how to find the Django admin script.
sudo ln -s ~/django_gis/django /usr/lib/python2.5/site-packages/django
sudo ln -s ~/django_gis/django/bin/ /usr/local/bin

Test that the Django Admin script can be found by entering the following... help
... which should return a help message.


Install the PostgreSQL packages...
sudo aptitude install postgresql python-psycopg2 postgresql-server-dev-8.3 flex

Test to see that PostgreSQL is working...
sudo -u postgres psql postgres

Welcome to psql 8.3.1, the PostgreSQL interactive terminal. Type: \copyright for distribution terms \h for help with SQL commands \? for help with psql commands \g or terminate with semicolon to execute query \q to quit
postgres=# \q

Next we need to alter the default configuration to allow django to connect to the postgres server.
sudo nano /etc/postgresql/8.3/main/pg_hba.conf

Update the access records to the following...
# Database administrative login by UNIX sockets
local   all         postgres                          ident sameuser

# "local" is for Unix domain socket connections only
#local   all         all                               ident sameuser
local   all     all     password

# IPv4 local connections:
#host    all         all          md5
host    all     all password

# IPv6 local connections:
#host    all         all         ::1/128               md5

Restart the PostgreSQL server, so that it loads the new access configuration...
sudo /etc/init.d/postgresql-8.3 restart

GEOS, proj4, PostGIS, and GDAL

(Refs: and

First, make a directory for downloading source code...
mkdir ~/downloads

The following commands download, build, and install the GEOS, proj4, PostGIS, and GDAL libraries. Run them all at once if you are feeling lucky, or one at a time if you want to watch the messages. Time to get a cup of coffee... this takes a while.
cd ~/downloads
tar xjf geos-3.0.0.tar.bz2
cd geos-3.0.0
sudo make install
cd ~/downloads
tar xzf proj-4.6.0.tar.gz
cd ~/downloads/proj-4.6.0/nad
tar xzf ../../proj-datumgrid-1.3.tar.gz
cd ~/downloads/proj-4.6.0
sudo make install
cd ~/downloads
tar xzf postgis-1.3.3.tar.gz
cd postgis-1.3.3
sudo make install
cd ~/downloads
tar xzf gdal-1.5.1.tar.gz
cd gdal-1.5.1
./configure --datadir=/usr/local/share
sudo make install

The GEOS library is now available in /usr/local/include/
The Proj4 files are now in /usr/local/bin/proj/

Update the shared library configuration file...
sudo nano /etc/

and add the following line to the end

Update the shared libraries...
sudo /sbin/ldconfig

Test that GDAL is installed correctly...

Usage: gdalinfo [--help-general] [-mm] [-stats] [-nogcp] [-nomd] [-noct] [-checksum] [-mdd domain]* datasetname

Test that Python, Django, GDAL, and GEOS are all working together

Start up Python...

Python 2.5.2 (r252:60911, Apr 21 2008, 11:17:30) [GCC 4.2.3 (Ubuntu 4.2.3-2ubuntu7)] on linux2 Type "help", "copyright", "credits" or "license" for more information.

Check to see if Python can use the GDAL library...
>>> from django.contrib.gis.gdal import HAS_GDAL
>>> print HAS_GDAL

>>> from django.contrib.gis.tests import test_gdal >>>
....................... BEGIN - expecting IllegalArgumentException; safe to ignore. ERROR 1: IllegalArgumentException: points must form a closed linestring END - expecting IllegalArgumentException; safe to ignore. ...................... ---------------------------------------------------------------------- Ran 45 tests in 0.094s OK

Now check to see if Python can use the GEOS library...
>>> from django.contrib.gis.tests import test_geos

Testing WKT output. ... ok Testing HEX output. ... ok Testing KML output. ... ok Testing the Error handlers. ... BEGIN - expecting GEOS_ERROR; safe to ignore. GEOS_ERROR: ParseException: Expected number but encountered ',' GEOS_ERROR: ParseException: Unknown WKB type 255 END - expecting GEOS_ERROR; safe to ignore. GEOS_ERROR: ParseException: Unexpected EOF parsing WKB ok Testing WKB output. ... ok Testing creation from HEX. ... ok Testing creation from WKB. ... ok Testing EWKT. ... ok Testing GeoJSON input/output (via GDAL). ... ok Testing equivalence with WKT. ... ok Testing Point objects. ... ok Testing MultiPoint objects. ... ok Testing LineString objects. ... ok Testing MultiLineString objects. ... ok Testing LinearRing objects. ... ok Testing Polygon objects. ... ok Testing MultiPolygon objects. ... BEGIN - expecting GEOS_NOTICE; safe to ignore. GEOS_NOTICE: Duplicate Rings at or near point 60 300 END - expecting GEOS_NOTICE; safe to ignore. ok Testing Geometry __del__() on rings and polygons. ... ok Testing Coordinate Sequence objects. ... ok Testing relate() and relate_pattern(). ... ok Testing intersects() and intersection(). ... ok Testing union(). ... ok Testing difference(). ... ok Testing sym_difference(). ... ok Testing buffer(). ... ok Testing the SRID property and keyword. ... ok Testing the mutability of Polygons and Geometry Collections. ... ok Testing three-dimensional geometries. ... ok Testing the distance() function. ... ok Testing the length property. ... ok Testing empty geometries and collections. ... ok Testing `ogr` and `srs` properties. ... ok Testing use with the Python `copy` module. ... ok Testing `transform` method. ... ok Testing `extent` method. ... ok Testing pickling and unpickling support. ... ok ---------------------------------------------------------------------- Ran 36 tests in 0.246s OK

If the tests ran correctly, you now have GeoDjango installed. You can now proceed to Part 4 - Creating a GeoDjango project.

GeoDjango on Slicehost: Creating a GeoDjango Project (4 of 4)

This final post goes over how to create a test GeoDjango project.

Creating a Website Directory Structure

Create a location for the website directory structures.
mkdir ~/website

Create a directory structure for the website.
mkdir -p ~/website/{public,private,log,cgi-bin,backup}

Create a GeoDjango Project

Create a default Django project...
cd ~/website/ startproject test_geodjango

Add the GDAL library path to the Django project settings...
nano ~/website/

Add the following line so your Django project can found the GDAL library...
GDAL_LIBRARY_PATH = '/usr/local/lib/'

Test out the project (using the internal web server)...
python ~/website/ runserver 8001

You should see a response like the following...
Validating models... 0 errors found Django version 0.97-pre-SVN-7547, using settings 'test_geodjango.settings' Development server is running at Quit the server with CONTROL-C.

Even though the server is running, you would be able to see a Django page since port 8001 is not open. Quit the server for now...

Create a new view

Create a new view file for the project...
nano ~/website/

and insert the following code
from django.http import HttpResponse
import datetime

def current_datetime(request):
now =
html = "It is now %s." % now
return HttpResponse(html)

Edit the URL configuration file
nano ~/website/

Replace the content with the following code which will be used to display a Django test page...
from django.conf.urls.defaults import *
from test_geodjango.views import current_datetime

urlpatterns = patterns('',
(r'^time/$', current_datetime),

Configure Apache to recognize the new virtual host
sudo nano /etc/apache2/sites-available/

Add the following to configure the virtual host
# Place any notes or comments you have here
# It will make any customization easier to understand in the weeks to come

# domain:
# public: /home/demo/website/

# Admin email, Server Name (domain name) and any aliases

# Index file and Document Root (where the public files are located)
#DirectoryIndex index.html
DocumentRoot /home/demo/website/

Order deny,allow
Allow from all

SetHandler python-program
PythonPath "['/home/demo/website/'] + sys.path"
PythonHandler django.core.handlers.modpython
SetEnv DJANGO_SETTINGS_MODULE test_geodjango.settings
PythonDebug On

# Custom log file locations
LogLevel warn
ErrorLog /home/demo/website/
CustomLog /home/demo/website/ combined

Enable the site and reload apache to use the new configuration...
sudo a2ensite
sudo /etc/init.d/apache2 reload
sudo apache2ctl graceful

On your local computer, edit the /etc/hosts file and add the following line to setup an override

Test it out, by opening up a Web browser and go to You should see a page that shows something like...
It is now 2008-05-24 11:22:33.991752.

Create a PostGIS database

Next we will create a PostGIS database for the project... (I have included the shell prompts to make it more clear what user/context is running the command.)
myslice ~: sudo su - postgres
postgres@myslice:~$ createuser -SDRP testuser
postgres@myslice:~$ createdb -O testuser test_geodjango_db
postgres@myslice:~$ createlang plpgsql test_geodjango_db
postgres@myslice:~$ psql -d test_geodjango_db -f /usr/share/lwpostgis.sql
postgres@myslice:~$ psql -d test_geodjango_db -f /usr/share/spatial_ref_sys.sql
postgres@myslice:~$ psql test_geodjango_db
test_geodjango_db=# ALTER TABLE geometry_columns OWNER TO testuser;
test_geodjango_db=# ALTER TABLE spatial_ref_sys OWNER TO testuser;
test_geodjango_db=# \q
postgres@myslice:~$ exit
myslice ~:

Configure the Django database configuration

Edit the settings file for the Django project...
nano  ~/website/

And update the database connection parameters
DATABASE_ENGINE = 'postgresql_psycopg2'
DATABASE_NAME = 'test_geodjango_db'
DATABASE_USER = 'testuser'

Test the database connection

Open up a Python shell for the project...
cd ~/website/
python shell

Python 2.5.2 (r252:60911, Apr 21 2008, 11:17:30)
[GCC 4.2.3 (Ubuntu 4.2.3-2ubuntu7)] on linux2
Type "help", "copyright", "credits" or "license" for more information.

>>> from django.db import connection
>>> cursor = connection.cursor()

If the cursor = connection.cursor() command did not produce any response, then the database connection was successfully made.

Add a Django App and Define a Model

(see and
Create a Django application and edit the model configuration file...
python startapp testapp
nano testapp/

And replace with the following...
from django.contrib.gis.db import models

class District(models.Model):
name = models.CharField(max_length=35)
num  = models.IntegerField()
poly = models.PolygonField()
objects = models.GeoManager()

class School(models.Model):
name  = models.CharField(max_length=35)
point = models.PointField()
objects = models.GeoManager()

Install the Django App

Edit the settings configuration file...

and edit the INSTALLED_APPS section as follows

Sync the database with the Django model...
python syncdb

which should produce the following...
Creating table testapp_school
Creating table testapp_district
Installing custom SQL for testapp.School model
Installing custom SQL for testapp.District model

Test access to the database from the command line...
python shell
>>> from testapp.models import District, School
>>> d1 = District(name='Test District 1' , num=1, poly='POLYGON((-96 29,-95 29,-95 30,-96 29))' )
>>> qs1 = District.objects.filter(poly__bbcontains='POINT(-95.362293 29.756539)')
>>> qs1

Python should echo back a District object. This returns an array with a single district record, since the district polygon's bounding box contains the point.

Try again a query that does not return any points (since the point is not contained within the district polygon)...
>>> qs2 = District.objects.filter(poly__contains='POINT(-95.362293 29.756539)')
>>> qs2


Displaying GeoDjango Results on a Web Page

Create another view file for the project...
nano ~/website/

and add a view that returns the results of a GeoDjango query...
from django.http import HttpResponse
import datetime
from models import District, School

def test_geodjango_view(request):
now =
pointWkt = 'POINT(-95.362293 29.056539)'
districtSet = District.objects.filter(poly__contains=pointWkt)
html = "It is now %s." % now
html += "

if len(districtSet)>0:
for district in districtSet:
html += "%s is in %s" % (pointWkt,
html += "%s is not within any districts" % pointWkt
return HttpResponse(html)

Update the URL configuration...
nano ~/website/

to match the following...
from django.conf.urls.defaults import *
from test_geodjango.views import current_datetime
from test_geodjango.testapp.views import test_geodjango_view

urlpatterns = patterns('',
(r'^time/$', current_datetime),
(r'^test/$', test_geodjango_view),

Reload the Apache configuration...
sudo /etc/init.d/apache2 reload
sudo apache2ctl graceful

If you get a message saying that httpd is not running, try reloading the Apache configuration again...
sudo /etc/init.d/apache2 reload
sudo apache2ctl graceful

Test out the web page...

Point a browser at:
You should see something similar to
It is now 2008-06-04 21:26:17.022444.

POINT(-95.362293 29.056539) is in Test District 1

if that works, you now have a working (although trivial) GeoDjango website served via Apache. Now its up to you to make do something interesting. Good luck...