Fear and Loathing on the Learning Curve: Observations on Life, Tech and Web Design from a Slightly Misanthropic Mind

Your first Heroku Django app

I few weeks ago I decided to rewrite an old PHP site in Python + Django, and settled on host­ing it on Heroku, as I didn’t have time to go through the rig­mar­ole of set­ting up my own VPS just to demo the site. Heroku is a fant­astic plat­form for app host­ing but there are a few gotchas that I ran into on the way which I thought I’d share.

A quick over­view of the plat­form, then, in case you’re not famil­iar with it – Heroku is a cloud host­ing ser­vice for applic­a­tions, rather than serv­ers, and sup­ports sev­eral soft­ware stacks, Python among the new­est added. It plugs into your git work­flow so you can deploy with a simple git push, and your app gets com­piled down into a runtime known as a “slug” which Heroku serves from its net­work of app serv­ers. Nearly all configuration/management of your app is done from your local com­mand line via the heroku “tool­belt” app. The pri­cing struc­ture is such that you can run a simple app with a small data­base for free.

Your applic­a­tion slug is read-only, and each of Heroku’s web server pro­cesses (known as “dynos”) runs in its own isol­ated envir­on­ment so your app itself must be state­less – you can only per­sist stuff to data­base, any­thing else (e.g. uploaded media files) must be writ­ten to and served from another envir­on­ment, such as S3. You can serve static files (e.g. JS, CSS, images) from your app, but you won’t be able to change them without doing a new deployment.

The first issue I encountered: Heroku detects your app’s soft­ware stack by look­ing for sev­eral com­mon files, and in the case of Django apps it’s look­ing for settings.py. However in my Django pro­ject struc­ture I like to main­tain sep­ar­ate con­figs for com­mon, local and pro­duc­tion in sep­ar­ate dir­ect­or­ies, so I had to expli­citly tell Heroku where to find my set­tings by set­ting the DJANGO_SETTINGS_MODULE envir­on­ment vari­able thusly: heroku config:add DJANGO_SETTINGS_MODULE=myapp.config.heroku.settings (repla­cing myapp with your app name).

Heroku expects your Python app to use virtualenv, and also pip to install and man­age packages.

I found that in order to run my app scripts on the remote side I also needed to update my Python path, which requires set­ting another envir­on­ment vari­able: heroku config:add PYTHONPATH=/app:/app/myapp

Next I needed to set up what Heroku calls the “shared data­base” addon, which is basic­ally a 5MB free PostgreSQL instance that your app can have access to. It took me a while to fig­ure out that this isn’t provided by default, you have to enable it by run­ning heroku addons:add shared-database . Heroku addons provide their access details via envir­on­ment vari­ables, so once you’ve enabled the shared data­base you’ll find (via heroku config, which lists them) that you’ve got a new vari­able called SHARED_DATABASE_URL con­tain­ing the URL and login details for your app’s PostgreSQL db (you also need psycopg2 installed, via pip install psycopg2).

Because Heroku’s stack detec­tion magic didn’t like my app struc­ture, I found I also needed to manu­ally add some bits to my set­tings file to intro­spect the envir­on­ment and set up the data­base con­nec­tion based on the SHARED_DATABASE_URL var. The code is here (at the bot­tom). A normally-structured Django app will have this code auto­mat­ic­ally added dur­ing deploy­ment, but mine didn’t.

I also had to manu­ally tell South (the migra­tion lib­rary I was using) to use the PostgreSQL backend, by adding the fol­low­ing to my pro­duc­tion set­tings file: SOUTH_DATABASE_ADAPTERS = { 'default': "south.db.postgresql_psycopg2" } .

The final big obstacle came when deploy­ing – I run collectstatic on my Django apps to gather all my static files together into one loc­a­tion, and this wasn’t work­ing on Heroku. It turns out that the collectstatic com­mand runs in its own vir­tual envir­on­ment, sep­ar­ate from the one in which the web server dyno runs, so the lat­ter never has access to the col­lec­ted files (the applic­a­tion slug is read-only, remem­ber?). I found this help­ful link on the sub­ject, which explains how to com­bine the collectstatic and guni­corn start com­mands into one in your Procfile (which tells Heroku how to run your app) so they share an envir­on­ment. Check the com­ments on that post for details on how to adjust your urls.py file to prop­erly point to the static files too.

Worth not­ing that in that pre­vi­ous link, the Procfile com­mand to start guni­corn refers to a $PORT var – this is present in the web server dyno envir­on­ment: you have to bind to the port they spe­cify, or noth­ing will work – but this is pretty much handled for you.

To get media files work­ing prop­erly and serving from S3, I used django-mediasync to copy my local media files to my S3 bucket (one time only), then used these set­tings in my pro­duc­tion settings.py to hook it all up, via django-storages and boto. Note the last line, which instructs easy_thumbnails (if you’re using it) to use the same stor­age engine as the app default. Without this, easy_thumbnails won’t be able to save gen­er­ated thumb­nails to S3. Took me a while to dia­gnose that one.

A couple of other bits: if you want to send email from your app, you’ll need to use your own server or use the free Sendgrid addon which allows up to 200 mes­sages a day, bey­ond which you’ll need to hand over some cash money. Heroku don’t provide out­go­ing email facil­it­ies themselves.

Lastly, it’s worth not­ing that Heroku’s archi­tec­ture will auto­mat­ic­ally put a run­ning app into a sleep state if it’s not accessed for a while. This can have the effect of mak­ing the first request in a while quite slow. I got around this by set­ting up a free Pingdom account to ping my app every few minutes, which keeps it alive.

Hope this sheds a little light on some of the less-documented aspects of run­ning Python/Django apps on Heroku. It’s a great plat­form and I found get­ting to grips with it to be a really use­ful learn­ing exper­i­ence. Good luck!