Extending Django Settings for the Real World

A basic Django installation keeps its global variables in a file called settings.py. This is perfect for simple deployment because it allows the developer to overwrite Django variables like INSTALLED_APPS or SESSION_ENGINE very easily. You simply update the variable like so:

SESSION_ENGINE = 'django.contrib.sessions.backends.cache'

From within the shell, you can see the result:

./manage.py shell
>>> from django.conf import settings
>>> settings.SESSION_ENGINE
'django.contrib.sessions.backends.cache'

Many people have two environments in which they work, and therefore a typical settings.py file will have something like this at the end:

try:
    from local_settings import *
except ImportError:
    pass

This overwrites variables from a file called local_settings.py, overriding any existing variables in the settings.py file. Try it. Add the import code above into your settings.py file and create a new file called local_settings.py in the same directory as the settings.py file and add this to it:

SESSION_ENGINE = 'django.contrib.sessions.backends.cached_db'

Now, if you enter the shell like you did above and request settings.SESSION_ENGINE, you’ll get ‘django.contrib.sessions.backends.cache’. This is very handy because, in a typical situation, you can have a settings.py file which works for all your environments and then have a local_settings.py file for each environment that overrides the variable values.

Problems with the Standard Settings files

Unfortunately, in this scenario, variables from the settings.py file cannot be interpreted in the local_settings.py file and therefore, you couldn’t do something like this:

INSTALLED_APPS += ('debug_toolbar',)

In this situation, you’ll get a NAME_ERROR in which INSTALLED_APPS is undefined rather than  (‘django.contrib.auth’,’debug_toolbar’,).

A Modest Proposal

What we do at Yipit is to put all of our variables in a settings directory:

settings/
__init__.py (where the variable for all environments are)
active.py (optional - defines the environment we're in - not under version control)
development.py (shared by all the development environments)
production.py (live site)

This allows us to create an init.py file for all the variables that are the same across all environments. The init.py file requires no imports (except whatever you may need from Python itself, or other libraries). Then, each file imports from init.py in the way you might imagine:

production.py:

from settings import *
#Alter or add production specific variables

development.py:

from settings import *
#Alter or add development specific variables

active.py:

from settings.development import *
#This file denotes which environment we're in.
#This active.py file creates a development environment

Note: If you’re not that familiar with Python, ‘from settings’ accesses settings/init.py.

In more complex scenarios, you may also want to inherit settings from files other than setting/init.py, and this system fully supports that option. For example, you may have a settings/staging.py files that pulls from settings/init.py and then settings/development.py could pull from staging. It’s really up to you.

This approach has some shortcomings, notably that you can’t dynamically change variables - but that’s really not the point of settings. Now, you can change variables on a per-environment basis like this (in, say, development.py):

INSTALLED_APPS += ('debug_toolbar',)

Which will set INSTALLED_APPS as (‘django.contrib.auth’,’debug_toolbar’,). Here is our manage.py file:

#!/usr/bin/env python
import sys
import traceback
from os.path import abspath, dirname, join

from django.core.management import execute_manager

SETTINGS_ACTIVE_CONTENTS = "\033[1;32mfrom settings.local import *\033[1;33m"
try:
    from settings import active as settings
except ImportError, e:
    print '\033[1;33m'
    print "Apparently you don't have the file settings/active.py yet."
    print "Create it containing '%s'\033[0m" % SETTINGS_ACTIVE_CONTENTS
    print
    print "=" * 20
    print "original traceback:"
    print "=" * 20
    print
    traceback.print_exc(e)
    sys.exit(1)

sys.path.insert(0, abspath(join(dirname(__file__), "../")))
sys.path.insert(0, join(settings.PROJECT_ROOT, "apps"))

if __name__ == "__main__":
    execute_manager(settings)

Final Thoughts

If we make further changes to our settings configuration, we’ll do a follow-up post. Some modifications we are considering:

  • Using Chef to hold many of the systemwide configuration parameters (usernames, machine addresses, etc…) in order to move that information away from the application layer and onto the environment layer.

  • Creating an additional settings file that imports active.py for calculated settings. For example, if a read replica database has not been declared, but the application expects one, have the default database act as the read replica.

If you have a different way of handling settings, we would love to hear from you in the comments below.

Adam Nelson is CTO at Yipit.

To find out about future posts, you can follow along using

Django, Setup