Django settings.py for the real world

The settings.py is the central configuration for all Django projects. In previous chapters you already worked with a series of variables in this file to configure things like Django applications, databases, templates and middleware, among other things.

Although the settings.py file uses reasonable default values for practically all variables, when a Django application transitions into the real world, you need to take into account a series of adjustments, to efficiently run the Django application, offer end users a streamlined experience and keep potentially rogue attackers in check.

Switch DEBUG to False

One of the first things that's necessary to launch a Django application into the real world is to change the DEBUG variable to False in a project's settings.py file. I've briefly mentioned in previous chapters how Django's behavior changes when switching DEBUG=False to DEBUG=True in settings.py. All these behavioral changes associated with the DEBUG variable are intended to enhance project security and performance. Table 5-1 illustrates the differences between having a project run with DEBUG=False and DEBUG=True.

Table 5-1. Django behavior differences between DEBUG=True and DEBUG=False in settings.py

As you can see in table 5-1, the changes enforced by changing DEBUG=True to DEBUG=False in settings.py are intended for publicly accessible applications (i.e. production environments). You may not like the hassle of adapting to these changes, but they are enforced to maintain a heightened level of security and maintain high performance on all Django projects that run in the real world.

Define ALLOWED_HOSTS

By default, the ALLOWED_HOSTS variable in settings.py is empty. The purpose of ALLOWED_HOSTS is to validate a request's HTTP Host header. Validation is done to prevent rogue users from sending fake HTTP Host headers that can potentially poison caches and password reset emails with links to malicious hosts. Since this issue can only present itself under an uncontrolled user environment (i.e. public/production servers), this validation is only done when DEBUG=False.

If you switch to DEBUG=False and ALLOWED_HOSTS is left empty, Django refuses to serve requests and instead responds with HTTP 400 bad request pages, since it can't validate incoming HTTP Host headers. Listing 5-1 illustrates a sample definition of ALLOWED_HOSTS.

Listing 5-1 Django ALLOWED_HOSTS definition

ALLOWED_HOSTS = [
    '.coffeehouse.com',
    '.bestcoffeehouse.com',
]

As you can see in listing 5-1, the ALLOWED_HOSTS value is a list of strings. In this case it defines two host domains, that allow bestcoffeehouse.com to act as an alias of coffeehouse.com. The leading .(dot) for each domain indicates a sub-domain is also an allowed host domain (e.g. static.coffeehouse.com or shop.coffeehouse.com is valid for .coffeehouse.com).

If you want to accept a single and fully qualified domain (FQDN) you would define ALLOWED_HOSTS=['www.coffeehouse.com'], which would only accept requests with an HTTP Host www.coffeehouse.com. In a similar fashion, if you want to bypass this security feature and accept any HTTP host value in a request -- which I don't recommend, see below -- you can define ALLOWED_HOSTS=['*'], where '*' represents a wild-card to accept any HTTP host value.

Be careful with the SECRET_KEY value

The SECRET_KEY value in settings.py is another security related variable like ALLOWED_HOSTS. However, unlike ALLOWED_HOSTS, SECRET_KEY is assigned a default value and a very long value at that (e.g. 'django-insecure-3*w@re!%88p%w%+-3^g_z=pna5zot51cfjt4t^!6=u7sp7qo1!').

The purpose of the SECRET_KEY value is to digitally sign certain data structures that are sensitive to tampering. Specifically, Django by default uses the SECRET_KEY on sensitive data structures like session identifiers, cookies and password reset tokens. But you can rely on the SECRET_KEY value to cryptographically protect any sensitive data structure in a Django project^[2].

The one thing the default data structures signed with the SECRET_KEY have in common, is they're sent to users on the wider Internet and are then sent back to the application to trigger actions on behalf of users. It's in this scenario we enter into a trust issue. Can the data sent back to the application be trusted ? What if a malicious user attempts to simulate another user's cookie or session data to hijack his access ? This is what digitally signed data prevents.

Before Django sends any of these sensitive data structures to users on the Internet, it signs them with a project's SECRET_KEY. When the data structures come back to fulfill an action, Django re-checks these sensitive data structures against the SECRET_KEY again. If there was any tampering on the data structures, the signature check fails and Django halts the process.

The only remote possibility a rogue user has to successfully pull an attack of this kind is if the SECRET_KEY is compromised -- since an attacker can potentially create an altered data structure that matches a project's SECRET_KEY. Therefore you should be careful about exposing your project's SECRET_KEY. If you suspect for any reason a project's SECRET_KEY has been compromised you should replace it immediately -- only a few ephemeral data structures (i.e. sessions, cookies) become invalid with this change, until users re-login again and the new SECRET_KEY is used to re-generate these data structures.

Define administrators for ADMINS and MANAGERS

Once a Django project is made accessible to end users, you'll want to define a set of users to notify them when certain events happen in a Django project (e.g. security issues or other critical factors). To this end, Django defines two sets of administrative groups defined in settings.py: ADMINS and MANAGERS.

The purpose of having two administrative groups in settings.py is mostly due to historical reasons in Django, where by default, users assigned to ADMINS received certain kinds of notifications and users assigned to MANAGERS received other types of notifications. However, it's often a common practice to alias the value of one group to the other (e.g. MANAGERS = ADMINS). In addition to the default events tied to each administrative group -- which I'll describe shortly -- it's also good practice to leverage these administrative groups to trigger notifications for custom events (e.g. notify administrators of user sign ups or business logic edge cases) vs. hard-coding administrative users/emails elsewhere in a project's codebase.

By default, both the ADMINS and MANAGERS variables are empty. The values assigned to both variables need to be a list of tuples, where each tuple is composed of a first value representing the name of a person and the second part of the tuple the person's email. Listing 5-2 shows a sample definition of ADMINS and MANAGERS.

Listing 5-2. Django ADMINS and MANAGERS definition

ADMINS = [('Webmaster','webmaster@coffeehouse.com'),('Administrator','admin@coffeehouse.com')]
MANAGERS = ADMINS

As you can see is listing 5-2, the ADMINS variable is assigned two tuples with different administrators. Next, you can see the MANAGERS variable is aliased with the ADMINS value. You can of course define different values for MANAGERS using the same syntax as ADMINS, but in this case, I just gave both variables the same values for simplicity.

By default, events that trigger notifications to both groups are limited and happen under certain circumstances.

By default, ADMINS are sent email notifications of errors associated with the django.request or django.security packages, if and only if DEBUG=False in settings.py and Django's AdminEmailHandler logging handler is enabled -- which it's by default -- in the LOGGING variable also in settings.py. This is a pretty narrow criteria, as it's intended to notify only the most serious errors -- for requests and security -- and only for production environments, in which DEBUG=False and default logging is set up. For no other events or conditions are the ADMINS notified by email.

By default, MANAGERS are sent email notifications of broken links (i.e. HTTP 404 page requests), if and only if DEBUG=False and the Django middleware django.middleware.common.BrokenLinkEmailsMiddleware is enabled. Because HTTP 404 page requests aren't a serious problem, by default BrokenLinkEmailsMiddleware is disabled. This is an even narrower criteria than for ADMINS, because irrespective of a project being in development (DEBUG=True) or production (DEBUG=False) the BrokenLinkEmailsMiddleware class needs to be added to MIDDLEWARE variable in settings.py for MANAGERS to get notifications. For no other events or conditions are the MANAGERS notified by email.

Now that you know the purpose of ADMINS and MANAGERS, add users and emails as you see fit to your project. Remember you can always leverage the values in ADMINS and MANAGERS for other custom logic in a Django project (e.g. notify administrators of user sign ups or business logic edge cases).

Use dynamic absolute paths

There are some Django variables in settings.py that rely on directory locations, such is the case for STATIC_ROOT which defines a consolidation directory for a project's static files or the DIRS list of the TEMPLATES variable which defines the location of a project's templates, among other variables.

The problem with variables that rely on directory locations is that if you run the project on different servers or share it with other users, it can be difficult to keep track or reserve the same directories across a series of environments. To solve this issue you can define variables to dynamically determine the absolute paths of a project. Listing 5-3 illustrates a Django project directory structure, deployed to the /www/ system directory.

Listing 5-3. Django project structure deployed to /www/

+-/www/+
       |           
       +--STORE--+
                 |
                 +---manage.py
                 |
                 +---coffeestatic--+
                 |                 |
                 |                 +-(Consolidated static resources) 
                 |
                 +---coffeehouse--+
                                  |
                                  +-__init__.py
                                  +-settings.py
                                  +-urls.py
                                  +-wsgi.py
                                  |
                                  +---templates---+
                                                  +-app_base_template.html
                                                  +-app_header_template.html
                                                  +-app_footer_template.html

Typically a Django settings.py file would define the values for STATIC_ROOT and DIRS in TEMPLATES as illustrated in listing 5-4.

Listing 5-4. Django settings.py with absolute path values

# Other configuration variables omitted for brevity
STATIC_ROOT = '/www/STORE/coffeestatic/'

# Other configuration variables omitted for brevity
TEMPLATES = [
		{
		'BACKEND': 'django.template.backends.django.DjangoTemplates',
		'DIRS': ['/www/STORE/coffeehouse/templates/',],
		}
	    ]

The issue with the setup in listing 5-4 is it will require editing if you deploy the Django application to a server where the /www/ directory isn't available (e.g. due to restrictions or a Windows OS where directories start with a leading letter C:/).

An easier approach illustrated in listing 5-5 is to define variables to dynamically determine the absolute paths of a project.

Listing 5-5. Django settings.py with dynamically determined absolute path

from pathlib import Path

BASE_DIR = Path(__file__).resolve().parent.parent
PROJECT_DIR = Path(__file__).resolve().parent

# Other configuration variables omitted for brevity
STATIC_ROOT = BASE_DIR / 'coffeestatic'

# Other configuration variables omitted for brevity
TEMPLATES = [
		{
		'BACKEND': 'django.template.backends.django.DjangoTemplates',
		'DIRS': [ PROJECT_DIR / 'templates' ],
		}
	    ]

The variables defined at the top of listing 5-5 rely on the Python pathlib module to dynamically determine the absolute system path relative to the settings.py file. The PROJECT_DIR=Path(__file__).resolve().parent statement gets translated into the /www/STORE/coffeehouse/ value, which is the absolute system directory of files like settings.py. And to access the parent of /www/STORE/coffeehouse/ you simply make an additional call to the parent method and define the BASE_DIR variable so it gets translated into the /www/STORE/ value.

The remaining statements in listing 5-5 use pathlib's string concatenation functionality to use the PROJECT_DIR and BASE_DIR values and set the absolute paths in the STATIC_ROOT and TEMPLATE_DIRS variables. In this manner you don't need to hard code the absolute paths for any Django configuration variable, the variables automatically adjust to any absolute directory irrespective of the application deployment directory.

Use multiple environments or configuration files for Django

In every Django project you'll eventually come to the realization that you have to split settings.py into multiple environments or files. This will be either because the values in settings.py need to change between development and production servers, there are multiple people working on the same project with different requirements (e.g. Windows and Linux) or you need to keep sensitive settings.py information (e.g. passwords) in a local file that's not shared with others.

In Django there is no best or standard way to split settings.py into multiple environments or files. In fact, there are many techniques and libraries to make a Django project run with a split settings.py file. Next, I'll present the three most popular options I've used in my projects. Depending on your needs you may feel more comfortable using one option over another or inclusively mixing two or all three of these techniques to achieve an end solution.

Option 1) Multiple environments in the same settings.py file with a control variable

The settings.py file is treated as an ordinary Python file, so there's no limitation to using Python libraries or conditionals to obtain certain behaviors. This means you can easily introduce a control variable based on a fixed value (e.g. server host name) to conditionally set up certain variable values.

For example, changing the DATABASES variable -- because passwords and the database name change between development and production -- changing the EMAIL_BACKEND variable -- since you don't need to send actual emails in development as you do in production -- or changing the CACHES variable -- since you don't need a cache to speed up performance in development as you need in production.

Listing 5-6 illustrates the setup of a control variable called DJANGO_HOST based on Python's socket module, the variable is then used to load different sets of Django variables based on a server's host name.

Listing 5-6 Django settings.py with control variable with host name to load different sets of variables.

from pathlib import Path

BASE_DIR = Path(__file__).resolve().parent.parent
PROJECT_DIR = Path(__file__).resolve().parent

INSTALLED_APPS = [
    'django.contrib.admin',
    'django.contrib.admindocs',
    'django.contrib.auth',
    'django.contrib.contenttypes',
    'django.contrib.sessions',
    'django.contrib.messages',
    'django.contrib.staticfiles',
]

MIDDLEWARE = [
    'django.middleware.security.SecurityMiddleware',
    'django.contrib.sessions.middleware.SessionMiddleware',
    'django.middleware.common.CommonMiddleware',
    'django.middleware.csrf.CsrfViewMiddleware',
    'django.contrib.auth.middleware.AuthenticationMiddleware',
    'django.contrib.messages.middleware.MessageMiddleware',
    'django.middleware.clickjacking.XFrameOptionsMiddleware',
]

ROOT_URLCONF = 'coffeehouse.urls'

TEMPLATES = [
    {
        'BACKEND': 'django.template.backends.django.DjangoTemplates',
        'DIRS': [ PROJECT_DIR / 'templates' ],
        'APP_DIRS': True,
        'OPTIONS': {
            'context_processors': [
                'django.template.context_processors.debug',
                'django.template.context_processors.request',
                'django.contrib.auth.context_processors.auth',
                'django.contrib.messages.context_processors.messages',
            ],
        },
    },
]

# Import socket to read host name
import socket

# If the host name starts with 'live', DJANGO_HOST = "production"
if socket.gethostname().startswith('live'):
    DJANGO_HOST = "production"
# Else if host name starts with 'test', set DJANGO_HOST = "test"
elif socket.gethostname().startswith('test'): 
    DJANGO_HOST = "testing"
else:
# If host doesn't match, assume it's a development server, set DJANGO_HOST = "development"
    DJANGO_HOST = "development"

# Define general behavior variables for DJANGO_HOST and all others
if DJANGO_HOST == "production":
    DEBUG = False
    STATIC_URL = 'https://static.coffeehouse.com/'
else:
    DEBUG = True
    STATIC_URL = 'static/'

# Define DATABASES variable for DJANGO_HOST and all others
if DJANGO_HOST == "production":
   # Use mysql for live host
   DATABASES = {
    'default': {
        'NAME': 'housecoffee',
        'ENGINE': 'django.db.backends.mysql',
        'USER': 'coffee',
        'PASSWORD': 'secretpass'
    }
  }
else: 
   # Use sqlite for non live host
   DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.sqlite3',
	'NAME': BASE_DIR / 'coffee.sqlite3',
    }
  }

# Define EMAIL_BACKEND variable for DJANGO_HOST
if DJANGO_HOST == "production":
    # Output to SMTP server on DJANGO_HOST production
    EMAIL_BACKEND = 'django.core.mail.backends.smtp.EmailBackend'
elif DJANGO_HOST == "testing":
    # Nullify output on DJANGO_HOST test
    EMAIL_BACKEND = 'django.core.mail.backends.dummy.EmailBackend'
else: 
    # Output to console for all others
    EMAIL_BACKEND = 'django.core.mail.backends.console.EmailBackend'

# Define CACHES variable for DJANGO_HOST production and all other hosts 
if DJANGO_HOST == "production":
   # Set cache
   CACHES = {
        'default': {
            'BACKEND': 'django.core.cache.backends.memcached.MemcachedCache',
            'LOCATION': '127.0.0.1:11211',
            'TIMEOUT':'1800',
            }
        }
   CACHE_MIDDLEWARE_SECONDS = 1800
else: 
   # No cache for all other hosts
   pass

The first line in listing 5-6 imports the Python socket module to gain access to the host name. Next, a series of conditionals are declared using socket.gethostname() to determine the value of the control variable DJANGO_HOST. If the host name starts with the letters live the DJANGO_HOST variable is set to "production", if the host name starts with test then DJANGO_HOST is set to "testing" and if the host name starts with neither of the previous options then DJANGO_HOST is set to "development".

In this scenario, the string method startswith is used to determine how to set the control variable based on the host name. However, you can just as easily use any other Python library or even criteria (e.g. IP address) to set the control variable. In addition, since the control variable is based on a string, you can introduce as many configuration variations as needed. In this case we use three different variations to set settings.py variables -- "production","testing" and "development" -- but you could easily define five or a dozen variations if you require such an amount of different set ups.

Option 2) Multiple environment files using configparser

Another variation to split settings.py is to rely on Python's built-in configparser module. configparser allows Django to read configuration variables from files that use a data structure similar to the one used in Microsoft Windows INI files. Listing 5-7 illustrates a sample configparser file.

Listing 5-7. Python configparser sample file production.cfg.

[general]
DEBUG: false
STATIC_URL: https://static.coffeehouse.com/
[databases]
NAME: housecoffee
ENGINE: django.db.backends.mysql 
USER: coffee
PASSWORD: secretpass 
[security]
SECRET_KEY: django-insecure-3*w@re!%%88p%%w%%+-3^g_z=pna5zot51cfjt4t^!6=u7sp7qo1!

As you can see in Listing 5-7, the format for a configparser file is structured in various sections declared between brackets (e.g. [general], [databases]) and below each section are the different keys and values. The variables in listing 5-7 are used for a production environment placed in a file named production.cfg. I chose the .cfg extension for this file, but you can use the .config or .ini extensions if you like, the extension is irrelevant to Python, the only thing that matters is the data format in the file itself.

Similar to the contents in production.cfg, you can create other files with different variables for other environments (e.g.testing.cfg, development.cfg). Once you have the configparser file or files, then you can import them into a Django settings.py. Listing 5-8 shows a sample settings.py that uses values from a configparser file.

Listing 5-8. Django settings.py with configparser import.

from pathlib import Path

BASE_DIR = Path(__file__).resolve().parent.parent
PROJECT_DIR = Path(__file__).resolve().parent

INSTALLED_APPS = [
    'django.contrib.admin',
    'django.contrib.admindocs',
    'django.contrib.auth',
    'django.contrib.contenttypes',
    'django.contrib.sessions',
    'django.contrib.messages',
    'django.contrib.staticfiles',
]

MIDDLEWARE = [
    'django.middleware.security.SecurityMiddleware',
    'django.contrib.sessions.middleware.SessionMiddleware',
    'django.middleware.common.CommonMiddleware',
    'django.middleware.csrf.CsrfViewMiddleware',
    'django.contrib.auth.middleware.AuthenticationMiddleware',
    'django.contrib.messages.middleware.MessageMiddleware',
    'django.middleware.clickjacking.XFrameOptionsMiddleware',
]

ROOT_URLCONF = 'coffeehouse.urls'

TEMPLATES = [
    {
        'BACKEND': 'django.template.backends.django.DjangoTemplates',
        'DIRS': [ PROJECT_DIR / 'templates' ],
        'APP_DIRS': True,
        'OPTIONS': {
            'context_processors': [
                'django.template.context_processors.debug',
                'django.template.context_processors.request',
                'django.contrib.auth.context_processors.auth',
                'django.contrib.messages.context_processors.messages',
            ],
        },
    },
]

# Access configparser to load variable values
import configparser
config = configparser.SafeConfigParser(allow_no_value=True)

# Import socket to read host name
import socket

# If the host name starts with 'live', load configparser from "production.cfg"
if socket.gethostname().startswith('live'):
    config.read(PROJECT_DIR / 'production.cfg')
# Else if host name starts with 'test', load configparser from "testing.cfg"
elif socket.gethostname().startswith('test'):
    config.read(PROJECT_DIR / 'testing.cfg')
else:
# If host doesn't match, assume it's a development server, load configparser from "development.cfg"
    config.read(PROJECT_DIR / 'development.cfg')

DEBUG = config.get('general', 'DEBUG')
STATIC_URL = config.get('general', 'STATIC_URL')
DATABASES = {
    'default': {
        'NAME': config.get('databases', 'NAME'),
        'ENGINE': config.get('databases', 'ENGINE'),
        'USER': config.get('databases', 'USER'),
        'PASSWORD': config.get('databases', 'PASSWORD')
    }
  }

SECRET_KEY = config.get('security', 'SECRET_KEY')

As you can see in Listing 5-8, configparser is loaded into Django via django.utils.six.moves, which is a utility to allow cross-imports between Python 2 and Python 3. In Python 2 the configparser package is actually named ConfigParser, but this utility allows us to use the same import statement using either Python 2 and Python 3. After the import, we use the SafeConfigParser class with the argument allow_no_value=True to allow processing of empty values in configparser keys.

Then we rely on the same prior technique using Python's socket module to gain access to the host name and determine which configparser file to load. The configparser file is loaded using the read method of the SafeConfigParser instance. At this juncture all configparser variables are loaded and ready for access. The remainder of listing 5-8 shows a series of standard Django settings.py variables that are assigned their value using the get method of the SafeConfigParser instance, where the first argument is the configparser section and the second argument is the key variable.

So there you have another option on how to split the variables in settings.py into multiple environments. Like I mentioned at the start, there's no best or standard way of doing this. Some people like configparser better because it splits values into separate files and avoids the many conditionals of option 1, but other people can hate configparser because of the need to deal with the special syntax and separate files. Choose whatever feels best for your project.

Option 3) Multiple settings.py files with different names for each environment

Finally, another option to split Django variables into multiple environments is to create multiple settings.py files with different names. By default, Django looks for configuration variables in the settings.py file in a project's base directory.

However, it's possible to tell Django to load a configuration file with a different name. Django uses the operating system(OS) variable DJANGO_SETTINGS_MODULE for this purpose. By default, Django sets this OS variable to <project_name>.settings in the manage.py file located in the base directory of any Django project. And since the manage.py file is used to bootstrap Django applications, the DJANGO_SETTINGS_MODULE value in this file guarantees configuration variables are always loaded from the settings.py file inside the <project_name> sub-directory.

So let's suppose you create different settings.py files for a Django application -- placed in the same directory as settings.py -- named production.py, testing.py and development.py. You have two options to load these different files.

One option is to change the DJANGO_SETTINGS_MODULE definition in a project's manage.py file to the file with the desired configuration (e.g. os.environ.setdefault("DJANGO_SETTINGS_MODULE", "coffeehouse.production") to load the production.py configuration file). However, hard coding this value is inflexible because you would need to constantly change the value in manage.py based on the desired configuration. Here you could use a control variable in manage.py to dynamically determine the DJANGO_SETTINGS_MODULE value based on a host name -- similar to the process described in the previous option 1 for settings.py.

Another possibility to set DJANGO_SETTINGS_MODULE without altering manage.py is to define DJANGO_SETTINGS_MODULE at the OS level so it overrides the definition in manage.py. Listing 5-9 illustrates how to set the DJANGO_SETTINGS_MODULE variable on a Linux/Unix OS so that application variables in the testing_settings.py file are used instead of the settings.py file.

Listing 5-9. Override DJANGO_SETTINGS_MODULE to load application variables from a file called testing.py and not the default settings.py

$ export DJANGO_SETTINGS_MODULE=coffeehouse.testing_settings
$ python manage.py runserver
System check identified no issues (0 silenced).
Django version 4.0.1, using settings 'coffeehouse.testing_settings'
Starting development server at http://127.0.0.1:8000/
Quit the server with CONTROL-C.

In listing 5-9 we use the standard Linux/Unix syntax export variable_name=variable_value to set an environment variable. Once this is done, notice the Django application that uses the development server displays the start-up message "using settings 'coffeehouse.testing_settings'".

If you plan to override the DJANGO_SETTINGS_MODULE at the OS level to load different Django application variables, be aware that by default OS variables aren't permanent or inherited. This means you may need to define the DJANGO_SETTINGS_MODULE for every shell from which you start Django and also define it as a local variable for run-time environments (e.g. Nginx, Apache).