User-Defined Django Settings
Smarter is a Django application, and you can set any Django setting yourself, without changing Smarter’s code: a session’s lifetime, the administrators’ email addresses, a Content Security Policy, a setting of a Django package that you add. Your settings supersede Smarter’s own values, in every environment.
Danger
A user-defined setting is a hammer. Use it with care.
It replaces Smarter’s value of that setting outright, in every environment where it applies, however carefully Smarter’s value was built. Smarter does nothing to stop you, and doesn’t warn you:
Smarter’s settings are often computed, not constant.
base.py,base_aws.pyand each environment’s module build many settings from other settings and fromsmarter_settings: the database, the caches and their separate Redis databases for sessions and Celery, storage on S3 or on disk, allowed hosts, CORS and CSRF origins, cookie security, the Content Security Policy’s sources. Your value replaces the result of all that, and nothing that was derived from the old value is computed again.Lists and dictionaries are replaced whole, not merged. A
LOGGING,CACHES,DATABASESorMIDDLEWAREthat leaves out one of Smarter’s entries removes it, with whatever it did.Only the type is checked, never the meaning.
SESSION_COOKIE_SECURE: falsein prod is a valid setting: the session cookie is then sent without HTTPS. A wrong value can stop the platform, or, worse, keep it running with weaker security, and nothing tells you so.It applies to every pod. In Kubernetes, the api, the workers, beat, and the init and migrate jobs all read the same settings.
So, before you override a setting:
Read how Smarter sets it, in
smarter/settings/, and what else is built from it.Prefer a
SMARTER_*setting where one exists (see Configuration): Smarter derives every dependent setting from it, correctly.Change as few settings as you can, and for a list or a dictionary, start from a copy of Smarter’s whole value.
Try the change locally, and then in alpha, before prod.
To undo a setting, delete it from your file or your variables: Smarter’s own value applies again.
Important
Where your settings go depends on where Smarter runs. There are two places, and each one is read in one kind of environment only:
Where Smarter runs |
Put your settings in |
Never in |
|---|---|---|
Local development, on your computer, with Docker Compose ( |
|
The Helm chart: nothing local reads it |
alpha, beta, next, prod, and every other environment on AWS (Kubernetes) |
|
|
A setting that you put in user-settings/settings.yaml changes your own computer only. To change
alpha or prod, change the Helm values that their deployment uses.
In both places, the settings are the same YAML, and the same code applies them:
smarter.settings.user. You can also set one setting at a time with an environment variable named
DJANGO_<SETTING>, in either place.
How your settings supersede Smarter’s
Django reads one settings module, named by DJANGO_SETTINGS_MODULE: smarter.settings.local on your
computer, smarter.settings.prod in prod, and so on. Each of those modules builds on the ones before it,
and your settings come last. From lowest to highest priority:
smarter/settings/base.py, Smarter’s settings for every environment.smarter/settings/base_aws.py, Smarter’s settings for the environments on AWS. Local development skips it.The environment’s own module:
local.py,alpha.py,beta.py,next.pyorprod.py.Your settings file:
user-settings/settings.yamllocally, ordjangoSettingsfrom the Helm chart.Your
DJANGO_*environment variables, e.g.DJANGO_SESSION_COOKIE_AGE=3600.
A later step wins. So your file supersedes everything that Smarter sets, in every environment, and an
environment variable supersedes your file. For example, base_aws.py sets
SESSION_COOKIE_SECURE = True for alpha and prod. With this in the Helm values:
djangoSettings:
SESSION_COOKIE_SECURE: false
prod runs with SESSION_COOKIE_SECURE = False. If prod’s environment also had
DJANGO_SESSION_COOKIE_SECURE=true, it would run with True.
Every environment’s module ends the same way: it applies your settings, then logs its configuration. So what the log shows is what Django runs with, your settings included, and no line of Smarter’s settings can undo yours.
Local development
This is for your own computer only. See the note at the top of this page.
Copy the example to the file that Smarter reads. Git ignores
settings.yaml, so your settings stay on your computer:cp user-settings/settings.example.yaml user-settings/settings.yamlUncomment and edit the settings that you want, or write your own, one setting per key:
SESSION_COOKIE_AGE: 3600 ADMINS: - you@example.com
Restart the containers that load Django’s settings:
docker compose restart smarter-api smarter-worker smarter-worker-infrastructure smarter-beat
Docker Compose mounts user-settings/ read-only at /home/smarter_user/user-settings/ in those four
containers, which is where smarter.settings.user looks for settings.yaml. A missing file is not
an error: there are simply no user settings.
For a DJANGO_* variable, add it to .env, e.g. DJANGO_SESSION_COOKIE_AGE=3600, and recreate the
containers (make run), since Docker Compose reads .env when it creates them.
The test shards (make docker-test-sharded) don’t mount user-settings/, so your settings don’t change
their results. Tests that you run in the dev containers with docker exec do read your file.
alpha, prod, and every other environment on AWS
Here your settings go in the Helm chart’s values, never in user-settings/. Write them under
djangoSettings, as a map of setting names to values, in the values file that your deployment passes to
helm upgrade (or in helm/charts/smarter/values.yaml):
djangoSettings:
SESSION_COOKIE_AGE: 3600
ADMINS:
- ops@example.com
DATA_UPLOAD_MAX_MEMORY_SIZE: 10485760
What the chart does with it:
It writes the map, as YAML, to the Kubernetes Secret
<appName>-django-settings, e.g.smarter-django-settings. It is a Secret, not a ConfigMap, so a value may be a password, e.g. a database’s inDATABASES. Anyone who may read the namespace’s Secrets may read it.It mounts the Secret read-only at
/home/smarter_user/user-settings/settings.yamlin every pod that loads Django’s settings: the api, the two Celery workers, Celery Beat, and the init and migrate jobs. That is the same path that Docker Compose uses locally, so the same code reads it.A change to
djangoSettingschanges the pods’checksum/django-settingsannotation, sohelm upgraderolls the api, worker and beat pods, which then read the new settings.
djangoSettings defaults to {}: no user settings. The values schema allows only upper case setting
names as its keys, so a misspelled session_cookie_age fails helm upgrade before anything changes.
For a DJANGO_* variable, add it to env in the same values. The chart stores env in the Secret
<appName>-env, which every pod reads as its environment:
env:
DJANGO_SESSION_COOKIE_AGE: "3600"
Note
The migrate job runs before helm upgrade applies the release’s Secrets, so it reads the previous
release’s djangoSettings. On the first upgrade to a chart that has djangoSettings at all, the
Secret doesn’t exist yet, and the migrate job runs with no user settings; its volume is optional for
exactly that reason. A migration rarely depends on a setting of yours. The init job, the api, the
workers and beat always read the new release’s settings.
DJANGO_* environment variables
Besides the file, smarter.settings.user reads every environment variable whose name starts with
DJANGO_, and sets the Django setting named by the rest of it: DJANGO_SESSION_COOKIE_AGE=3600 sets
SESSION_COOKIE_AGE. It does so in every environment, right after it applies the file, so:
a variable supersedes the same setting in your file, and every value in Smarter’s settings;
it is the quickest way to change one setting, without writing a file.
Where to set one:
Where Smarter runs |
Where the variable goes |
|---|---|
Local development |
|
alpha, prod, and every other environment on AWS |
|
A variable’s value is text, and is converted to the type of the setting’s value, as Writing values
explains: true for a boolean, 3600 for an integer, a,b or a JSON list for a list. A nested
setting, such as LOGGING, is easier to write in the file. DJANGO_SETTINGS_MODULE is Django’s own,
and is not a setting; it is left alone.
Writing values
A file’s value is YAML, and is used as it is. An environment variable’s value is text, and is converted to the type of the setting’s value before it: the value from Smarter’s settings and your file, or else Django’s own default for that setting.
Kind of value |
In the file (YAML) |
As an environment variable |
|---|---|---|
Integer |
|
|
Float |
|
|
Boolean |
|
|
String |
|
|
None |
|
|
List |
One |
Comma-separated, |
Tuple, or a list of pairs |
A list: YAML has no tuples, and Django accepts a list wherever it takes a tuple, e.g.
|
A Python literal: |
Dictionary, nested |
An indented mapping, e.g. |
A JSON object, or a Python dict for keys that aren’t strings, on one line. The file is easier |
A class or a function |
Its dotted path, as a string: |
The same dotted path |
user-settings/settings.example.yaml has a worked example of each of these, which a test checks.
A setting that Smarter doesn’t define, nor Django, has no value before yours to take a type from. An environment variable for it is a JSON value, else a Python literal, else a plain string.
Rules
A setting’s name is upper case, as Django requires:
SESSION_COOKIE_AGE, notsession_cookie_age. The name of aDJANGO_*variable is upper-cased for you.Your value replaces Smarter’s whole setting. A dictionary or a list is not merged with Smarter’s. To change one logger, write the whole
LOGGING; to add one cache, write the wholeCACHES. Copy Smarter’s value fromsmarter/settings/and edit it.Settings that Smarter computes from other settings are not computed again. For example, the Content Security Policy’s sources are built from
STATIC_URL, andSTORAGESfrom whether AWS is configured. If you changeSTATIC_URL, the policy keeps the sources of the old value; change it too.Prefer a
SMARTER_*setting where one exists. Many Django settings are derived fromsmarter_settings(see Configuration), e.g.ALLOWED_HOSTSfromSMARTER_ALLOWED_HOSTS, and everything derived from it changes with it. Use a user-defined Django setting for what Smarter doesn’t configure itself.A name that neither Django nor Smarter defines is set anyway, for a package that reads it, and the log warns about it, in case it is a typo.
A file or a variable that can’t be applied stops Django, with an error that names it: a file that isn’t YAML or isn’t a mapping, a key that isn’t a setting name, or a variable whose value isn’t of the setting’s type, e.g.
DJANGO_SESSION_COOKIE_AGE=an hour. In Kubernetes, the pods don’t start, andkubectl logsshows the error.Values are never logged. The log names each setting that you set, never its value, which may be a password.
Checking what applied
When Django loads its settings, the log has a line for your file and one for your variables, naming each setting:
smarter.settings.user.apply_user_settings() Applying 2 setting(s) from /home/smarter_user/user-settings/settings.yaml: ADMINS, SESSION_COOKIE_AGE
smarter.settings.user.apply_user_settings() Applying 1 setting(s) from DJANGO_* environment variables: SECRET_KEY
To see the value that Django runs with, ask Django, in the container or the pod:
# locally
docker exec smarter-api python manage.py diffsettings | grep SESSION_COOKIE_AGE
# on Kubernetes (prod's namespace, as an example)
kubectl exec -n smarter-platform-prod deploy/smarter-api -- python manage.py diffsettings | grep SESSION_COOKIE_AGE
Reference
smarter.settings.user, which reads and applies your settings: its technical reference.user-settings/README.mdanduser-settings/settings.example.yaml, in the repository’s root.djangoSettingsinhelm/charts/smarter/values.yaml, andtemplates/secret-django-settings.yaml.Django’s settings reference, for every setting that Django defines.