How can I keep a production Python environment secure?

Question

Most of my work is creating websites in Django (a Python web framework) and deploying them to my own or clients' servers. I work from a virtualenv to separate site from system packages and have perhaps 60-80 packages installed in there and that lot is shared between two-dozen sites.

This obvious limitation to this approach is needing to test every site if I upgrade a package it uses. I consider that a fair trade-off for not needing to keep on top of umpteen separate virtualenvs.

And that is essentially my whole problem. How on earth are you supposed to keep on top of virtualenv deployments? People just seem to treat them like a dumping ground but if the programming universe has learnt anything this past week from the Ruby on Rails explosion, using old versions of software is unacceptable.

I have a simple script that attempts to check for current package versions with the latest pip counterpart but it's quite inacurrate. It also doesn't differentiate between security upgrades and feature upgrades (which require days of testing and fixing).

I am looking for something better. I am looking for something that can let me know if Django has a new security release out, or if something is end-of-life. I want something to help me (and other Python devops) not become the next batch of people crying after a wave of kids with scanners and scripts convert our servers into a botnet.

Does such a thing exist?

Answer 1

Major GNU/Linux distributions have specialized security teams responsible for keeping all packages in the distribution secure. If you cannot afford it to spend enough resources to match up with these teams, then (if the highest security is the goal) the best solution is probably to rely on their work and use packages from the distributions. Distributions with staged releases (such as Debian) try to patch packages, such that their dependencies do not break.

Of course, if you use the distribution's packages, you lose the flexibility of the Python virtual environment. So, this seems to be another tradeoff triangle: high security, low costs, installation flexibility. Pick two.

Answer 2

We use buildout to create deployments that are reproduceable and isolated.

A buildout acts like a virtualenv in that it lets you install eggs, keeping them separate from your Python installation, and it acts like pip in that you can control what versions are used. But buildout is also a build tool, with recipes you can make it build arbitrary parts as needed to fit your application and testing.

My virtual env does not contain anything beyond the zc.buildout egg to create new bootstraps for new projects.

A typical buildout I build has several config files, to fit development, continuous integration, and production environments, but the base.cfg invariably looks something like:

[buildout]
extends = 
     versions.cfg

newest = false

eggs = customer_project_name

[django-base]
recipe = djangorecipe
settings = settings
eggs = ${buildout:eggs}
project = customer_project_name
projectegg = ${:project}

[python]
recipe = zc.recipe.egg
interpreter = python
eggs = ${buildout:eggs}
initialization =
   import os
   os.environ['DJANGO_SETTINGS_MODULE'] = '${django:projectegg}.settings'

with a versions.cfg to pin all versions to prevent accidental upgrades.

Because we use a versions.cfg file, it is easy to parse all versions files to see what projects are affected (it's just a ConfigParser file, after all), and upgrading the package is just a question of updating those pins and re-testing the project.

Once tested, push the updated versions updates to the customer server, rerun buildout.

Note that upgrading security fixes should be a dedicated maintenance task anyway, as you need to plan, test and find a upgrade window for your customers.

Answer 3

I recommend you use Docker, it provides a good basic Security Structure, updates automatically and is absolutely portable. Deploying a Django site is a complete pain but as I said since I use Docker my development and production environment are almost identical so only push and pull =)

Some good reads on the Topic:

http://pawamoy.github.io/2018/02/01/docker-compose-django-postgres-nginx.html

https://wsvincent.com/django-docker-postgresql/

Answer 4

For getting notified when exploits exist there are RSS feeds, services and products out there that will let you know about exploits. At the very least any admin should probably be following CVE or NVD (which includes CVE).

How to manage exploits in libraries used by your application, but not in the virtualenv? For example ImageMagick, the python executable itself or a system library that python uses. I think your only option is to upgrade the platform, and then do the required vetting of the virtualenv and application.

At my work we create our own package (RPM) which includes python. We are in the same boat if there is a system library security flaw (upgrade the platform, vet the application). However application libraries or python itself are easily managed as they are in the toolchain package. We simply rev that, run it through our continuous integration build which rebuilds all the projects and generates all new packages for the upgrade toolchain.

There is no vertically integrated solution specific to python or django that I know of.

Answer 5

You should have a look at the features of requires.io. This seems what you are looking for.

It's free for any Bitbucket or GitHub-Account.

Apart from that:

I am working in a context, where I have to work with sensitive data for the public sector. And we use CentOS as our deployment platform. Mostly we rely on the security updates of the .rpmpackages which come with CentOS. And for some cases we do our own packaging of selected needed "python-packages". Our infrastructure team tracks known security issues of the self packaged stuff and does the repackaging. This is how it works for us.

---

P.S.: If you are on the container side of things, there is anchore which provides a similar looking service of scanning container images.