7

Most of my work is creating websites in Django (a Python web framework) and deploying them to my own or clients' servers. I work from a virtualenv to separate site from system packages and have perhaps 60-80 packages installed in there and that lot is shared between two-dozen sites.

This obvious limitation to this approach is needing to test every site if I upgrade a package it uses. I consider that a fair trade-off for not needing to keep on top of umpteen separate virtualenvs.

And that is essentially my whole problem. How on earth are you supposed to keep on top of virtualenv deployments? People just seem to treat them like a dumping ground but if the programming universe has learnt anything this past week from the Ruby on Rails explosion, using old versions of software is unacceptable.

I have a simple script that attempts to check for current package versions with the latest pip counterpart but it's quite inacurrate. It also doesn't differentiate between security upgrades and feature upgrades (which require days of testing and fixing).

I am looking for something better. I am looking for something that can let me know if Django has a new security release out, or if something is end-of-life. I want something to help me (and other Python devops) not become the next batch of people crying after a wave of kids with scanners and scripts convert our servers into a botnet.

Does such a thing exist?

  • 2
    What was the Ruby on Rails explosion? Did it involve a freight train carrying precious jewels and fertilizer? – Robert Harvey May 30 '13 at 21:57
  • @RobertHarvey Dodging the joke, many, many people running old versions of Rails got hit by a 5 month old and easy to execute remote exploit. The problem I'm describing here for Python developers exists in all software stacks (but the answers will differ). – Oli May 30 '13 at 22:43
2

Major GNU/Linux distributions have specialized security teams responsible for keeping all packages in the distribution secure. If you cannot afford it to spend enough resources to match up with these teams, then (if the highest security is the goal) the best solution is probably to rely on their work and use packages from the distributions. Distributions with staged releases (such as Debian) try to patch packages, such that their dependencies do not break.

Of course, if you use the distribution's packages, you lose the flexibility of the Python virtual environment. So, this seems to be another tradeoff triangle: high security, low costs, installation flexibility. Pick two.

Distribution packages vs. virtual environment

1

We use buildout to create deployments that are reproduceable and isolated.

A buildout acts like a virtualenv in that it lets you install eggs, keeping them separate from your Python installation, and it acts like pip in that you can control what versions are used. But buildout is also a build tool, with recipes you can make it build arbitrary parts as needed to fit your application and testing.

My virtual env does not contain anything beyond the zc.buildout egg to create new bootstraps for new projects.

A typical buildout I build has several config files, to fit development, continuous integration, and production environments, but the base.cfg invariably looks something like:

[buildout]
extends = 
     versions.cfg

newest = false

eggs = customer_project_name

[django-base]
recipe = djangorecipe
settings = settings
eggs = ${buildout:eggs}
project = customer_project_name
projectegg = ${:project}

[python]
recipe = zc.recipe.egg
interpreter = python
eggs = ${buildout:eggs}
initialization =
   import os
   os.environ['DJANGO_SETTINGS_MODULE'] = '${django:projectegg}.settings'

with a versions.cfg to pin all versions to prevent accidental upgrades.

Because we use a versions.cfg file, it is easy to parse all versions files to see what projects are affected (it's just a ConfigParser file, after all), and upgrading the package is just a question of updating those pins and re-testing the project.

Once tested, push the updated versions updates to the customer server, rerun buildout.

Note that upgrading security fixes should be a dedicated maintenance task anyway, as you need to plan, test and find a upgrade window for your customers.

  • That's not really the area of the build I'm struggling with. I use fabric. Once you've deployed something with zc.buildout and it has created your virtualenv, how do you make sure you aren't missing security updates to software you use? – Oli May 30 '13 at 16:30
  • 1
    Oli: we use buildouts for development to make sure we can test any upgrades. Security upgrades need to be tested too. I wouldn't want things to unexpectedly break because a security upgrade stepped in that closed something off. Security updates are a maintenance service that you need to plan for. – Martijn Pieters May 30 '13 at 16:33
  • @Oli: But a security upgrade can be a simple as updating the versions.cfg pin, and rerunning bin/buildout. – Martijn Pieters May 30 '13 at 16:34
  • On a deployment question this would be helpful information but I need to politely reiterate that this question is all about the maintenance and security side of things. My problem is keeping track of what versions of packages are installed in numerous places, what versions of packages are available, and what sort of upgrade any upgrades are. – Oli May 30 '13 at 22:38
  • 1
    @Oli: The problem with that is that you'd need to write some sort of periodic parser; that'd be much easier with well-maintained version pins in a parseable format. Your next problem is distinguishing between major and minor releases. We have always just relied on good contacts with the community and to be informed of security problems through those channels instead of automated processes. – Martijn Pieters May 30 '13 at 22:41
1

I recommend you use Docker, it provides a good basic Security Structure, updates automatically and is absolutely portable. Deploying a Django site is a complete pain but as I said since I use Docker my development and production environment are almost identical so only push and pull =)

Some good reads on the Topic:

http://pawamoy.github.io/2018/02/01/docker-compose-django-postgres-nginx.html

https://wsvincent.com/django-docker-postgresql/

  • Hello nyx00, can you include a summery of the articles you linked in your answer? That would improve your answer and keep its value when the linked articles disappear. – Kasper van den Berg Oct 1 '18 at 14:42
0

For getting notified when exploits exist there are RSS feeds, services and products out there that will let you know about exploits. At the very least any admin should probably be following CVE or NVD (which includes CVE).

How to manage exploits in libraries used by your application, but not in the virtualenv? For example ImageMagick, the python executable itself or a system library that python uses. I think your only option is to upgrade the platform, and then do the required vetting of the virtualenv and application.

At my work we create our own package (RPM) which includes python. We are in the same boat if there is a system library security flaw (upgrade the platform, vet the application). However application libraries or python itself are easily managed as they are in the toolchain package. We simply rev that, run it through our continuous integration build which rebuilds all the projects and generates all new packages for the upgrade toolchain.

There is no vertically integrated solution specific to python or django that I know of.

0

You should have a look at the features of requires.io. This seems what you are looking for.

It's free for any Bitbucket or GitHub-Account.

Apart from that:

I am working in a context, where I have to work with sensitive data for the public sector. And we use CentOS as our deployment platform. Mostly we rely on the security updates of the .rpmpackages which come with CentOS. And for some cases we do our own packaging of selected needed "python-packages". Our infrastructure team tracks known security issues of the self packaged stuff and does the repackaging. This is how it works for us.


P.S.: If you are on the container side of things, there is anchore which provides a similar looking service of scanning container images.

Not the answer you're looking for? Browse other questions tagged or ask your own question.