Developing Python on Windows and deploying to Linux

Question

I have a client who would prefer to host their application on Linux. However, my coworkers and I have very little experience with Linux. This is a short project with a low budget, so making choices that save time and money are not just desired, they're a must. We are also heavy on continuous integration and automation, much of which we already have figured out in Windows and can reuse from previous projects. That said, having the development team learn Linux and rebuild our automation so we can develop on the same environment to which we intend to deploy is most likely not a viable option. (For a larger project, perhaps, but not this one.) The team is familiar enough with Python that writing the application in Python is a viable option (even though most of our development is done in .NET), although we would need to figure out a good packaging mechanism that can run on Windows and be pushed to a Linux box. I don't anticipate needing any libraries unavailable on Windows. Most likely, we will only need psycopg and sqlalchemy in terms of libraries with native components.

All this makes the notion of having developers create the application on Windows, deploying to a Linux testing environment, and then pushing to production after thorough testing seem like a fairly attractive option, but I'm skeptical about it. Linux and Windows are very different operating systems, and I'm concerned about gotchas that could creep up and make life very difficult. Are there any real concerns with doing this (beyond the typical file path differences and other common things easily solved by good coding practices)? I think that a lot of shared hosting providers host on Linux, and I can't imagine everyone using them has developed on Linux for the past umpteen years. Is this done more commonly than I'm aware?

Answer 1

Don't know about python, but I've moved Java applications from Windows to Linux and vice-versa. Java makes the "write once, run anywhere" claim which may not be 100% true, but with very little work I was able to make it true enough (basically everything works great on Linux, a few issues on Windows).

I'll use W and L for Windows and Linux:

W: files and folders are case insensitive. L: case sensitive. Test file name capitalization carefully on Linux because Windows hides these issues.

Windows has a more granular file permissions system, that lets you use intersections of various groups and permissions. Linux has a simpler system of one group and one user per file or folder. Plus an execute-bit. Well, there are some other little things like setting the execute bit on a folder makes the permissions cascade the way they do in Windows vs. being set to the user and group that created each file the way they do by default in Linux. These issues mostly come into play when zipping and unzipping files, for instance during an install.

W: drives are mounted in the root folder as letters. L: Drives are mounted anywhere as anything. A single file can appear multiple places in your file system (symlinks).

Folder separator: W: \ L: /

Path separator: W: ; L: :

End of line in a text file: W: \r\n L: \n

Default character set: W: ISO-8859-1 L: UTF-8

You need to know which Linux distribution you are targeting. Two areas of difference are how System V init scripts are handled and how super-user tasks are performed (sudo vs su). Also you mentioned an install script. Apt and Yum are popular, but you need to work with the tool that your distribution uses. Use yum on RedHat, apt on Debian, etc.

This is why you need a Linux machine for testing, whether virtual or physical. It must use the same exact distribution you are targeting. Have someone set up a dual-boot on an old server or something. I also strongly recommend cygwin for every developer. The file permissions aren't quite the same as Linux, and you can set it to case sensitive (though it's more useful case insensitive on Windows) but it makes a pretty reasonable test bed.

It doesn't hurt you to know both (Windows and Linux) and once you do, you can make an informed choice about what works best for you. I was a Windows-only developer for the first 10 years of my career. I've been almost purely Linux for the last 4-6 years, so some of my Windows information might be old. I still run Windows in a virtual machine to do testing on Internet Explorer.

One thing you will get used to quickly on Linux is that you can solve most problems by Googling the error message. 90% of command line tools tell you how they work if you type "man ." If you really need it, most source code is easily available, depending on the distribution. When I solve a problem in Linux, I feel like I learned something about how computers really work. In Windows, I feel like I just keep blindly trying stuff until something works. When I find the solution, I'm lucky to remember everything I tried, no less know what it all means.

So I'd encourage you to spend some of your own time learning Linux and this job might be a way to get paid for some portion of that learning. But don't mistake temptation for opportunity. If the time frame is truly short, or money is tight, you may have to say that you have to deploy to what you know (Windows) or not take the job.

Answer 2

Python code

As soon as your application is a well-behaved egg, or a set of eggs, and you can easily deploy it into a virtualenv using pip or easy_install, you should be fine on any platform.

Things to pay attention to:

• Native-code dependencies. You might depend on some eggs that include these (like psycopg); make sure that such an egg is equally supported on Windows and Linux.
• File paths. Make sure you don't use a hard-coded "\" anywhere; always use path.sep, path.join, etc.
• File names. Unix filesystems are case-sensitive, so 'Foo' and 'foo' are different files. Make sure you preserve the case of file names.

App servers

Most probably you're writing a web app. Part of it is usually non-Python: the web server, DB server(s), etc. Things may be not as cross-platform: for instance, there's a Windows version of nginx, but not of uwsgi.

Bite the bullet

While you may do all of your development, unit testing, and part of integration testing, on Windows, there's no way around deploying your app on Linux.

Install Linux in a VirtualBox, or use a VPS (Amazon AWS, DigitalOcean, etc). Try to mimic the target deployment environment of your customer as closely as possible.

Install non-Python components, like the web server, memcached, or anything else platform-specific, on that machine. Ask your customer to configure these things for you if they have strong opinions about that. You can keep Postgres and other platform-independent things where you like.

Deploy regularly on the Linux machine. Usually it's just ssh linux-machine, pip -U install <your-eggs>, and, for example, sudo service uwsgi restart. You don't have to be expert sysadmins to cope with that. But if you learn some Linux lore along the way, be sure that it will be useful in the future.