So, I've finally gotten myself to a point where I'm comfortable enough with Python (using Pyramid as my framework of choice) to undertake a rather large personal project. As it's a personal project, I have the luxury of taking my time having absolutely no deadlines other than self-imposed ones.

I love learning new frameworks, languages, etc. so if given a reason, I don't mind pushing back on development for a month or so while learning a new language and framework (takes longer when you're doing it on your own time ;)).

I recently learned about CPython's GIL (Global Interpreter Lock), which raised my eyebrow a bit. If I understand it correctly, this means that if I have a Queue in my web app and have threads that complete jobs in the queue, then the code is locked until the thread for that request is complete, meaning that any subsequent request is locked while the previous thread is executing.


Has anyone in "real world" applications found this to be a problem? Is it worthwhile to learn a language that supports real concurrency out of the box, such as Erlang? I'm most interested in any benchmarks that anyone has done in real world apps and whether or not anyone has seen any real noticeable issues with the GIL.

2 Answers 2


I have run up against the GIL in server side programming in almost every instance where I need something to scale to millions of concurrent users on multiple core machines.

Python is great for command line tools and things that don't need true concurrency to extract every last bit of performance from a given piece of hardware.

But for things that really need to squeeze everything out of something like a Sun T2000, you don't want to write anything in Python, it will be a operational maintenance nightmare running 32 separate processes and trying to management them all.

I abandoned Twisted in favor of Erlang a few years ago, Python just doesn't cut it in the large scale concurrency space. The transparent distributed nature of Erlang means it scales horizontally as well as vertically.


The lock doesn't prevent you switching between threads. It prevents multiple threads from running at the same time. This only has an effect for multiple cores. Python's threads do not take advantage of multiple cores. Really, that is the only difference.

To reiterate, the code is not locked while the request is executed. The requests will happen side-by-side as the computer switches between threads. This will function exactly the same as any other language running a single-core machine. Only if you have multiple cores will you see a difference.

This is probably not a concern for a web application. Your web application will spend most of its time waiting for the network connection. The amount of time spent doing CPU heavy work is so small you won't notice. The bottleneck will still be at the network so the fact that you couldn't take advantage of multiple cores was not really a big deal.

If you really do need to make use of multiple cores python has a multiprocessing module. This allows you to control seperate processes which will allow you to execute using multiple cores. It works somewhat differently then a threading module, but I find it easier to work with.

My own experience is that I created a little library that uses python, network communication, and multiple processes to distribute python work over a cluster of computers. It works great. I'm able to easily distribute work (as long as it can be split up easily) across a collection of computers. So basically, nothing in python prevents working with this sort of thing.

  • 2
    Actually performance actually gets significantly worse running multiple threads on a multicore machine, because of the way the GIL is implemented. Check out this video for details. It's kinda scary once you see what's really going on under the hood. Commented Sep 21, 2011 at 0:16
  • @MasonWheeler, yes you are right. I forget about that because I use eventlet which doesn't use real threads and doesn't hit that problem. (It still can't use multiple cores) Commented Sep 21, 2011 at 0:47
  • Here's an updated link to the video mentioned above. But I'm wondering if the issues mentioned here are still issues with all the new features Python has come out with since 2011. Commented Jun 1, 2023 at 20:49
  • 1
    @NathanWailes, just re-run David Beazley's Performance Experiment from slide 3. On a MacBook Air with lots of cores, I see python 3.10.11 counting to 100 M twice, sequentially, in 10.4 seconds, and the pair of threads contending for the GIL accomplish it in 10.5 seconds, for 1% overhead, with minimal sys-mode time. Looks like the "GIL battle!" that he elucidated is an issue long since fixed at this point.
    – J_H
    Commented Jun 4, 2023 at 1:34
  • @J_H So should that be the accepted answer, then? If you add that as an answer, I'll upvote it. Commented Jun 4, 2023 at 13:28

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.