Me and my friend are developing a web-app in Python + Flask + PostgreSQL. We have been working on it for the past few months and have developed a lot of schema/use-cases specific to Python + Flask + PostgreSQL. Now, all of a sudden, we plan to move to another NoSQL database (Neo4j) because it somehow fits better to what the core of our web-app is going to be. Python supports Neo4j through embedded/rest api bindings, but uses a technology JPype, which is rather unmaintained, to say the least.

So, this question of using Scala arose today. Ours is going to be next-to-realtime application, so we can't afford to have the lag/overhead of having intermediate steps of Python->Java requests (Neo4j standalone server is based on embedded Neo4j). So, whereas I am in favour of spending a month or two and learning Scala/Lift, he is in favour of carrying on with Python and porting to Scala whenever need arises, even though when we know that Python+bindings will be a bit slower as compared to the native Neo4j support for Java.

In the past few months, we had done a lot of work in Python + Flask + PostgreSQL already. If we port to Scala, we will need to port all of it to Scala.

Would it be wise to port now? Are there any personal experiences or advices from you all? Or is this just premature optimization?

P.S.: I am aware of the learning curve of Scala/Lift.

  • 5
    " Or is this just premature optimization?" It's always premature optimization until you've measured where your bottlenecks are. Are you at a point where you can measure if issues related to python are having a great enough to justify learning scala?
    – Wilduck
    Mar 15 '12 at 20:17
  • As I said, we are working on a next-to-realtime app. Obviously, doing the steps A->B will definitely take less time than A->C->B. That's my point. We are aware of the bottlenecks, and that is why this issue of using Scala arose. We are aware of it all, its just that I want to port now to save hassle later on, and my maybe my partner wants to avoid the hassle now and port when need arises.
    – c0da
    Mar 15 '12 at 20:21
  • 3
    That's not what I meant, obviously it will be faster, but can you measure specifically that you need speed in that part of your application more than anywhere else? If it is not 100% abundantly clear that you need to improve speed in a specific area more than any other, you're almost certainly optimizing prematurely.
    – Wilduck
    Mar 15 '12 at 20:25
  • The key here is to have an objective measure of how much time you could save by switching to scala. Exactly how much more time does A->C->B take? Is it worth switching based on that objective number that you determined. Until you have that number, you cannot reasonably make a decision.
    – Wilduck
    Mar 15 '12 at 20:27
  • 10
    My feeling is that you should carry on in Python, get a prototype up and running, put it live ASAP and see if it takes off. In all probability it won't (along with 99% of other web-apps), so you'll have saved yourself a lot of effort. Learn Scala in the meantime and use it for your next project!
    – Luigi Plinge
    Mar 15 '12 at 21:03

Obviously you're not going to get a definite answer here, because it really depends on your individual case and, in particular, who's working on the project. I'm not sure what you intend to gain though, but it sounds like you would have to throw away a lot of work.

Scala is slightly more buzzword-compliant these days, but Python is no slouch when it comes to functional programming either. Still, it all depends on the specifics of your workload.

Just remember Knuth's advice about premature optimization; he's a pretty bright guy. I'd say if you're even remotely close to finishing, then don't change. It's far more important to get some working code as quickly as possible than to "do it right" the first time. Because you simply don't know what you will have done wrong until you can test it.

Then, once you know what your choke points are, design something that addresses your real concerns.


Another option, if you don't use many Python 2.6+ features might be to use Jython. Since this targets the same JVM as Scala, it should allow tighter, more efficient integration.

We provide a Jython console inside our Eclipse application so users can write their own scripts which have full access to the power of our Java back end. We can use Jython and Java classes interchangeably and seamlessly move between them. I find it a really flexible environment to work in.

The only real downside is that Jython is stuck at Python 2.5, so subsequent innovations in the language are not yet available.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.