14

Are there any resources out there on how to architect large, industrial strength Python projects? I'd like to start on an ambitious personal project, but I'm not really sure how to architect it and what practices to put into place to make the project successful.

Are there any resources out there discussing how to make large Python projects work?

4
  • 2
    I think this is a valid question, but as it stands it's a bit too general. Can you be a bit more specific about the nature of the application? (ex, there isn't much Python-specific advice I could think of that would apply to both large web applications and large GUI applications). Nov 24, 2011 at 7:00
  • 1
    @DavidWolever: seems to me it's more appropriate to Programmers than to Stack Overflow (which is why I voted to migrate it). It's not a specific programming question but a question covering programming process. Sure, it's made more appropriate to SO by specifying Python, but I believe it's still more appropriate on Programmers. Nov 24, 2011 at 7:03
  • Ah, a fair point. I understood it to mean “what Python-specific things should I know”, which I believe would be appropriate on StackOverflow, but you're right — a general discussion about “ architecting large stuff” would be better on Programmers. Nov 24, 2011 at 7:06
  • To anyone wondering where to find programmers.stackexchange.com, it is now called softwareengineering.stackexchange.com. So the comments are recursively pointing back here... so maybe the comments will lead you to stackoverflow :D. See stackoverflow.blog/2010/12/17/…
    – zardosht
    Oct 20, 2020 at 19:02

3 Answers 3

11

I don't think there's anything specific to large Python projects that wouldn't apply to large projects in other programming languages as well.

In any large project, you ought to aim for loose coupling and high cohesion. A large project is manageable if it is made up of orthogonal, as-self-contained-as-possible subprojects. Tests are useful for the process of creating the subcomponents one by one, as well as keeping them working while modifying them later on as you better find out what they should be, as the whole starts to take shape.

0

Python is definitely good for large projects. This, IMO, is one of the reasons Python is so popular. Actually, there are several large projects that use Python such as Chandler - A note to self system. There is also this link I found on SO.

As far as how YOU can do it, get somebody'(s) help, and start off with a small project on your own computer. Migrate that to SourceForge and get more developers. If you stick with this project you can make a big Python project. You really don't need a resource for this. Just start coding. And if you need any help, feel free to email me:-).

Hope this helps!

2
  • I'm not so sure Chandler is a good example. It's a dead project, isn't it? It looks like the project hasn't been active since 2009. Nov 24, 2011 at 13:50
  • @BryanOakley : Yes. But when it was active, it was a good example. I used it show that you CAN use Python for a big project.
    – Dynamic
    Nov 24, 2011 at 15:22
-1

I think Python is a bad choice for more serious projects. I have seen multiple projects that used some moderately large python code that completely failed to build/configure/run properly.

One project used Rasa. I was supposed to create a few Rasa dialog files and run them on Rasa. A few experts in the team tried for 2 weeks but (1) never got Rasa installed on my completely fresh installed ubuntu machine (2) the version running in docker also never worked as it should

Several other projects used machine learning and vision. These are not "big" projects per se but have tons of dependencies. Usually based on some code from github. I'm not talking 10-years old projects (but even then?), but maybe 1 or 2 years old. Invariably they break somewhere during the build. Mostly saying that some package could not be found/build/missing directories/missing dependencies/something not installed/"exit code 1". If you are lucky, and dig really deep really long, you may be able to fix the program by doing additional installs. But often it's totally unclear how to resolve the problem. Maybe older version of python, previous versions of libraries, etc.

2
  • "bad choice for more serious projects" , then check my project github.com/iperov/DeepFaceLive
    – iperov
    Dec 20, 2022 at 16:09
  • Your answer could be improved with additional supporting information. Please edit to add further details, such as citations or documentation, so that others can confirm that your answer is correct. You can find more information on how to write good answers in the help center.
    – Community Bot
    Dec 28, 2022 at 0:26

Not the answer you're looking for? Browse other questions tagged or ask your own question.