How representative is Ohloh? [closed]

Question

My colleague recently pointed me to Ohloh, a website providing statistics on FOSS based on versioning repositories. It's quite a fun procrastination tool, e.g. to compare programming languages by active projects:

Which makes me wonder: how representative is such a comparison? Can we draw conclusions from this such as "Javascript is the most used programming language in FOSS, followed closely by Python, Java and C++"? Or are there some big caveats to take into account?

Answer 1

This is the problem with pretty graphs without context: you have no real way of telling of the correlations implied have any grounding in reality. JavaScript is an oddball because it used by projects whose primary language is any of the other options (e.g. a web application written in Python is more than likely to have some JavaScript with it). The only way you can compare apples to apples with JavaScript in that list is if you restrict it to server-side JavaScript.

The other thing to keep in mind is that programming language popularity does not tend to be an absolute the way Oloh makes it sound. For example, C++ remains heavily used in games, both commercial and open source, but far more web applications are written in Ruby or Python. Languages do not exist in a vacuum and tend to have greater or lesser popularity and applicability by domain.

Many of the broader takeaways are obviously true. Not many people are using FORTRAN anymore and virtually none without a paycheck. C++ is probably trending down on the whole with more and more applications development going to languages with managed memory. However, the graph itself is of very suspect usefulness.

Answer 2

While JavaScript is probably the most used, it crawled one of my MVC apps and said that it was a JavaScript project simply because of all the default JavaScript files that are in there by default (JQuery, Bootstrap, etc). So if this is counting the projects as a single language (which it probably is), then it's probably accurate assuming all web projects are JavaScript projects and C#/Java/Python/Ruby projects will mostly only get categorized that way if they're console/desktop apps.

Answer 3

You've picked the graph with:

The lines show the count of projects with at least one line of code changed in a month

Of the four choices, this is the one saying the least about how much any language is used. The one showing the number of lines of code change and taking C into into account is a better indicator of actual popularity. It indicates that most coding in OSS is done in C, which doesn't seem surprising considering projects as the 20 MLOC sized Linux kernel.

Answer 4

It depends to which kind of representativity you are looking for. There is something like 600.000 projects taken into account by Ohioh, so you can consider it as a big enough data set. However, some of these projects are not relevant, and they are often more numberous than believed. For instance I know in Gnome (which seems to be indexed), about 60% of all the projects didn't receive any commits during the last 2-3 years. You have to throw them away if you are interested in "modern" trends.

An other point is that the number of projects is maybe a too coarse metric of the language popularity: Most of the projects are very small, and a few are very big (their sizes typically follow a Pareto distribution). So in fact you will probably be more interested in the cumulative size of the relevant projects or the projects activity. You have then to define what's a project size (number of lines of code? number of files?). If you look at the monthly evolution of the number of commits per language, you can see that C, HTML and Java are quiete close. And the C activity is slowing down since 2006. If you look the number of projects that received at least one commit per month, the C language is even in the second place, after the HTML language! The monthly number of lines of code changed is probably affected by important refactoring/restructuration in some big projects.

I fear the data are also very heterogeneous. Eclipse and Apache are probably pro-Java, since most of KDE projects will probably written in C/C++. The Linux kernel is mainly written in C. So if you focus on a particular subcommunity, the results may sharply change.

At the end of the day, the presented data can be relevant for a trend analysis, but you have to clearly define what you are looking for and adapt your observations consequently.

Answer 5

It would be a good idea to compare with Github and Google Code and anywhere else that there is a lot of FOSS development. I don't know how to do a language frequency chart for Google Code. However, Github does offer that, and they even did a blog post about it in December 2010 or early 2011.

Regardless of the results of a frequency comparison chart by language, you still need to take into account all the caveats mentioned already in the prior answers. It isn't meaningful to infer anything about popularity when aggregating without first thinking about your data. As others have mentioned, you probably want to do some or all of the following before doing a frequency comparison:

• Remove data from any projects that have been inactive for one year or longer
• Account for the fact that there is so much C because of Linux usage in FOSS, and not overweight that
• Account for anything else that gets included but isn't actually part of the code being developed (very vague sounding, I realize, but this might be different depending on the language e.g. like function libraries)