Surviving MATLAB and R as a Hardcore Programmer [closed]

Question

I love programming in languages that seem geared towards hardcore programmers. (My favorites are Python and D.) MATLAB is geared towards engineers and R is geared towards statisticians, and it seems like these languages were designed for people who aren't hardcore programmers and don't think like hardcore programmers. I always find them somewhat awkward to use, and to some extent I can't put my finger on why. Here are some issues I have managed to identify:

• (Both): The extreme emphasis on vectors and matrices to the extent that there are no true primitives.
• (Both): The difficulty of basic string manipulation.
• (Both): Lack of or awkwardness in support for basic data structures like hash tables and "real", i.e. type-parametric and nestable, arrays.
• (Both): They're really, really slow even by interpreted language standards, unless you bend over backwards to vectorize your code.
• (Both): They seem to not be designed to interact with the outside world. For example, both are fairly bulky programs that take a while to launch and seem to not be designed to make simple text filter programs easy to write. Furthermore, the lack of good string processing makes file I/O in anything but very standard forms near impossible.
• (Both): Object orientation seems to have a very bolted-on feel. Yes, you can do it, but it doesn't feel much more idiomatic than OO in C.
• (Both): No obvious, simple way to get a reference type. No pointers or class references. For example, I have no idea how you roll your own linked list in either of these languages.
• (MATLAB): You can't put multiple top level functions in a single file, encouraging very long functions and cut-and-paste coding.
• (MATLAB): Integers apparently don't exist as a first class type.
• (R): The basic builtin data structures seem way too high level and poorly documented, and never seem to do quite what I expect given my experience with similar but lower level data structures.
• (R): The documentation is spread all over the place and virtually impossible to browse or search. Even D, which is often knocked for bad documentation and is still fairly alpha-ish, is substantially better as far as I can tell.
• (R): At least as far as I'm aware, there's no good IDE for it. Again, even D, a fairly alpha-ish language with a small community, does better.

In general, I also feel like MATLAB and R could be easily replaced by plain old libraries in more general-purpose languages, if sufficiently comprehensive libraries existed. This is especially true in newer general purpose languages that include lots of features for library writers.

Why do R and MATLAB seem so weird to me? Are there any other major issues that you've noticed that may make these languages come off as strange to hardcore programmers? When their use is necessary, what are some good survival tips?

Edit: I'm seeing one issue from some of the answers I've gotten. I have a strong personal preference, when I analyze data, to have one script that incorporates the whole pipeline. This implies that a general purpose language needs to be used. I hate having to write a script to "clean up" the data and spit it out, then another to read it back in a completely different environment, etc. I find the friction of using MATLAB/R for some of my work and a completely different language with a completely different address space and way of thinking for the rest to be a huge source of friction. Furthermore, I know there are glue layers that exist, but they always seem to be horribly complicated and a source of friction.

Answer 1

It's probably a bad idea to approach domain specific languages with the mindset required for programming at large is, or for programming general programs with general purpose languages. Being domain specific, they will likely require a steeper learning curve and an uncomfortable mind set in order to be most efficiently used. I consider writing code in Matlab equivalent to writing highly optimized, domain specific code (on par with, for example, writing efficient and clean OpenGL code). I've also seen them move more and more towards becoming useful as libraries to be used in other languages - see, for example, http://www.mathworks.com/matlabcentral/fileexchange/12987-integrating-matlab-with-c

I would say, use the same process for these DSL as you would for any others:

• Carefully select the problems which you are solving using Matlab or R, to make sure that they are exactly the kinds of problems which they are best at solving. For example, use Matlab to manipulate your vectors, and not for the rest of your work, if you can avoid it
• Generally, mix/match the solution to restrict the portions that you program in Matlab or R to the exact subset of the problem which they are built to handle.
• Follow the mindset of a typical user in the domain that the languages are built for, when designing and building your solution - adapt a vector-mathematical attitude towards the world before starting to work on a Matlab program, for example; possibly write up your work on paper, using standard math notation, first
• Do the extra work required to build yourself a comfortable work environment, and obtain the tools required for doing the job, even if different from the standard for the DSL. If you're an emacs user, for example, consider using the matlab mode for emacs to do your work; make sure it works as well as the modes you've set up for other languages
• Be ready to switch out. Especially if you have to come back to the language often, make sure to build yourself a reliable ecosystem where the work you do in the DSL is contained to the domain specific work, only, and it's as easy as possible to switch to another language for the rest of your work. Remind yourself, more often than usual, to look for ways to do the non-DSL specific work in other systems

Answer 2

I'll preface this by noting that I'm familiar with MATLAB, but not R.

The reason that MATLAB doesn't do well with OO, string processing, or custom data structures is that it's not meant to do those things. There are lots of languages for OO, lots that do a good job with string processing, and lots more that support crazy custom data types. None of them are any good at matrix multiplication, because they weren't designed for it.

Just optimizing the vector and matrix operations that MATLAB does is hard enough without dealing with user-defined types or pointers or what not (if it wasn't hard, they wouldn't be able to charge so much for it). Adding fast vector support to existing general-purpose languages is hard to do, too---it adds a large overhead for a feature that few programmers will ever use (too few programmers understand linked lists, how are they expected to use eigenvalue decomposition?).

MATLAB is so alien to you because it was designed to let scientists and engineers to do matrix multiplication and ODE calculations very fast. MATLAB doesn't measure up to your definition of a "hardcore" language because it was never supposed to. Trying to think about MATLAB in terms of Python or D is like trying to think about LISP or Haskell in terms of C or about Verilog and VHDL in terms of JavaScript---they solve different problems and approach problem solving in radically different ways. To be fair, MATLAB made some (okay, a lot of) bizarre language-design choices that I just can't wrap my head around, even from the perspective of a domain-specific language. But there's no particular reason why an astronomer should care that celestial body X is exactly 48 AU away from celestial body Y as opposed to 48.0 AU. All scientific measurements contain some error and it's absurd to say (from the perspective of the average MATLAB customer) that a given quantity is an integer and not a real number with a tiny fractional component.

Now, thankfully, some libraries are coming onto the scene which do exactly as you suggest: good support for scientific computation in a general-purpose language. For Python, there's NumPy/Matplotlib which has some rough edges but otherwise provides reasonable MATLAB functionality inside Python. The reason there haven't been other project like this is that the libraries are incredibly difficult to write and serve a market already covered by MATLAB and FORTRAN.

If you absolutely have to use MATLAB or R, you can't approach programming in them like a "hardcore" programmer, you have to approach it like a "hardcore" scientist or engineer. For LISP, you have think in recursion. In MATLAB, you just have to think in matrices. Brush up on linear algebra (MIT's lectures on the topic are a great review). Otherwise, the only way to "survive" MATLAB is with practice to recognize when a loop can be replaced with a vector operation or when your problem reduces to finding the eigenvalues of an outer product.

Answer 3

Your repeated use of the term "hardcore programmer" in reference to yourself. along with your insinuation that the designers of R and MATLAB are not, strikes me as very silly and encourages people not to take your criticisms seriously.

If you would like to read some serious criticism of R, you would do well to read this piece by Ross Ihaka, one of R's designers. It seems significantly harder core to me to design R than to use D or Python.

Answer 4

The extreme emphasis on vectors and matrices to the extent that there are no true primitives.

It depends what you call a true primitive. In R, a vector is a true primitive; that is, all variables are vectors. Likewise, in MATLAB all variables are matrices.

The difficulty of basic string manipulation.

In MATLAB, string manipulation is powerful but I agree that the code is often ugly and unintuitive (at least for now). For R, there is the stringr package, which is as nice to use as tools in any other language.

Lack of or awkwardness in support for basic data structures like hash tables and "real", i.e. type-parametric and nestable, arrays.

In R, vectors have names which works like a hash. There are also the hash and filehash packages. Not sure about MATLAB implementations, but you can call JAVA or .NET versions easily if you want.

They're really, really slow even by interpreted language standards, unless you bend over backwards to vectorize your code.

Once you get the hang of vectorisation (I'm sure you will, if you're really hardcore) you'll curse having to use loops when you return to other languages. Speed of execution is a tradeoff for speed of programming.

They seem to not be designed to interact with the outside world. For example, both are fairly bulky programs that take a while to launch and seem to not be designed to make simple text filter programs easy to write. Furthermore, the lack of good string processing makes file I/O in anything but very standard forms near impossible.

They both can read and write data in pretty much any format. They can both be called from most other programming languages. Or from a command prompt. You can create GUIs with them. How is that not interacting with the outside world? If you're struggling with your text filter program, ask on stackoverflow.

Object orientation seems to have a very bolted-on feel. Yes, you can do it, but it doesn't feel much more idiomatic than OO in C.

Agreed; they're primarily procedural languages.

No obvious, simple way to get a reference type. No pointers or class references. For example, I have no idea how you roll your own linked list in either of these languages.

Agreed in R. In MATLAB, references are called handles.

You can't put multiple top level functions in a single file, encouraging very long functions and cut-and-paste coding.

Nonsense. Just create multiple files.

Integers apparently don't exist as a first class type.

They do. See int8, int16, int32 and int64.

The basic builtin data structures seem way too high level and poorly documented, and never seem to do quite what I expect given my experience with similar but lower level data structures.

They are suited to doing data analysis. Please give specific examples of unexpected behaviour.

The documentation is spread all over the place and virtually impossible to browse or search. Even D, which is often knocked for bad documentation and is still fairly alpha-ish, is substantially better as far as I can tell.

There are many kinds of documentation. Start with ?some_function, RSiteSearch('some concept'), rseek.org, and the sos package. Not to mention the manuals that come with the install. Or a good book.

At least as far as I'm aware, there's no good IDE for it. Again, even D, a fairly alpha-ish language with a small community, does better.

Try Architect or RStudio or the Revolution Analytics IDE. See "IDEs and editors for R" section of the Stack Overflow info page for links and more options.

Answer 5

MATLAB can integrate with Java and C/C++. You can implement all of your non-numerical workload in these languages, and invoke them from MATLAB.

their use is necessary

Is there a reason why it is necessary? Are you working on an existing MATLAB code base written by other people? Is it a work requirement? (or class requirement if you're in school) If not, you may consider using SciPy or NumPy instead.

Unfortunately, in my personal opinion, if this situation is being forced onto someone, it is not always survivable. Even in college, not every engineering student can get used to the MATLAB way of computational thinking.

Answer 6

I work with MATLAB, Python and C (and sometimes C++), and I consider myself to be (primarily) a software developer, sitting opposite colleagues who tend to be data scientists, mathematicians, or other domain specialists.

Although I would be the first to admit that it is not a general purpose programming language in the sense that C or Python is, I actually enjoy writing scripts in MATLAB rather a lot, particularly for things like time-series analysis or image processing.

There are a couple of features of the language which, although generally implemented quite inefficiently, are a joy to use. For example, take logical indexing: I can create a logical vector or matrix that selects a region-of-interest, and name it "isInROI", performing a filter operation to select elements from the vector or matrix "data" in that region is then simply a matter of writing: "roiData = data(isInROI)".

It is moments like this that really make me appreciate MATLAB, and allow me to overlook its other, numerous and much-discussed sins.