11

I found a website saying that calculus and linear algebra are necessary for System Programming.

System Programming, as far as I know, is about osdev, drivers, utilities and so on. I just can't figure out how calculus and linear algebra can be helpful on that. I know that calculus has several applications in science, but in this particular field of programming I just can't imagine how calculus can be so important.

The information was on this site: http://www.wikihow.com/Become-a-Programmer

Edit: Some answers here are explaining about algorithm complexity and optimization. When I made this question I was trying to be more specific about the area of System's Programming. Algorithm complexity and optimization can be applied to any area of programming not just System's Programming. That may be why I wasn't able to came up with such thinking at the time of the question.

Improve this question
15
  • 6
    It's not so much direct application as changing your thought process.
    – SomeKittens
    Jul 26, 2012 at 16:33
  • 3
    I don't do a lot of system programming (ok - I don't do any), but I imagine that if you are running a series of performance tests and you want to do statistical analysis on the results, calculus and linear algebra could come into play. They might also be needed when doing complexity analysis of algorithms.
    – FrustratedWithFormsDesigner
    Jul 26, 2012 at 16:46
  • 2
    @Telastyn: Really? That's interesting. How does that work?
    – FrustratedWithFormsDesigner
    Jul 26, 2012 at 17:39
  • 2
    Can you provide a link to the web site? It may add some context that'll help you get a more specific answer.
    – Caleb
    Jul 26, 2012 at 17:59
  • 2
    @FrustratedWithFormsDesigner - The analogy is from a value to a pointer is like derivation. You start with a function and derive it to get a new function that means something else (an int that refers to a memory address rather than a number). You can even get the second derivative (a pointer to a pointer) that means kinda the same thing but is different. Then to undo that, you need to integrate (dereference the pointer), which has some caveats (the lost constant when deriving a function vs object slicing). Hopefully that makes sense, it's been ages...
    – Telastyn
    Jul 26, 2012 at 18:13

8 Answers 8

Reset to default
7

I would imagine its not very important if you're writing non GUI utilities on top of a modern operating system without working on its internals. Its probably a different story if you are working on changing a modern operation system or developing a new one.

If you're working with video hardware or a bare metal windowing system, you're going to need knowledge of linear algebra to efficiently update graphics. I haven't looked myself, but I bet you could find examples in the source code for X, KDE and Gnome.

If you're working with hardware regarding digital signal processing then calculus will be very important. I imagine there are some devices which do their heavy lifting with the system's CPU instead of a local microprocessor and these often interface to analog electrical systems.

Also calculus plays an important role in performance analysis besides just linear algebra when attempting curve fitting to the data.

Improve this answer
2
  • Even just doing directx or opengl programming will require some linear algebra understanding.
    – Rig
    Jul 26, 2012 at 20:43
  • @Rig: Learn as you go, that's how I did it. Learning from math side, IMHO, is bass-ackwards.
    – Coder
    Jul 26, 2012 at 21:58
10

SomeKittens' comment is right on the money: You need calculus and linear algebra because those courses change the way that you think and the way that you understand the world. Linear algebra is all about mapping from one domain to another; calculus covers the way functions behave. They're powerful tools themselves, but the techniques you learn when studying those fields also become part of your mental picture of the world.

You also need those courses because people will expect you to be able to think in those terms. I don't often see my colleagues taking the derivative of a polynomial on their white boards, but I do often see sketches of functions with the tangent drawn at some interesting point, or the area under the curve shaded. We don't care enough about the actual values to bother calculating them, but understanding about how the values change is essential, and it's part of everyday conversations.

Any undergraduate computer science degree will require calculus, linear algebra, statistics, logic, and other math courses not because programmers need to apply the techniques directly on a regular basis (although they might, depending on what they do), but because you need that knowledge to understand the material that comes later.

Improve this answer
3
  • 3
    If I could upvote this more than once I would.
    – Mr.Mindor
    Sep 25, 2012 at 21:16
  • Though a useful reply, but doesn't answer the question. E.g. I came here because I was recently doing some hacking on r600g driver for fun and also am interested in improving math background, which is weak. Would be really nice to find an idea to implement in the driver requiring extending my math knowledge, preferably something out of abstract algebra, topology, but not necessarily limited to. A bunch of queries like "drivers development math" didn't come up with a single paper, this page is the closest I could get.
    – Hi-Angel
    May 6, 2017 at 16:06
  • @Hi-Angel I disagree. It may not answer the question in the way you expected, and it may not answer your separate "what can I work on to improve my math skills" question, but I contend that advanced math is useful to a system programmer chiefly in that it changes your perspective and provides deeper understanding. Graphics programmers actually use calculus and linear algebra on a regular basis, system programmers less so. But understanding those topics is still important.
    – Caleb
    May 6, 2017 at 16:40
7

I'll go ahead and say that I don't think calculus or linear algebra are likely to be important for systems programming.

I certainly think calculus and linear algebra are worth learning in general -- I'm a math guy! And, as other answers point out, there is some indirect relevance, as performance analysis and algorithm design can use advanced math. However, I don't think systems programming is any more dependent on those kinds of math than most other fields that aren't generally thought of as mathematical.

Improve this answer
3
  • Ever seen a GPU?:) It's a bit mathematical, esp. to write drivers for it... but it's true: if you're stuck, you can always use math.stackexchange.com :))
    – Aadaam
    Jul 26, 2012 at 20:39
  • I'm a computer graphics guy too; I resemble that remark! And you're right: anyone who wants to write a graphics card driver needs to understand computer graphics (and the linear algebra and basic calculus that entails) as well as systems programming.
    – comingstorm
    Jul 26, 2012 at 21:20
  • As far as I can see, one needs to know trigonometry and linear algebra to do something useful out of graphics API, but not to implement this API in a graphics driver. There you mostly deal with switching a bunch of registers based on application requests. The only place to apply math in graphics driver I can imagine are optimizations to its compiler. Correct me if I wrong — I want to be. I actually came here because I want to improve my math background and also am do little hacking on r600g, and it'd be nice to combine both.
    – Hi-Angel
    May 6, 2017 at 16:15
5

I suspect it's true around the edges. System programmers have to be far more concerned about performance and reliability, so algorithm analysis might be important, and calculus is sometimes needed for proofs of Big-Oh analysis. Subjects like queuing theory and discrete optimization (that's mathematical optimization not code optimization) can a play a role too. However I think those would mostly apply to folks working on the bleeding edge of operating systems and network protocols, not so much the person working on the USB 3.0 driver.

Improve this answer
1
  • ⁺¹ for discrete optimization, seems to be interesting and related.
    – Hi-Angel
    May 6, 2017 at 16:32
2

Your definition of Systems Programming pretty well aligns with the answer in Wikipedia.

If you think about what it's providing - ie. a software interface into hardware, then it starts to make sense why calculus and linear algebra are handy skills to have.

Abstracting out that low level interface requires you to understand how the device operates. Electronics devices are still bound by the laws of physics. Calculus and linear algebra provide a means to model the behavior of the device. Modeling the device lets you provide a service into its functionality.

That having been said, those two fields are not the end-all be-all for Systems Programming. I know quite a few EE's who didn't do as well with calculus and linear algebra but can still explain what the device is doing fairly succinctly.

Improve this answer
3
  • Is that definition of Systems Programming right or wrong?
    – Victor
    Jul 26, 2012 at 17:11
  • 2
    That's not very accurate. Interfacing with a CPU requires only the knowledge of the provided instructions.
    – DeadMG
    Jul 26, 2012 at 18:03
  • @DeadMG - I can't argue in the case of a CPU. I had been thinking more along the terms of raw hardware like an accelerometer or a direct IO interface. That having been said, most of those are fronted with some sort of a chip to provide the interface. I think my comments are more appropriate for the logic within the chip than perhaps the device driver layer.
    – user53019
    Jul 26, 2012 at 19:51
2

General web application and/or administrative programming doesn't involve much application of linear algebra or calculus, but many specialist fields do. If you deal with geometry you're bound to run into linear algebra. Most physics programming also deals with both algebra and calculus. As well as just about anything having to do with waveform manipulation, such as sound and radio programming. In general it is more important to understand discrete mathematics which, amongst other things, deals with set theory, graph theory and formal (boolean) logic which is useful in many applications like information management, databases and other places where data and/or logic combine. In the case of Systems programming I don't see that many applications.

Improve this answer
2
  • 2
    I think you can "get by" without ever using calculus or linear algebra in most programming domains. However, if you are comfortable with the concepts you would be surprised in how many different problems you can apply them to. Measuring and predicting clock drift, battery sustainment time come to mind of things I just used calculus for recently. Triangulation and path prediction involved linear algebra for a problem I worked on last year. None of the problems required using those topics as the previous extremely complicated and poor approximation methods that others used demonstrated...
    – Dunk
    Jul 26, 2012 at 22:22
  • 2
    (cont)...but making use of the more advanced math made for much more succinct and accurate implementations that worked how they were supposed to. IOW, calculus and linear algebra are powerful tools once you learn how to apply them and they'll help you rise to the top compared to most every other programmer, since most didn't care to learn the math sufficiently while in school. For me, it has meant getting to work on the really cool projects versus the mundane.
    – Dunk
    Jul 26, 2012 at 22:23
1

As others have mentioned, any mathematics courses in university can sharpen your problem solving and deductive reasoning skills. These are important to almost anyone.

But sometimes knowing some linear algebra can be helpful, particularly for some pretty good business ideas.

Improve this answer
1

System Programming, as far as I know, is about osdev, drivers, utilities and so on. I just can't figure out how calculus and linear algebra can be helpful on that.

With calculus it's pretty easy, as soon as one takes a closer look at the course contents. It is closely related to algorithm complexity, Big-O notation - stuff like that, pretty fundamental in programming.

Equations are what you get when estimating algorithm complexity. Three-level nested loops from 0 to N are N3, two-level nested loops are N2, one is N. Evaluation you get could look like (N3 + 2 * N2 + N) - that's an equation.

Now, if you want to better understand how quickly execution time will grow when N increases, this is closely related to derivatives / differentiation. Other parts of calculus you may find helpful are limits and asymptotic analysis - these will lead you to understanding Big-O notation, better scoring at programming interviews and possibly, better doing systems programming.

  • You are assigned to design file allocation table, what data structure are you going to use? Assuming that there are many small files which are rarely modified, what would be preferable? Assuming a relatively small amount of large files that are always appended at the end, are you going to use the same structure? How would you decide?

As for Linear Algebra, here programming applications shoot at you from the very first picture.

http://pad1.whstatic.com/images/thumb/c/c4/LaTeX-2m.jpg/251px-LaTeX-2m.jpg

If you ever will have to deal with raster graphics (eg in video drivers), pictures like above will be coming at you in your worst nightmares.

  • How come that test #12345 shows missing pixel? did I do something wrong implementing Bresenham? could it be just a mistake in test design not properly accounting for rounding errors?
Improve this answer
2
  • There is a trend where CS in every university I know is splitting away from math departments. I think it's a very smart move. Moreover, CS students are thought to figure out solutions from using existing solutions. A good CS student would read a document on container, would see a O(nsomething), open up the wikipedia, see the graph, and decide if the container is appropriate. That would take 2 minutes, just like checking reentrancy and parameter requirements for some API function call. And, IMHO, a profiler is almost always a better solution than theoretical math.
    – Coder
    Jul 26, 2012 at 22:16
  • 2
    @Coder, of course some computer science students end up being the ones who have to design the containers and algorithms in the first place. For them the math and the profiler are complementary tools: the math won't deliver a working implementation, and the profiler won't warn you that your testing data is just a fluke.
    – Charles E. Grant
    Jul 26, 2012 at 23:26

Not the answer you're looking for? Browse other questions tagged or ask your own question.