How to tackle massive Linux/makefile projects effectively?

Question

I have been developing Windows applications in C++ for like 10 years now. And recently I've started digging into some Linux projects, and I can't stand how unproductive I am...

I'm a fast learner, and I've been using Linux as a primary platform for some time now. And I do feel very comfortable with shell, OS principles and GUI. But when it comes to development, it feels like I'm back to school.

As soon as I open some larger project, I'm stuck. Most of them are makefile based, so basically when I try to navigate them with QT or CodeBlocks, at best, I can use intellisense on a per-file basis. And most of the time variables leak from scope.

Then there is a go-to-definition stuff, which seems nonexistent, try to join some larger project from sourceforge, and you're stuck for days, because navigating to definitions is so hard... grep -r "this_def" . --include "*.cpp" --include "*.h" seems so slow and clumsy.

And then, the debugging, gdb does work, but no matter what I do, it seems like it's light years behind WinDbg or VisualStudio debugger.

And these things are making me desperate, I want to write code, but it just goes so slow... I'm starting to think that Linux developers learn function definitions by heart and analyze code by eyes, but I can't believe it's so.

Has anyone gone through this? Is there something that I'm missing that could make me more productive?

Answer 1

Interestingly enough I periodically have the same problem in the opposite direction. I'm primarily a UNIX coder, but I periodically have to port stuff to Windows. I can't tell you the number of times I've wanted to pull my hair out because I can't find the appropriate check box for a compiler option buried in one of 35 preference setting pages for a project. I'd rather just open up the proj file and add the XML myself.

Moving in either direction, the secret is to have patience, and learn the tool set for the platform you are trying to work in. Of course you are going to be frustrated, it's new, and it's unfamiliar, and you are reduced to newbie status all over again. There is no way to avoid this.

In your particular case there are some additional tools you should be aware of. The first is DDD, a GUI front end for gdb. It's not as slick as Visual Studio, but it will hold your hand. However, I'd really recommend biting the bullet, and set about learning the ins and outs of gdb. In truth, if you are a regular user, there isn't a lot of difference between memorizing which commands to type vs memorizing which dialog box you have to bring up to change a setting.

You also need to know about tools like CScope and CTags. As much you may resist, I would suggest learning VIM or EMACS. They integrate well with tag tools I just mentioned. When in Rome, do as the Romans do. You can find extensions for VIM and EMACS that will do code completion for you. My own experience with tools that offer code completion is is that yes, it does saving some typing, but in general typing is easy. Thinking is what's hard. Your opinion may differ, particularly if you have carpal tunnel syndrome.

As for make. Make is admittedly horrible, but you probably just going to have to suck it up and learn it.

Answer 2

I have been developing Windows applications in C++ for like 10 years now. And recently I've started digging into some Linux projects, and I can't stand how unproductive I am...

Is there something that I'm missing that could make me more productive?

Develop on Windows, deploy on Linux.

This includes running unit tests both on your own (Windows) machine, and on the build server (Linux).

As a side effect, you'll learn how to write portable code.

Another positive effect is that using different compilers will generate more warnings and thus catch more bugs.

UPDATE: To all the Linux fanboys downvoting this answer: I don't say that everyone should develop on Windows! But using the platform you know very well is more productive than spending a lot of time on learning a new platform.

Answer 3

Your problem has been solved many times in the Linux world, however, unlike the Windows/Microsoft tools, it will not be handed to on a silver plate with a side dish of extras. You might need to do some work to get it.

I use a commercial editor (Visual Slick Edit, which is considered expensive by those that do not value their time as much as I do) for this exact problem. Eclipse with the CDT plugin are an open source way to go that has justifiable vast following. (No good for me as I often need ADA support)

What I do not do it try to reverse engineer the makefiles into some kind of project. I use the IDE's build in systems and manually add/remove files as needed. I am sure I could script it up, but the time is probably not worth it. For this I found eclipse a bit less usable than Slickedit (That could easily (and probably has) have changed since i last looked)

Linux has a vast array of tools, guys that know vi well out-perform me in all aspects of editing, it has references lookups etc, just a steep learning curve. I am certain Emacs can do it all as well, although have never used it.

Answer 4

For what it is worth, on Linux you have better building systems than plain old GNU make (which often goes with the horrible autoconf), for example omake and many others (cmake, scons ...).

Answer 5

one suggestion regarding how tedious it is using grep to search for code: set up bash aliases in your .bashrc file. So then its just a single command:

alias searchCode='find -iname \*.cpp | xargs grep $1'
alias searchCodeHeaders='find -iname \*.h | xargs grep $1'

there's probably better ways to write the command, but the idea is the same. Wanna search code ? write an alias called searchCode. Remember that although they're tedious and complicated, unix tools can also be used to make your life easier.

Answer 6

My 2c as someone who has developed C++ on both platforms and likes them both.

1) Makefiles are painful - the best advice I can give you is to try switching to another build system, if possible.

2) For code editing and browsing, there are some quite useful tools. Sure, they are not integrated, but it doesn't really matter when it comes to getting things done. vim+ctags+grep will just get you there. Of course, there are IDEs as well, but frankly I didn't like anything I tried: Eclipse+CDT, KDevelop, Code::Block. You may come to different conclusion, though.

3) For debugging, just stick to command-line gdb. Sure, it is quite behind Windbg when it comes to features, but for most purposes it is just fine. Graphical front-ends (ddd, KDbg) were pretty buggy the last time I tried them, but again things may have changed :)

The bottom line is - yes you need to put some learning effort, but after that you'll be just as productive as you are on Windows.

Answer 7

To all the rest of the good advices that you already received, I would like to add a couple of links, respectively to ack and pss.

They are aimed at programmers who have to specifically care about source code, trying to improve over grep.

Answer 8

Great answers. Adding to them,

When I made this move, the mistake I did was trying to jump into the code without giving the due diligence to the GNU build system, which came back to bite me when I wanted to make code changes. Spend the couple of days it takes to understand how AutoMake/AutoConf/Make suite of tools work, you will be very quick after that.

Regarding tools - Eclipse + CDT / GDB + DDD goes indeed a long way.

Answer 9

Here's some advice to get it working easier:

1. Start from the beginning. This means you'll use bash script as a makefile replacement first, and then simple makefiles once you need dependencies. Keep it simple at the beginning. Your makefile does not need to handle 15 different unix platforms -- just make it compile with dependencies. (your initial makefile is going to look like: all: g++ -o main main.cpp) (more complex example can be found from http://sivut.koti.soon.fi/~terop/Makefile -- when starting from scratch, just copy the file to the project, change filenames, mkdir objs directory, change libs you want)
2. Forget the IDE. It's only going to make you slower. Learn to read the code and remember the file hierarchy of your project. Normal commandline tools like 'cd dir' and 'emacs foo.cpp' needs to be so automatic that you don't think twice typing it.
3. Forget syntax highlighting and intellisense. It's never going to work the same way it works on visual studio, and the hints its giving is only going to confuse you since it's different from what you've used to. Learn to not rely on them.
4. Gdb is a good debugger. You'll get a call stack. Do not even try to step around the code, it's not going to work very well. Use valgrind too. Breakpoints are good too, but don't rely on them too much; only rarely used with gdb.
5. If your compilation is anything else than simple 'make' in the shell, you need to rethink your edit-compile-debug-edit -cycle.
6. Here's a trick that visual studio cannot do: Show both header file and .cpp file on the screen at the same time -- so that both are visible without flipping between them with tabs. Emacs can do it. So can notepad. Visual studio or IDE cannot.
7. Learn emacs keybindings. It's going to make you faster. A nice hint is that all the keys are located very near in the keyboard. The command sequences are longer, but it's still faster to type them.
8. There is easy trick to replace go-to-definition: write the code to same file, and use esc-< and C-s ::myfunction in emacs to find the function you want. The process is different if you've used to many short files - your code is going to look different. (learn ctrl-f in notepad, and you'll get the idea). But you definitely do not need to do it in shell.
9. Text editor startup time is very important. If it takes longer than 1 second to open it, it's no good. Every IDE is broken because of this requirement. Notepad and emacs are ok.
10. goto-line in emacs is important feature for programming. Bind it to some key. I use C-x g