I am using egrep to search our codebase, but at around 300MB of source code that starts to get a little slow. How do you search your codebase?
closed as not constructive by user40980, GlenH7♦, Jalayn, EL Yusubov, ChrisF♦ Apr 26 '13 at 22:47
As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance. If this question can be reworded to fit the rules in the help center, please edit the question.
Check out ack from http://betterthangrep.com/.
ack is a tool like grep, designed for programmers with large trees of heterogeneous source code.
ack is written purely in Perl, and takes advantage of the power of Perl's regular expressions.
From the site:
Top 10 reasons to use ack instead of grep.
- It's blazingly fast because it only searches the stuff you want searched.
- ack is pure Perl, so it runs on Windows just fine.
- The standalone version uses no non-standard modules, so you can put it in your
Searches recursively through directories by default, while ignoring .svn, CVS and other VCS directories. Which would you rather type?
$ grep pattern $(find . -type f | grep -v '\.svn') $ ack pattern
ack ignores most of the crap you don't want to search
- VCS directories
- blib, the Perl build directory
- backup files like foo~ and #foo#
- binary files, core dumps, etc
- Ignoring .svn directories means that ack is faster than grep for searching through trees.
Lets you specify file types to search, as in --perl or --nohtml. Which would you rather type?
$ grep pattern $(find . -name '*.pl' -or -name '*.pm' -or -name '*.pod' | grep -v .svn) $ ack --perl pattern
Note that ack's
--perlalso checks the shebang lines of files without suffixes, which the find command will not.
File-filtering capabilities usable without searching with
ack -f. This lets you create lists of files of a given type.
$ ack -f --perl > all-perl-files
Color highlighting of search results.
- Uses real Perl regular expressions, not a GNU subset.
Allows you to specify output using Perl's special variables. Example:
ack '(Mr|Mr?s)\. (Smith|Jones)' --output='$&'
Many command-line switches are the same as in GNU grep:
- -w does word-only searching
- -c shows counts per file of matches
- -l gives the filename instead of matching lines
- Command name is 25% fewer characters to type! Save days of free-time! Heck, it's 50% shorter compared to
Plus, there's this:
[BinaryMuse ~]: ack --thpppt _ /| \'o.O' =(___)= U ack --thpppt!
With Visual Studios "Find in Files".
If your code base gets too big it might be time to split it up in parts. Then you'll know in which part to do your search and it should go faster.
Well, the first way to search is your IDE, whether it's CodeBlocks, Eclispe, IntelliJ or Visual Studio (or whatever) it's bound to have an index of the existing code in the project, and its dependencies.
When we want to peek at someone else's code at work, we have an instance of OpenGrok. It's wickedly fast, really, usually takes less than a second to give you the results of your search. I've only used it though, not configured it, so I don't know about the admin side of things.
I've seen places that use a Google Search Appliance for searching code. That's probably overkill, though, even for 300MB of source code. You could set up something a little more cheaply using Apache Solr and a custom web interface. There might even be an off-the-shelf solution along those lines (though I'm not personally aware of any).
Other than that, I also typically use Visual Studio's built-in "Find in Files". Since it has the option to do either a regex-based or sub-string based search, and a sub-string based search is much faster, you might try something like that as well.
Cscope with vim for overall project view. It enables jumping to definition, callers, symbols search or grep for identifiers or to jump to a file. Cscope works very well with C code but fumbles with C++. Exuberant ctags with taglist plugin for vim for per file overview. Exuberant ctags works well with C++ (actually over 40 languages are supported) but does not have support for jumping to callers.
Then there is good old grep for quick and dirty searching within a source tree.
Actually in some ways grep may be your best friend! Grep will handle huge amounts of data and you can chain several greps together to get exactly what you want.
Of course being able to only grep subsets of your code base can be useful too.
My main development environment is Mac OS X and I find Spotlight very helpful when e.g. I'm looking for a snippet of code that I can re-use.
When I'm on Linux I'll typically use a combination of find + grep.
The built-in "Search in source code" on Github which is very fast
The "Find in files" function of the IDE if I'm looking for locations that were not pushed to the repository yet
You can get very specific with Mac OSX's Finder
locate and the different flavors of
fgrep if you can search without regex to get results faster.
Many environments these days will use a sort of indexer to help the user locate where various functions and variables are defined. That's great if you have it, and not so great if you don't. If you are primarily interested in where the routine or variable is defined, there are a number of techniques that can be used to facilitate that search. If however you are looking to determine where they are used and you don't have an indexer, 'grep' (or its derivatives) are probably your best bet.
Function headers. As part of the coding standards at most places I've worked, each routine requires a comment of a specified structure to precede it. Part of this comment includes the routine name followed by a space, which in turn is followed by a special character. Depending upon the company, this special character has typically been either a '-' or a ':'. Not only does this make searching for the routine easy when you know it is in the file you're in, but it also helps get clean and readable output when performing ...
grep -lrw 'routineName -' *
Routine naming conventions. We tend to name routines using the pattern below. Consequently, based solely on the module abbreviation in the name, we can usually figure out quite quickly where the routine is defined.
< module abbreviation > < noun > < verb >
Variable naming conventions. We tend to name our variables in a manner similar to the way we name our routines. Again, the module abbreviation helps us determine where the variable is defined.
< module abbreviation > < description >
Hope this helps.