3

I need a programming language for text editing and processing (replace, formatting, regular expressions, string comparison, word processing, text analysis, etc.). Which programming language is more powerful and has more functions for this purpose?

Since I work PHP for my web projects, I currently use PHP, but the fact is that PHP is a scripting language for web applications; my current project is offline.

I am curious if other programming languages such as Perl, Python, C, C++, Java, etc. have more functionality for this purpose, and worth of shifting the project?

5
  • well that depends on your os. linux/unix has a great set of tools for text editing, but you are not going to find them on a windows machine....
    – NWS
    Mar 27, 2012 at 8:02
  • @NWS, ...unless you install Cygwin. Mar 27, 2012 at 8:21
  • @PéterTörök Correct - but i think OP would be looking for a more 'native' option...
    – NWS
    Mar 27, 2012 at 8:27
  • @NWS I'm on Linux, but I am looking for a languge to write a set of scripts to do so, not looking for a text editor.
    – Googlebot
    Mar 27, 2012 at 8:30
  • 3
    Personally I'm a fan of Perl for this sort of thing, after all it was designed as a text processing language. Its also good for scripting and projects in. May 6, 2012 at 18:27

6 Answers 6

21

Every popular language has facilities for text editing and processing; it's mostly a matter of personal preference. Since you are already familiar with PHP, I'd suggest Perl, as the syntax is similar, and PHP's regular expression facilities are heavily based on Perl's. Furthermore, Perl was originally designed as a text processing language, and although it has largely outgrown its origins, it might be a little bit easier to find related resources.

I'd advice against a language with a large stack, like Java or .NET, unless of course you are interested in learning the stack as well.

The bottom line is: this is less about the language and more about you. Even PHP would make a good choice, depending on your needs. Granted, there is some tweaking necessary, but if the time it takes you to learn a new language is significantly more than learning how to use PHP in an offline manner, then PHP would possibly be the better choice.

5
  • 2
    +!: Perl is excellent for text processing. Unless you have very large files (100Mb+) that will be processed repeatedly, there is no need to look any further. Jun 4, 2012 at 4:11
  • +1 - Good answer. Really depends on how many hoops you want to jump through. I mean, even C can be used to do text processing.
    – tehnyit
    Jun 4, 2012 at 13:32
  • @tehnyit C was in my answer (check revision history), but only because I wasn't properly caffeinated at the time I was writing it (I didn't even mean C, I meant C++).
    – yannis
    Jun 4, 2012 at 13:34
  • @YannisRizos Glad you can see my sacarsm..:-)
    – tehnyit
    Jun 4, 2012 at 14:37
  • Suggesting PHP with its terribly bad unicode support? No way. No +1 from me.
    – meisterluk
    Feb 9, 2013 at 9:58
12

As you are on Linux, I would suggest looking at Tcl, Perl and Python. Depending on what you actually want to do, these three should be able to handle it. Also educate yourself about sed, AWK and grep, etc... and use Bash, sh, and or tcsh scripts to your advantage also. (There is no reason why you cannot use a sh script to split the work into three bits and use Tcl, Perl and Python for each bit according to their aptitudes!)

I'm not sure compiled languages like C/C++, Java, etc. offer much unless you need performance and/or detailed error messages.

4
  • very subtle approach to use gain the features of several languages. Shell is really a flexible tool in this direction.
    – Googlebot
    Mar 27, 2012 at 12:40
  • +1 for including sed. I'm surprised that nobody else has mentioned it. Mar 27, 2012 at 18:16
  • All C would really offer is the opportunity to work very hard to achieve what you could do in very little programmer effort in other languages. C's strength is in building the other languages. (C++ has better string handling, but then again, virtually every post-C language has better string handling.) Feb 10, 2013 at 12:54
  • I would prefer Ruby over Python here for its regex capabilities.
    – nawfal
    Jul 21, 2014 at 23:18
10

Perl and Ruby would be good choices.

However, what you really want might be AWK, which is rather old, but it will do what you want. There are a few good books on AWK, but all of them are more than 10 years old. However, don't let that phase you; it's still a powerful tool.

6
  • 1
    +1 for pointing out AWK. It was always only a name to me, but I must give it a try.
    – Googlebot
    Mar 27, 2012 at 10:52
  • I am not very good at it, but I keep meaning to learn, as soon as the right task shows up to use it
    – Zachary K
    Mar 27, 2012 at 12:17
  • 1
    @ZacharyK, the trick is not to wait for the right task, and then learn AWK, but rather to go ahead and start learning AWK, and you will find yourself stumbling over candidate tasks in short order. My first exposure to AWK came at a FORTH Interest Group meeting. About three minutes into the guy's talk, I hit him with a flying tackle and refused to let him up until he promised to get me a DOS version: I had a current task that was suitable. (AWK wound up saving me a HUGE amount of headache on that job, and spawned an article for the Austin PC User's Group newsletter.) Jun 4, 2012 at 3:22
  • The problem is that I could say that about 20 other things as well. ;) Prolog is high on my list of thins that I would love to learn and I am sure I would find some cool way to use it as soon as I did
    – Zachary K
    Jun 4, 2012 at 3:39
  • 1
    IMO it is easier to just learn Perl than to deal with the limitations of AWK. Perl is pre-installed on most if not all Linux distributions. Jun 4, 2012 at 4:13
6

Learn Regular Expressions (regex)

Regex is a small technology that you can master in a week or two (1-2 hours/day), but it's so useful that it will pay off that investment of time on the first project you use it on. Most programming languages support them including PHP. There is a wonderful book on regex, Mastering Regular Expressions by Jeff Friedl. It starts slow, but speeds up quickly and really has everything you need to know about Regex. It's one of my favorite technical books.

AWK

Once you know Regex, you can do amazing things with awk (gawk), sed, and grep, or really with any modern programming language. Regular Expressions are not a programming language, so I suppose that's technically not an answer to your question, but Awk is a complete language and it rocks for one-off text processing.

emacs

One other thing that I feel compelled to mention is emacs. For crunching data from one format to another, I write little combinations of regular expressions and elisp right in the same buffer as the data I'm transforming. Ctrl-x Ctrl-e evaluates the previous lisp code. Ctrl-x r d deletes a currently highlighted rectangle. Between these functions, you can experiment with transforming text in a buffer until you have just what you want. It's truly interactive and wonderful. Not to mention that it's written by Richard Stallman, who also created the GNU Public License (copyleft) and started the movement most people call Open Source Software.

The Dark Side of emacs

The learning curve for emacs is very steep. It took me 6 months to be able to use emacs and 6 years to not think about it any more. Common regex characters require three back-slashes to escape them properly for emacs, which can lead to what Friedl calls, "Leaning Toothpick Syndrome."

Emacs a modeless editor, so there are lengthy key sequences for everything that can really wear out your little and ring fingers on each hand from holding down Alt, Ctrl, and Shift all day long. People jokingly call it Escape-Meta-Alt-Ctrl-Shift, but it's kind of true. Evil Mode for Emacs makes it much easier on the tendons.

Since some clown decided to add "Windows Keys" between the Alt and Ctrl keys on computer keyboards, using Emacs has become more difficult. I use an old ThinkPad keyboard from before the days of Windows Keys and have an extra in the closet in case it breaks. On newer keyboards, I like to pop off the offending keys. Hmm... That's probably too much information...

Conclusion

Learn Regex - it makes every language better!

0
3

Actually you can use PHP on the command line, so the assertion that it cannot be used offline is false. If you're already familiar with PHP and its string handling functions you should look into it.

I used to work with PHP and there were some behind the scenes stuff done with PHP scripts that could be run on the command line (with cron jobs even). It was a little weird to do PHP scripts offline, because you still need the preprocessor tags (that is, <?php ... ?>), but it was doable.

Alternatively, we use Python scripts as well for behind-the-scenes stuff, but it was more a matter of personal preference than anything.

3

It REALLY depends on what you need to do. If you mainly want to recognize things, and your input is lines, each line consisting of fields delimited by spaces, AWK is easy to learn, easy to use, and quite effective for what it does.

I've never really studied Perl, so I can't really comment on it. Ditto for Ruby and Python.

I don't think there's anything that runs it today, but TECO on the PDP-10 was in fact a programming language for editing, that came with an interpreter. I wrote a complete #include file processor in TECO, back in the late 1970s. (It was all I had on that machine that was suitable.)

GNU Emacs is actually a Lisp system that had certain fundamental design decisions resolved in ways that made writing editors and editor extensions easier. It also has a lot of low-level editing stuff compiled into the "kernel".

SNOBOL, while ancient, was specifically designed for that kind of thing. I am given to understand that it is still available, even today.

1
  • -1:No useful advice here, just a list of dead languages. I've written a lot of PDP-10 TECO too and it was painful. Ditto for SNOBOL. There is a reason it is dead language. Emacs LISP is ok, I've written a lot of that too, but it's not a reasonable choice for writing stand-alone filters. Jun 4, 2012 at 4:19

Not the answer you're looking for? Browse other questions tagged or ask your own question.