(Almost-)POSIX-compliant operating systems and Windows are known to distinguish between 'binary mode' and 'text mode' file I/O. While the former mode doesn't transform any data between the actual file or stream and the application, the latter 'translates' the contents to some standard format in a platform-specific manner: line endings are transparently translated to '\n' in C, and some platforms (CP/M, DOS and Windows) cut off a file when a byte with value 0x1A is found.

These transformations seem a little useless to me.

People share files between computers with different operating systems. Text mode would cause some data to be handled differently across some platforms, so when this matters, one would probably use binary mode instead.

As an example: while Windows uses the sequence CR LF to end a line in text mode, UNIX text mode will not treat CR as part of the line ending sequence. Applications would have to filter that noise themselves. Older Mac versions only use CR in text mode as line endings, so neither UNIX nor Windows would understand its files. If this matters, a portable application would probably implement the parsing by itself instead of using text mode.

Implementing newline interpretation in the parser might also remove some overhead of using text mode, as buffers would need to be rewritten (and possibly resized) before returning to the application, while this may be less efficient than when it would happen in the application instead.

So, my question is: is there any good reason to still rely on the host OS to translate line endings and file truncation?

  • 1
    -1 - With the zero context you provided, it is impossible to even approach answering this question.
    – Oded
    Nov 19, 2011 at 17:05
  • 1
    How is your question related to software development? Can you provide some context? Thanks.
    – Adam Lear
    Nov 19, 2011 at 17:11
  • I get easily confused between SO and programmers. My apologies. If this is out of scope for p.SE, can it be moved to SO instead? My reason for posting it here, is that it's more about general guidelines than specific programming questions. I'll update the post for context as well.
    – Rhymoid
    Nov 19, 2011 at 17:18

3 Answers 3


Without the translation, every Unix text-processing program would recognize just '\n' as an end-of-line marker, and every Windows text-processing program would recognize '\r' followed by '\n'. (And pre-OSX Mac programs would recognize '\r'.) And any program that writes text would have to explicitly write the local end-of-line marker, which means it would have to be aware of what OS it's running on.

And that just covers the relatively simple cases where an end-of-line is indicated by a sequence of characters. Other schemes are common (though less so these days); see IBM mainframes and VMS, for example.

With the translation, programs can just treat text as text, and we don't need three or more different "hello, world" programs.

This does sometimes cause problems when you need to process a foreign file (a Windows text file that's been copied to a Unix system, or vice versa). Cygwin, a Unix-like environment running under Windows, is a rich source of such issues. But usually the best solution is to translate the file before processing it. And most of the time, programs deal with text files that were created on the same OS anyway.

It's better to write one program that can translate between formats, than to require every program to deal with all different formats. And inevitably someone would write a tool that understand Unix and Windows formats, but breaks when confronted with an old Mac text file, and someone else would get the interpretation just a little bit wrong because the wheel they reinvented wasn't perfectly round.

  • Would there be any disadvantage to having the I/O library behave like PostScript does, and accept either CR or LF as a newline, but ignore any CR that follows a non-ignored LF or any LF that follows a non-ignored CR?
    – supercat
    Jun 1, 2015 at 6:55

While Unix (MacOS-X is Unix for the purpose of this) and Windows consider files as a stream of bytes and differs only in some little details about what is a text file and what isn't, when other OS come into play you have more variety. Three examples:

  • in some OS, text file are a set of fixed width lines and lines are padded with spaces up to the fixed width (and you know understand why C and C++ don't guarantee that spaces at the end of a line are conserved)

  • others OS have a notion of permanent line numbers and the C runtime has to skip in,

  • the system call (to use a Unix terminology) used to open a text file may be different than the one used to open a binary one.

You could probably imagine a way not to specify it, you'd have to drop the interoperability for your C or C++ programs with those abiding the OS convention.

  • 2
    That sounds arcane. Mainframe-like arcane. Could you name the operating systems in the first two cases?
    – Rhymoid
    Nov 19, 2011 at 18:50
  • For the first one, I'm just an indirect source, and I don't remember the name if I ever knew it. VMS could be in the case (its file handling is quite complex, IIRC it has even indexed file as an OS feature, but I've never programmed it at that level) and mainframes are obviously candidates. For the second one, TOPS-20 (a PDP-10 OS) was in that case. Nov 20, 2011 at 9:06

With reading in text mode, you have something like 'characters', binary knows only bytes.

At least in Ruby 1.9 you can define an encoding, when you read a file in text mode.


File.open('sample.txt', 'r') # Read in default encoding
File.open('sample.txt', 'r:utf-8') # Read a utf-8 file
File.open('sample.txt', 'r:utf-8:cp1251') # Read a utf-8 file and convert it to cp1252

Doing this, you have a possibility to make differences between bytes and characters. When you read binary, you have only bytes.

  • 1
    Interesting (it would be much more useful), but very language-specific. For most operating systems, this conversion step is probably implemented in Ruby itself, while newline conversion can be done by 'deeper' parts of the host OS.
    – Rhymoid
    Nov 19, 2011 at 21:12

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.