1

If you had a function that took a handler to a file and then performed some actions that moved the pointer, would you expect it to put the pointer back when it was finished? Additionally, would you generally expect it to rewind to the start before it began its work?

def hash_contents(fp):
    original_pos = fp.tell()
    fp.seek(0)
    # do hashing / whatever
    fp.seek(original_pos)

I've looked around a little to see what the most common assumptions are but I'm not sure what a good rule to follow is. For example, shutil.copyfileobj copies from whatever position the file started in and then consumes to the end.

What are the pros / cons and are there any standard rules of thumb to follow when designing these sort of functions?

This is actually a subquestion of something I was asking over at StackOverflow (Prefer BytesIO or bytes for internal interface in Python?) that really seems more suitable over here.

  • Is there a reason the function has the handle as a parameter? Could you use the filename and close the handle at the end of the function? – Jonathan van de Veen Feb 19 '15 at 14:07
  • Actually, that's sort of the crux of my other question on stackoverflow. Originally I was using file_paths but that was clumsy and inflexible because generally my 'files' are pulled from the network (S3) and shouldn't need to go to disk. My SO question is if it's better to use raw bytes or handles. I'm starting to think raw bytes objects are the way to go to save on thinking about seek etc. Thoughts? – Aidan Kane Feb 19 '15 at 14:28
2

In most cases, it is expected that any function that acts on a file will move the file pointer. There are rare cases where I have used functions to peek ahead in the file, which where expected to leave the file pointer as it was found. In all other cases, I expect the file pointer to move as a side effect of the function.

Functions that do not move the file pointer would break things severely in most case where I access file like objects. However, I might expect a function to has the contents of the file to leave the file pointer in its original position. However, there are cases where I would want X bytes from the current location hashed, and might not care where the pointer moved. I would expect the behavior to be specified for such a function.

While side-effects of function calls are generally discouraged, this is one case where they are expected. Unless you are dealing with code that has reason to do random access (databases of some sort) the expectation is that repeated calls of a function will advance through the file.

Database type code might have code that moves the position to the location of the data, and different code to read that data. Often the index and data files are separate, so it may be appropriate to do repeated reads after a repositioning.

In one case, I expected that a read would track where the file pointer was, so that a subsequent write would overwrite the data just read. While this was the documented behavior, it failed drastically. The write rewound the file and appeared not to have written anything. The result was a loop where data was read to the location requiring and overwrite followed by a file rewind. I changed the code to read from one file and write all data to the new file with the required changes applied.

  • Useful, thanks. Good to hear of a few different scenarios. Looking around in std libs it seemed that generally the apis move the file pointers and don't document it anywhere. And actually, having used those apis I just hadn't even thought about it before. This gives a bit more insight so I'm going to mark it as the correct answer. – Aidan Kane Feb 20 '15 at 11:53
4

Changing the state of a file handle is such a heavy-weight side-effect that I would want it heavily documented in the interface precisely what the method does, and then it doesn't matter which way you decide.

  • Ok. Thanks for the comment. I'm starting to think that file-like objects are just too heavy an abstraction for what I'm doing (my need to document the mutation behaviour of such a simple api is a worrying sign). – Aidan Kane Feb 19 '15 at 14:30
2

Looking at it from the perspective of other languages, in both Java and C++ many libraries and interfaces accept a generic InputStream/istream as a parameter when they need to read something (and then to read a file, you'd pass FileInputStream/ifstream).

Aside from the benefit of being able to read from other input sources than files, input streams are pretty much always considered to be one-way-only, which makes it immediately clear to the programmer that you shouldn't expect it to rewind the file pointer.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.