1

I want to create a stream class The input stream should read/parse a continuous range from left to right providing convenience methods. The implementation isn't a problem but choosing a consistence API.

A basic example would be:

input_stream is("Hello! 42");

is.peak(); // 'H'
is.read_n(7); // "Hello! "
is.read<int>(); // 42

I also want to support an operation known as read until to read the input stream until a specific string occurs. On first glance, this doesn't sound very problematic but if you take a closer look there are 3 possible outcomes:

input_stream is("Hello! 42");

is.read_until("!"); // ?
  1. function returns "Hello!", input stream points to " 42"
  2. function returns "Hello", input stream points to " 42"
  3. function returns "Hello", input stream points to "! 42"

I want to support all 3 variants with 3 different functions which should be unambiguous to the reader but it is really hard to explain what actually happen. Here are my ideas:

• read_until_include(cmp) - process the cmp and returns the part before + cmp

• read_until_exclude(cmp) - process the cmp but returns only the part before

• read_up_to(cmp) - does not process the cmp and returns only the part before (leaves the cmp in the input stream)

I am not a big fan of my own idea. Could anyone please give me an advice where to go from here?

EDIT

First of all, thank you for your answers. I think I removed to many important aspects so the questions is to general. First of all, the target language is C++20. And the overall implementation shall not use any heap or exceptions (bare metal embedded). So everything is preallocated and we take views out of the input stream.

So the API should look like this:

Error read_until(std::string_view &dst, std::string_view cmp);

An error code indicates the success of the method. For read_until it would return something like EOF if cmp wasn't found, otherwise OK(0, default initialized).

As I said before, there 3 possible versions for read until:

  1. read (into dst) the part before cmp and cmp itself. Similar to a regex of (.*?cmp)

  2. read (into dst) the part before cmp and process but skip cmp itself. Similar to a regex of (.*?)cmp

  3. read (into dst) the part before cmp and stop at cmp itself. Similar to a regex of (.*?)(?=cmp)

If I try to reduce my API to only one version and construct the other two cases from it, it doesn't feels straight forward.

Below, all versions (ignoring error handling for now)

Version 1

Construct version 2

std::string_view dst;
is.read_until(dst, "\r\n");
dst.remove_suffix(2); // Size of cmp has to be known here.

Construct version 3

std::string_view dst;
is.read_until(dst, "\r\n");
is.seek(-2); // Move read position. Size of cmp has to be known here.

Version 2

Construct version 1

std::string_view dst;
is.read_until(dst, "\r\n");
dst = std::string_view(dst.data(), dst.size() + 2) // Size of cmp has to be known here.

Construct version 3

std::string_view dst;
is.read_until(dst, "\r\n");
is.seek(-2); // Move read position. Size of cmp has to be known here.

Version 3

Construct version 1

std::string_view dst;
is.read_until(dst, "\r\n");
is.read_n(2)
dst = std::string_view(dst.data(), dst.size() + 2) // Size of cmp has to be known here.

Construct version 2

std::string_view dst;
is.read_until(dst, "\r\n");
is.read_n(2); // Size of cmp has to be known here.

Looking at all versions now, version 1 looks best to me. All other may require to reconstruct the string_view. What is your opinion to this?

1
  • 3
    "I want to support all 3 variants with 3 different functions" - I think that you should seriously consider not supporting all 3 variants. I know it's tempting to provide convenience methods, but just because you can do something, doesn't necessarily meant that you should. Consider, for example, supporting only option 3, along with read_n and skip_n; this lets the client code decide. In fact, maybe read_until, read_n and skip_n should be methods on a "stream reader" that wraps the stream. Alternatively, the read_* methods could take an output parameter. Jul 24 '21 at 20:52
2

Slightly differently named API functions doing almost the same thing are a recipe for confusion and coding errors. You should have very good reasons for including all three versions. Since read_up_to() can be used easily to construct the other two cases if necessary, it would be my personal choice, but depending on the planned usage one of the other versions might be preferable, especially when you support back-positioning easily.

Another option that should be considered for a general use stream library would be to support parsing using more powerful paradigms, such as regular expressions or full-blown parsers. If your users only have an operation like read_up_to() they will most likely attempt to perform complex parsing tasks using the available operation. To see this in action, look at all the requests for regular expressions to parse HTML...

1
  • Thank you. For parsing messages which are separated by a delimiter, read_until_exclude would be my choice. But than, libraries like boost asio's read_until is more a version of read_until_include. My target platforms for this library, where stream is a small part, are embedded systems (cortex-m).
    – Viatorus
    Jul 24 '21 at 18:12
1

I agree that you should probably only implement one version, but I disagree about which one. I'd go for option 2 (consume and discard the delimiter) for the following reasons:

  1. It's functionally equivalent to std::getline (not necessarily the best reason, but the underlying behavior might make it more intuitive to use).

  2. It's the simplest to use for reading strings between delimiters in a loop:

    v2:

    while (!is.at_eof()) {
        auto line = is.read_until("\n");
        ... process string
    }
    

    v1:

    while (!is.at_eof()) {
        auto line = is.read_until("\n");
        line.pop_back();
        ... process string
    }
    

    v3:

    while (!is.at_eof()) {
        auto line = is.read_until("\n");
        ... process string
        is.read_n(1);
    }
    

    (Of course, this will get more complex for all three versions if you want to handle Windows-style "\r\n" line endings here too.)

  3. It's effectively how the underlying input works, and could be implemented without any additional buffering. (If you allowed the user to control the size of the stream buffer, they could actually set it to zero, and this version of the function would still work.)

    There's no fpeek() operation in the C or C++ standard libraries. If you want to get the input from the file, you have to read it. The character doesn't stay unread on the file stream.

    (I'm not sure it's really relevant, but note that: C++ input iterators are a dodgy hack around this. If you pass a file input stream to an input iterator, it actually changes the file stream position in the constructor by reading the first character! However, really the file stream is the iterator, and should have it's own set of algorithms, because it can't be used directly with some of the functions in the algorithm header. Since we are providing those separate algorithms to read input (yay), we don't need to care about C++ iterators. :-) )


Some other things to consider:

  • The basic function should probably take a single char. If you want to match any one character in the delimiter string, you could provide a second version taking a string. (Matching a whole exact string should definitely be a separate function IMO).

  • Maybe provide a version that takes a function predicate, so we could do something like: is.read_until([&] (char c) { return c == something; });.

  • It might be preferable to take the output string as a reference parameter. This could help avoid repeated string allocation (see the while loops above).

1
  • Thank you. I made a big edit. Please have a look. I will for sure add a function which takes a function predicate. Good idea!
    – Viatorus
    Jul 25 '21 at 13:28

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.