How do you deal with errors in enumeration / list processing (lowish-level API)

Question

Ive struggled with variants of this problem many times, experimenting with different solutions, happy with none.

Abstract: Enumerating a list of items, an error for one item does not affect the others. Potential callers have varying requirements on how to treat these errors, goal of the API is to simplyfy use and reduce complexity.

A more general case in which this problem occurs would be processing a list of independent items, and partial success is acceptable / desired, i.e. it's not run as a transaction.

---

Example scenario:

I am dealing with a low level C-Style enumeration API like this:

 // gets called for each "stuff" item
 typedef bool (*tEnumStuffCallback)(stuff * item, void * callbackArg);

 // calls "callback" for each "stuff" item. 
 // caller enumeration context in callbackArgs
 void EnumStuff(tEnumStuffCallback callback, void * callbackArg)
 {
     stuff * item = 0;
     for(;;)
     {
         int err = stuff_get_next_item(&item);
         if (err == no_more_stuff)
           return;

         // === TODO: handle "real" errors ===

         // pass item  to callback, stop enumeration if requested
         if (!callback(item, callbackArg))
           return;
     }
 }

For backward compatibility and "caller expectations", I want/need to preserve the pure C style without exceptions. A higher-level wrapper, built on top, might look like this:

 EnumStuffs(std::vector< boost::shared_ptr<stuff> > & items);

The caller may need one of the following strategies when enumeration errors occur:

• Stop enumeration on first error, e.g. because caller will ignore the entire list and enumeration is costly. A C++ client might want to throw an exception from the callback (allowed).
• Ignore the items - e.g. when enumerating connected devices, and looking for a specific one to be available, and device not found errors are handled separately
• Ignore, but log diagnostics in a caller-specific way (i.e. caller needs to know abotu errors)
• Caller needs detailed list of errors for further analysis, e.g. items may be "retried" on certain types of errors

Note that this is the callers choice, I can't force a decision.

---

Note / Context Example idealized, in practice, I have multiple enumerators involving different types; sometimes I have to provide the enumerator, sometimes the enumerator function comes from the OS. Error information can be more complex than a single int. The underlying API involves dozens of rather confusing types already, I want to avoid introducing the same amount of additional/wrapper types. Bridging different development styles is actually part of my job description, opportunities for educating either end is tricky (I have to pick my battles).

---

My ideas / solutions

• provide a separate callback for the errors. Enables all scenarios, but bloats the interface.

• Replace the enumerator function with a stateful object. Increases number of types, does not provide backward "style" compatibility.

• pass the errors to the enumerator. that's sometimes a bit better than the previous solution, but usually requires an "enumeration context" structure instead of separate arguments. Goes against the plan to avoid additional types for each enumerator

• introduce an "error type", caller can either request a "list of errors" as a result, or get the "stop/throw at first error" behavior by default. I've used that before, but in the end the detailed information is typically used only to build some diagnostics, and isn't "liked" as it makes it "complicated to deal with errors".

---

Questions

What are your thoughts? Do you have ideas for, or have you used other approaches? Under what circumstances would you use / not use above approaches?

---

Answer 1

Short Version:

You say it's the caller's choice how to handle errors, so implement your first idea. It's simple and one more argument can't be considered interface bloat, in fact, in your case it's even desired as it tightens your contract between caller and callee! It's quite easy to use and the caller manages all the errors.

---

Long Version:

provide a separate callback for the errors. Enables all scenarios, but bloats the interface.

I can see no interface bloat when you just pass a second function pointer for error handling if it's the callers responsibility to handle them. Three arguments ain't that much. You'd have to pass a third argument anyway when you'd try to apply a strategy pattern. Do you want one callback for each error? Why not have one callback for all errors? The callback could take the current error as an argument and react to it. It should have a way to stop the enumeration or ignore the item, too, which can be done by a custom return type (best is probably a simple enum: continue_processing/ignore_item/abort_processing). Usually I'd advise using a strategy pattern here, but if you can't control the callers, that is, you can't make them write strategies or you want to keep the interface in pure C-Style, then use the simple function pointer callback (basically a "hole in the middle"-pattern)! TBH, imho it's even desired to have the error-handler in the methods signature, as it communicates the contract much clearer.

Replace the enumerator function with a stateful object. Increases number of types, does not provide backward "style" compatibility.

Homogeneity is sacred. Always strive to have one clear way of dealing with stuff, that'll make the API more intuitive to understand, so less learning of particular exceptions is necessary. Only do so if it isn't possible to provide a "standard" interface. I don't think you should encapsulate each enumeration in a stateful object just because you have a "hole in the middle" (I'll talk about that later as well). That could violate DRY principles. If you can have a base class on the other hand and just have objects that deal with the custom error handling, that'd be an acceptable design. But I wouldn't sacrifice homogeneity if there're other solutions that work equally well.

pass the errors to the enumerator. that's sometimes a bit better than the previous solution, but usually requires an "enumeration context" structure instead of separate arguments. Goes against the plan to avoid additional types for each enumerator

That's basically idea 1 just with a list of errors and handlers (the context). Imho it's just the complicated version of idea 1. Keep it simple.

introduce an "error type", caller can either request a "list of errors" as a result, or get the "stop/throw at first error" behavior by default. I've used that before, but in the end the detailed information is typically used only to build some diagnostics, and isn't "liked" as it makes it "complicated to deal with errors".

So you get a list of errors before you enumerate to which you can then subscribe with a callback? Again, sounds like a complicated version of idea 1. Again: Keep it simple.

What you are encountering is called "hole in the middle". The functional programmer's have long solved these kind of problems by passing functions (which are first class citizens in such languages). A function pointer (as in idea 1) can achieve the desired behaviour almost as easy. The interface/signature is just one function pointer more and as the responsibility of error handling belongs to the caller, it's even desired to have this in the interface/signature.