API Design - Observable vs Callback

Question

We as a team are writing an C# SDK which communicates with a Server endpoint. All our API's till now have been Task based. Like

Task DoOperationAsync()

Recently we across a need for API which gets an intermediate update (file location) before the final one if the user opts for it.

DoSomething(bool interestedInIntermediateFile)

There are two option which came on the table

1. Rx.IObservable < OpResult > DoSomething(bool interestedInIntermediateFile)

2. Task < OpResult > DoSomething(Action < string > intermediateFile)

Where OpResult { Enum CompletionReason, enum FileType, string FileLoc }

FileType - Intermediate, Final

CompletionReason - Reason (only applicable when FileType is final)

Wanted to know from the experts which would be a good API design.

Notes : We do see that some of the new future API would be observable based. So it will not be case that this is only observable API.

---

Adding a additional note - since it might not be so obvious from above.

CompletionReason contains a value only in the final update the intermediate update will have this as null. As the intermediate update, final result though related but not exact (in terms of populating fields) has been raised as a downside to observable. Is it a genuine downside.

Thanks in advance.

Answer 1

The decision on what technology to use is largely going to be about the skills in your team, and the audience to which your SDK is aimed, and to some extent whether or not your API will in the longer term benefit from being a provider of observable streams.

Rx.NET is a very powerful and flexible way of doing things, and I would recommend any .NET developer who needs to handle concurrency and/or streams of events to look into it.

(I assume that by "IObservable" you mean Rx.Net. Or are you looking into Akka Streams or some such?)

However, getting into the IObservable way of thinking can be a steep learning curve.

There is a lot of overlap between Task and Rx.NET. One explanation of the overlap is here but in my opinion is biased in favour of Task.

If you anticipate that your API will be dealing with asynchronous streams, such as returning long lists of data, or driving a push model, or wanting to combine multiple streams, then look into Rx.NET

If you anticipate that your API will benefit from LINQ over streams (with Rx.NET you can easily query IObservables as they are the 'push' version of IEnumerable, the 'pull' version), then look into Rx.NET. For example, look into .Where .Select .SelectMany .Buffer .Zip .Scan operators in Rx.NET and decide if you will have many uses for them.

If you see that your API will eventually become a push-model stream service and if you have the skills on hand, you might want to seriously consider Rx.NET

If it's just the small problem you described in your original post, it might not be worth the overhead of learning the Rx way unless you already have skills on hand. In essence, your chain of tasks (see Attached child tasks here ) is a dumbed down Rx IObservable. The advantage is that this code is intuitive for many developers. You don't need to view it as IObservable vs Callback, a Task is not a callback, and your child tasks (intermediate steps) can be returned as a collection of tasks (eg: you could compose your tasks as a list like final task + intermediate task, and then wait for When Any or some such, maybe attach the intermediate as child tasks).

---

Update: The concern you raise in your update that the operation completion message will be raised with a null value during intermediate steps makes no sense. Whichever approach you take, intermediate steps will not describe the operation completion reason, unless of course that intermediate step is post-hoc actualized as the final step. In the 1st scenario (Tasks): your intermediate action will not raise a completion reason (null). In the 2nd scenario (Observables): your intermediate actions will not raise a completion reason. What is the difference? What you should do in the case of the Observable is declare it an observable of "action results". This is semantically equivalent to Task (a future result). Declare X types of result message. Eg:

public class ResultMessage

public class FinalResultMessage: ResultMessage

etc

You can then implement Task or IObservable as you wish.

If you go the route of Observable. You can offer .Finally either in the subscriber or the observable to handle the FinalResultMessage.

In short, do not make the API an IObservable but IObservable if you go that route, and apply the same thinking for Task too.

Answer 2

It depends on your API consumer.

Overall Though, I would try to make sure that you API matches to some extent your underlying server calls.

By Providing an Observable you have essentially introduced state into the equation. I need to hang onto that Observable until its finished sending me updates. The client needs to associate that Observable with incoming updates from the server related to the original request.

If your server is sending a constant stream of data then This would be a good match. (although you might want to look into a Stream)

If your server splits the job into two operations, a Start and a PollToGetResult for example. Then you have introduced state where you needn't have. It would be better to expose the separate operations on your API