Assume I'm writing a library in Java for performing network calls. I want it to be efficient for cases like highly loaded reverse proxies etc (10s to 100s Krps coming in and out; client requests served in the same DC in under 1 ms; relatively simple processing of results), also, each incoming request might start several outgoing ones and combine results in a simple way.
The library uses non-blocking client sockets internally, but I want to expose some higher-level abstraction to the users of this library. I've started with
CompletableFuture, but I'm feeling like it might generate too much garbage: each "composition" (
thenCompose etc) creates a new object; also, it encourages using lambdas which are also separate objects (right?). The final future might consist of several stages (3-5 really: gathers, retries etc), so that feels like a bit too many objects per request.
What other options do I have that would provide at least some "composability" (read: can be used by library clients, and maybe by the library itself) but will generate less garbage?
Is it possible to have a "reusable" event loop of sorts? Or can the stream (reactive) paradigm help? Good things are said about coroutines which are equated to state machines for languages that don't support them natively, but I'm having trouble imagining how implementing that might even look like for a scenario like "call several services with retry, when done with some/all, return back a result, give means for the library clients to tap into that or enhance that"... We might have a number of processors connected by queues (blocking queue in the simplest form), but I have doubts about the efficiency of such system: these queues don't come for free, there will be elements added, elements chopped, all these will migrate through the whole "pipeline"... Or maybe Disruptor is my only hope? (Not that I understand how to apply it here.)
For these approaches pseudocode or a scheme describing how it might work or a link to an explanation is highly welcomed because I sort of know "about" them but don't understand how to use them or if they'll work in my case. Or maybe there's a still better approach that I completely missed?..
Edit: I've forgotten to mention that I'm actually extending an existing networking library to support asynchronous usage. The library reuses objects very heavily (buffers, sockets, response objects etc.), so, first, I thought maybe it was really important in this context; second, I'd like to pursue the same design goals if possible.
Some back of the envelope calculations supporting this desire: suppose there are 4 futures per request (the original one, retrying logic, gathering several results logic, and then client logic). Suppose there are 50K server requests coming per second and each translates to 2 client requests. So that's already
4 * 2 * 50_000 * (object_size) bytes per second assuming that callbacks are reused. Assume each object takes 20 bytes (header, at least one listener, result reference). Assume callbacks are reused so there are none created and collected. That's 8Mb per second, but easily more. That's 1 Gb of garbage per couple of minutes. Maybe not too much, really, but still would be interesting to understand if that could be improved.