2

Background: My team writers back ends, whose job is to receive input, perform calculation and either return the result to the caller or to another server down the line for additional processing. We write in C# and our code runs as Windows services.

A few years ago we used to communicate via TCP with other servers (which were, too, windows services written in C#). This approach had disadvantages, mainly its inability to scale to more then one server to handle requests.

Since then the team has started using Rabbitmq and there are no real problem we currently face. When we do projects with other teams we usually manage to convince them to use Rabbitmq client to communicate with our Rabbitmq server. We pass around XML's saying what calculation to be done and what are the arguments and return an XML as a result. The XML'S have XSD's and this approach worked well so far and suited our needs.

Although I am generally happy with the current way things are done, I fear that in a few years we will have a strong dependency in Rabbitmq (we already do) and experience tells me that we may want to change the communication method (like TCP seemed good and that was changed..)

The problem is this: whatever we may choose to use, it seems that we will be highly dependent on that method. And if we would try many things (that you some services in X and others in method and protocol Y) then we will have lower dependency, but on lots of stuff (and the team would have to know how to use multiple technologies), and that doesn't seem to worth it to me.

What can I do to reduce this dependency?

  • If the current method implemented with RabbitMQ is reliable, robust and suits the needs of the system. Why Is that a problem? What worries you most? MqRabbit getting deprecated over the years? Its reliability? Its limitations? – Laiv Mar 30 '17 at 20:51
  • I don't like the title. You're not reducing the dependency on communication, but on a specific technology (from TCP to RabbitMQ and from RabbitMQ to what ?). The question is good, though. – Machado Mar 30 '17 at 21:25
3

TL;DR Your applications should depend upon interfaces which describe message patterns and should not depend upon implementation details pertaining to RabbitMQ or any other message transport technology.

In general, different types of middleware tend to solve the same problems using common messaging patterns - for example:

  • Direct point-to-point messages
  • Message Broadcast / Multicast
  • RPC (Request-Reply)
  • Reliable Guaranteed message delivery
  • Traffic separation / segregation
  • etc.

I have used RabbitMQ in two large C# projects so far - in both cases I have written my own wrapper to encapsulate (hide) all of the RabbitMQ-esque "stuff", but patterns such as request-response or pub/sub are agnostic to any particular messaging technology.

Here are a few examples of the kinds of interfaces I use in the applications which use RabbitMQ - the most important aspect of them is that none of the applications know anything about RabbitMQ specifically, they only know about message patterns and rough data about the overall topology of the system (i.e. identifiers for send/receive endpoints, message routing information, message identifiers, etc.).


Firstly, the interfaces use general-purpose structures which include message-pattern information along with the actual message payloads - nothing here which necessarily needs to be tied to RabbitMQ:

public class Message<T>
{
    // Message data
    T Payload { get; set; }

    // Identifiying "Network Name" of the target/recipient
    // This might identify an application, a physical host address, etc.
    // In RabbitMQ terms, it's a Routing Key. 
    // If null, assume broadcast/fanout.
    string TargetName { get; set; }

    // Unique identifier for RPC (Request-Reply) pattern
    // - Perhaps a GUID would be appropriate for this
    string CorrelationTerm { get; set; }

    // Should the message layer include a provision to guarantee delivery?
    bool IsGuaranteedDelivery { get; set; }

    // TODO - anything else?  stuff like durability and persistence adds complexity
    // but it can be abstracted out here.
}

The interface to the message layer itself doesn't "leak" the names of any RabbitMQ-specific structures, all of the argument types and return types are part of the wrapper library:

public interface IMessageLayer : IDisposable
{
    // Fire-n-forget publishing
    void Publish<T>(Message<T> message);

    // General-purpose method for consumption of received messages
    ISubscriber<T> Subscribe<T>(SubscriptionParameters<T> parameters);

    // Request-Reply pattern for RPC
    Task<Message<TReply>> Request<TReq, TReply>(Message<T> request);
}

An example for a fairly minimal subscriber interface - a callback event for receiving a message, a couple of simple methods to start and stop subscribing.

public interface ISubscriber<T> : IDisposable
{
    event EventArgs<MessageReceivedEventArgs<T>> MessageReceived;
    void StartConsuming();
    void StopConsuming();
}

A short/brief example of how applications might use this:

Broker Connection

var connectionParams = new MessageLayerConfig { /* Broker details */ }
using (var myMessageEndpoint = MyMessageLayerFactory.Connect(connectionParams)) 
{
    // TODO - Use messaging patterns here
}

High-level message subscription pattern in an application:

var mySubscriptionParams = new SubscriptionParams<T> { /* Consumer args */ }
using (var mySubscription = myMessageEndpoint.CreateSubscriber(mySubscriptionParams))
{
    var tcs = new TaskCompletionSource<FooMessage>();
    mySubscription.MessageReceived += (s, e) =>
    {
        // TODO: Handle message
        tcs.TrySetResult( /* receivedFooMessage */ );
    }
    mySubscription.StartConsuming();
    await tcs.Task;
}

High-level message fire'n'forget publishing in an application:

var message = new Message<FooMessage> 
{
    Payload = myLittleFooMessage,
    TargetName = "SomeHostName.SomeAppName",
    IsGuaranteedDelivery = true,
};
myMessageEndpoint.Publish(message);

High-level Request-Reply pattern in an application:

var message = new Message<MyRequest> 
{
    Payload = myRequestData,
    TargetName = "SomeRpcServer.SomeRpcAppName",
    IsGuaranteedDelivery = false,
};
var task = myMessageEndpoint.Request(message);

if (await Task.WhenAny(task, Task.Delay(5000)) != task)
{
    throw new TimeoutException("Request wait time exceeded 5sec");
}
var response = await task;

In general, the key to flexibility is about keeping things simple and try as far as possible only to rely on the most well-known messaging patterns rather than trying to find ways of doing alot of "clever" stuff with RabbitMQ.

i.e. stick with the kinds of message patterns which are easy to replicate in other technologies (either other AMQP implementations or something like ZeroMQ), this way replacing the underlying framework should be fairly straightforward.

If you do decide to follow the rabbit down the hole with all kinds of advanced message patterns such as Last Value Cache, Shovels, Federated Exchanges, etc. then the work involved in replacing those patterns in future will be greater, because you'll have more patterns to replicate.

Lastly, a roll-your-own approach with RabbitMQ isn't the only option; people have already gone one better than this with frameworks such as MassTransit: http://masstransit-project.com/MassTransit/ which works at a much higher level than RabbitMq.Client - its interfaces are much closer to the kinds of message patterns which your applications should be interested in.

Although even using MassTransit, there's still an advantage to writing your own interfaces to wrapper it; the ability to rip out any underlying framework and replace it with something else with minimally intrusive changes to the applications which depend upon message patterns.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.