0

Recently I stood up a Redis server and tested the idea of storing application 'states' that only existed in our application and were never stored in our database. Don't know if we will ultimately do this but for the first time this information was available external to the application. What I did notice above all else was the speed of doing this thru Redis.

Now I have a crazy idea and I wonder if anyone has ever toyed with this before (or can point to any experience or papers related to the idea.) If my system requires message passing (say CQRS for example) can I just store the message data as a record in Redis and pass only the key? I realize that physical boundaries can affect this but in my situation everything (both front, middle and backend) will have access to the same information. Has anyone ever tried just passing keys for messages?

I can think of all the normal objections like writes failing, failed deliveries and so on but those problems also exist if I pass fully fleshed out messages. Why am I crazy?

  • 1
    What do you hope to gain by this? You've changed an "a -> b" exchange with "a -> r ; a -> b ; b <- r" i.e. increased the complexity quite a bit. (Not that there are no use cases, but you need to figure out what you're trying to gain first.) – Mat Feb 11 at 18:45
  • 1
    Passing a single key (certainly with a timestamp and some kind of CRC to ensure against transmission errors) will reduce the network traffic. A command in my system might have over 200 keys in addition to numerous text and numeric fields. Passing only one key would reduce not only the size of the payload but also the possibilities for error. I appreciate the point about the complexity but if I am just deserializing json into a local object for processing then how does it matter whether I am using a key to get that string from a high speed data store or reading it from a transport queue? – TerrierJack Feb 11 at 20:51
  • As I said, there are use cases. But you're not reducing overall traffic, you're increasing it (unless the other side doesn't always need the value). How could introducing an extra third party reduce the risk of error? You already need a "transport queue" or whatever to exchange the keys, so that complexity you already have, the external store is just extra. – Mat Feb 12 at 9:05
  • This may sound crazy too, but if you just pass keys and have the data in one place, you just have a glorified database system. You will not benefit from all the other messaging benefits (scalability, etc). – TMS Jul 13 at 22:47
1

If my system requires message passing (say CQRS for example) can I just store the message data as a record in Redis and pass only the key?

Yes.

The reference you should go review is Rich Hickey's talk The Language of the System, where he discusses names, values, references, and some of their implications.

But if you squint a bit, you may see that it looks a lot like Git. We apply some hashing function to some specific version file, which gives us a name. We then store the (key,file) pair in the object store. Then, anywhere else we want to refer to that specific version of the file, we can just use the name instead. Anything that has read access to the object store and a copy of the name can get the contents of the file for themselves.

An important element here: the name refers to the same data always; when we make a change to the file, we calculate a new name, and store that. So a name lookup fails, or it succeeds, but we don't ever have to worry that the data of a successful lookup will change depending on time.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.