---- Description

I'd like to optimize read operation in my service. It has basically put and get methods.

On put I accept some numbers in a string format and put them into thread-safe cache with timed expiration policy. On get I have to perform some math.

Put has concurrent nature and get is non-concurrent, each entry in cache expires within 2 min after it has been written, read performes every 2 sec.

---- Issue

When I put something into my cache I don't want to convert string to number, because of the performance hit. BUT when I read from cache I have to do this.

And here is the problem! I don't want to perform double work - convert same numbers over and over again in sequential readings. I want to do this only once.

Seems like I have to add a new cache with mapping string to parsed number! Maybe with the same eviction policy as in the first cache..

---- Question

Do you think introducing additional cache here is a good idea overall?

Or maybe such ideas can be called "over engineering"?

  • 2
    Do you have measurements that show any of this to be the cause of your code not meeting its performance requirements? – Blrfl Mar 22 '19 at 10:33
  • @Blrfl no that's just my guess.. But logically I can see that parsing itself took a huge amount of time, especially when we perform this operation over and over again for the same numbers – Developer87 Mar 22 '19 at 10:43
  • 2
    @Developer87: Do some measurements. Create two versions, one with and one without conversion in the PUT and measure how much that really impacts the performance. – Bart van Ingen Schenau Mar 22 '19 at 11:31
  • 1
    You send bytes over a network in text format instead of binary (probably xml or json) and you wonder if parsing them to their binary representation might be too slow? I think you might be optimizing at the wrong end here. – nvoigt Mar 22 '19 at 12:50
  • 2
    Converting strings to numbers is among the simplest and fastest things a computer can do. Just do it in the put method. Your time spent thinking about this (in the wrong direction) is most likely more than a clever optimization could ever save in cpu time during the entire lifetime of this application. – Hans-Martin Mosner Mar 22 '19 at 15:04

Cache invalidation is one of the hardest problems in computer science. Tread lightly here.

Performance is a very popular way to waste time playing with working code rather than move on to new challenges. Tread lightly here.

That said, if you have access to a good hashing data structure that you could test for previously converted numbers before converting you might be able to see some gains. It really depends on the usage. CPU's are fast and memory fetches are slow so don't expect this to pay off in every case. You're betting that the hash lookup will take less time than the conversion. You're betting that the hash look up will succeed often enough to save more time then is lost when it fails.

If you're into Big O it's easy to jump on this saying O(1) is better than O(n) but keep in mind in the real world n can be small making Big O meaningless.

Also understand that this is a space time trade off. Even if you can make this faster that doesn't mean you got that speed for free. CPU cycles are a limited resource but so is memory. This is going to cost memory. Think about how many numbers you're going to allow this thing to hash. If users get to decide these numbers think about what a hacker can now do to your memory foot print.

Even if you're confident that you have a winner because your math and performance tests say this is the way to go you can deploy only to find out that something is different between your test and operational enviroments that is causing a difference in the number of cache misses. Which means the computer happened to leave the number you want in some slower memory. This is hard to test for, hard to predict, and can mess with your performance numbers.

Just trying to answer the question: "which is better?" can be considered over engineering. But if you have tests that show this is where most time is being spent and making it faster here truly is important it might be worth a try.

However, before you whip out that hash table make sure you've asked: "Why are we converting the same string over and over anyway?" It might be a problem you can solve with a little change to your architecture. If so then slapping this on would just be a kludge. Make sure you really need to do this before you invest to much in doing it.

  • I've heard about this legendary cache invalidation problem before, but I'm not exactly convinced. I actually think threads are harder. We have a static property in our current application that holds a list of things. When someone edits the list of things, we simply null the underlying property and it forces a read from the database the next time the list is compelled. Disco. – Robert Harvey Mar 22 '19 at 14:46
  • 1
    What makes cache invalidation a hard problem are the cases when you're not sure when to invalidate. Sure there are cases when you do know. Doesn't mean you always do. Entire paradigms like reports and events where you just flat out admit the data is stale were invented to avoid this problem. But it still exists, tempting new programmers to pretend they know when to invalidate when they don't. – candied_orange Mar 22 '19 at 14:53
  • Well, some may say that hitting the DB is expensive too and we would be back to the idea of a 2nd cache. For some reason we have accepted caches as the solution to performance issues. That and more threads of course! And both are things hard to reason about and even harder to implement properly to gaing signficant improvements. Usually, the dumbest alternstives are better than countless levels of cache and oversized thread pool executors – Laiv Mar 22 '19 at 16:50
  • 1
    @laiv yes but part of what makes it a hard problem is that it's a solvable problem. When done well it abstracts away the levels. Done done poorly it leaks and you're not sure how much of the levels you even should know about. Now the halting problem is easy. I can just point to Turing's proof that it's unsolveable and I'm done. Easy. – candied_orange Mar 22 '19 at 17:16

Your service seems to be based on HTTP GET and PUT requests. And to make them happen, the HTTP libraries will do lots of string/number conversions (and other things) under the hood, so just one more string-to-number conversion inside the PUT implementation body will not be noticable (unless it's really big numbers with hundreds of digits).

5000 conversions from string to 32-bit integer will take a very small fraction of a second, no matter if you do it in Java, C++ or whatever decent language you choose.

If you really experience performance problems, use a profiler (there are many good options available for Java) to find the bottleneck. I bet it won't be the string-to-number conversion.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.