1

I have scenario, where I got stream and I need to check for some value. If I got any new value I have to store it in any of data structure.

It seems very easy, I can place conditional statement if-else or can use contain method of set/map to check either received is new or not.

But the problem is checking will effect my application performance, in stream I will receive hundreds for value in second, if I start checking each and every value I received then for sure it effect performance.

Anybody can suggest me any mechanism or algorithm to solve my issue, either by bypassing checks or at least minimize them?

  • 5
    If your requirement is to check every value, there is no way around that. You can't check anything without performing a check! If you think performing contains() calls will be too slow, first profile your code, then request faster hardware specifications if it is really too slow. Programmers are usually mistaken about such things. – Kilian Foth Aug 19 '14 at 10:21
4

Use a HashSet which will not add an element if it is already there, its contains() implementation pretty well optimized provided you have a decent hashCode implementation.

1

The likelihood is that using a hashed key for your data structure will be fast enough. Computing a hash function is typically just a few lines of code, and the look-up of a hashed key is just a few more. I would hazard a guess that a well-written Java implementation could handle something more than 1 million values per second on typical desktop hardware.

So write a naive implementation using some convenient hashed lookup (dictionary/map), paying attention to the hashing function. Then benchmark it. It will probably be fast enough, and if not you have several options to speed it up. Once you get beyond 10 million per second, you will have an interesting problem, but that's a different question.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.