We’re rewarding the question askers & reputations are being recalculated! Read more.

Hot answers tagged

15

A mainframe is designed for processing large amounts of information by the use of batch transaction processing. It particularly works well at running scalable software and dealing with massively parallel operations. Everything about mainframes is screamingly fast. Mainframes are typically built by IBM and usually run z/OS. A server (when referred to in the ...


13

I personally suggest Markov clustering. I have used it several times in the past with good results. Affinity propagation is another viable option, but it seems less consistent than Markov clustering. There are various other options, but these two are good out of the box and well suited to the specific problem of clustering graphs (which you can view as ...


10

Hierarchical Clustering This was recommended to me by a friend. According to Wikipedia: In this method one defines a similarity measure quantifying some (usually topological) type of similarity between node pairs. Commonly used measures include the cosine similarity, the Jaccard index, and the Hamming distance between rows of the adjacency matrix. Then ...


6

It can be expensive and painful, but in the end you need to have a local "cluster". Trying to simulate race conditions, contention and the like are very hard on a single PC (my interpretation of "local development environment"). From past experience I would suggest: Push very hard to get a production level cluster into your test/dev environment, You can ...


5

I've written a lot of .NET clustered applications over the years (exclusively for Active/Passive clusters) and can share what I've learned. I've always written my apps as Windows Services and include them as clustered resources in the Windows Cluster Administrator. When the cluster fails over from Node1 to Node2, the Cluster Administrator shuts down the ...


5

For your problem here, I think you should think of a way to map vertices-edges to a set of coordinates for each vertex. I am not sure if there is a better way to do this. But, I think you could start off by representing each vertex as a dimension and then, the edge value to a particular vertex would become the value you need to work with for that particular ...


5

Several possible downsides or issues you have to code for: Login sessions must either be stored in a central database (such as redis) that all clusters can access or connections must be made sticky so that a given client goes back to the same cluster process every time. Other server side state is either maintained separately by each clustered process or ...


3

If you are considering only a scaled out, clustered environment, with replicated Sessions, then you don't have a workaround. All of your objects must be Serializable. But if your application and Java EE architecture allows you to do a scaled out "clustered" environment without replicated Sessions, then you are fine. The only thing you will lose here is the ...


3

My first suggestion is to cut up your problem into two problems: first, figure out what you want and then figure out how to efficiently get what you want. You can't efficiently get something you haven't defined yet. I'll put some ideas in this answer that may help you find this definition. I suggest you make an inefficient implementation of the ideas you ...


3

Good question. Probably hard to say which is best approach, but will try from my experience. The best way to scale the Java based web application is to write it as stateless as possible (if you could). This allows you to horizontally scale the application, where you can add tomcat servers if there are more concurrent users. However, as you noted, there ...


3

For an algorithm with very little communication between computers (not sensitive to bandwidth limitations) that uses asynchronous communication (not sensitive to network latency), number of CPUs will dominate performance and networking is mostly irrelevant. For an algorithm with high communication between computers (very sensitive to bandwidth limitations) ...


3

I can recommend the Hangfire as a solution. Key highlights for your case (extracted from the web site), Persistent Background jobs are saved into a persistent storage – SQL Server, Redis, PostgreSQL, MongoDB and others. You can safely restart your application and use Hangfire with ASP.NET without worrying about application pool recycles. Reliable Once a ...


3

I'd suggest to define criteria of what "dead" means, then periodically poll for the "dead" condition and perform the swing over. Perhaps "dead" gets defined as "hasn't sent any messages to any of the nodes in X seconds". Whatever decision tree a human currently follows to ascertain whether or not to flip service. It may be 1 condition, 10, or dozens. How ...


2

Mbps is not the only measure to measure network speed, the other arguable more important one is setup time (how long it takes for a pair of CPUs to be ready to send a packet) which is in many cases an order of magnitude larger than straight up bandwidth. This is the reason why most algorithms try to chunk as much sending as possible This means that as soon ...


2

answer of the first question is, as you said, it will either timeout or get some error messages from server. it depends on how you design your program. I suppose some of nodes in the cluster are working as api servers as it's a reasonable configuration. If you don't have an api server in the cluster, I believe you must have to set it up in somewhere else. ...


2

I'm currently setting up a similar system (on a professional level) and this is the design I've chosen: Two Nginx loadbalancers (both active, both failover for the other, balanced with DNS round robin) Two MySQL Databases in master master replication mode Two Tomcat instances as a tomcat cluster Two Memcached instances for both caching and session state ...


2

Since you are moving away from a single master node (which is appropriate) you will have to change a few things. You will need to setup a Quorum. Since you already have 9 nodes, you are in good shape. For a Quorum to work you need 2n+1 nodes where (n) is the number of nodes that can go down and the system will still work. Within the Quorum a vote will take ...


2

Using modular decomposition you can create a tree that will contain all particles as leafs and upper nodes will cluster these. Based on that tree you can define measures that are applied to every node of it from the root to the leafs downwards. You stop this downwards traversal when the measurements reach user defined thresholds. One such measurement may be ...


2

The basic question "one or two servers" is the wrong question asked, IMHO. As I understand your question it should be more asking if the "Pull" and the "Push" functionality should be handled by the same server application. My plug would be to separate these functionalities as best as possible, makes the whole stuff easier to develop and maintain. That said,...


2

The entire point of Zookeeper (as I understand it) is to make restart AFTER the partition goes away simple by making sure that there's only ONE portion (the majority) that was changing during the partition. It can then bring the minority up to speed when they reconnect and everything is running again with no further intervention on your part. If you let two ...


2

You don't need any DB operation for synchronization between the operations. If you need all the 3 operations to be performed on your data sequentially : Send your Data to Queue A, which has consumers which perform operation A and at the end of it, send it to Queue B. Consumers on Queue B will perform Operation B and send the data to Queue C where ...


2

We have a cluster-environment too. We use Hazelcast for such jobs. With Hazelcast you could embed the codeblock for updating within a "Hazelcast-Lock-Section". It is not my favourite solution, but this is how it is done in our application (and maybe suits hazelcast your needs). I opt for a smaller and easier solution: I would write a small (buzzword-alarm: ...


2

Are you losing anything ? Depends on whether you are dependant on the j2ee specs. For example JCA, If not then stay away. Stateless services are way more flexible in terms of Scalability. And more over its easy to refactor to microservices.


2

The bandwidth is the distance/size scale of the kernel function, i.e. what the size of the “window” is across which you calculate the mean. There is no bandwidth that works well for all purposes and all instances of the data. Instead, you will need to either manually select an appropriate bandwith for your algorithm; or use an algorithm that automatically ...


1

It just occurred to me that although the 2 member scenario could indeed be made more available than it is right now, it would still not be more available than the 1 member scenario, and writing is in fact slower for 2 members, because every change needs to be propagated to another member. Only reads from ZooKeeper will improve under heavy load (assuming the ...


1

Using a messaging broker seems like the best tool to inform other applications that they should update rather than polling (which is fine really too). http://www.rabbitmq.com/ Locking the database table for writing is what you want to prevent race conditions.


Only top voted, non community-wiki answers of a minimum length are eligible