3

We have deployed a restful web service on a application server (Apache). The volume is getting higher and we do want to scale it. We will deploy two more Apache instances on two more machines.

How do I implement a layer which sends request to each of these based on a round robin strategy?

Currently say I have www.mywebservice.com/employees?id=1 and so on.

Now how I redirect these to different instances? Where would the layer come in is I am confused about as I would need some type of facade.

Improve this question
3
  • Ok like i said , we deploy the war to Apache server. The Apache server has been given a path in a class with annotations. So the server takes control of any requests coming in. So where would i put the code to implement round robin strategy . We do not want to use any product but to load balance it on our own. My question is since the server takes control of the requests ( due to configuration of the paths - /employees ) , how would we get control. Is there any way ?
    – Learner_101
    Feb 15, 2016 at 6:24
  • there isnt a good way of doing this after the networking level. once you hit your appache box and start running ocde against the request, your box is doing work. this is what you are trying to avoid
    – Ewan
    Feb 15, 2016 at 10:01
  • however, you can do it with simple linux networking setups or open source tools if you dont want to shell out for a hardware solution
    – Ewan
    Feb 15, 2016 at 10:02

2 Answers 2

Reset to default
2

The best way to scale servers in my experience is to do so at the networking level with a separate load balancer.

This sits in front of your webservers and directs incomming requests to a particular webserver and can have quite complex rules for doing so.

For example:

You can add version headers to the request and have the load balencer direct requests to the correct version of the service.

You can have blue/green groups of servers which you take out for no down time upgrades

You can assign load based on each servers response time

Etc

The problem with trying to implement this in your application code is that once the request has reached that level your box is already doing work. this is what you are trying to avoid with load balancing

Improve this answer
4

As Ewan said, you'll need a load balancer. Some of them are known as "reverse proxies".

Examples include Pound, HAProxy.

If your webservices are hosted on Amazon Web Services, you can use the built-in Elastic Load Balancing, which also includes auto-scaling and many more features.

Note that before you add more instances, you need to consider:

  • what is the constraining factor: CPU, memory, I/O. This is important in choosing the right servers/instances
  • where is the constraint: SSL/TLS decoding, web server, scripts, or database
  • if you use a database and the constraint is not on the database, whether you'll be using a single database shared my multiple front ends, or if you'll need to have multiple databases (and in that case, whether you're going for master-slave replication, master-master, etc.).
  • if you do writes, and have multiple database instances, how you're going to handle those writes
  • likewise, if you store temporary files, or use sessions with locally-stored data, how you're going to handle requests within a single session going to different instances
  • whether it may be needed to have sessions "stick" to a specific instance

Some cases are very easy to split among many instances. Others require a lot more planning, good knowledge of how your scripts work, and careful consideration about the consequences.

Also, in some cases, it might be easier, more effective, and cheaper, to add relevant caches in the right places that to try to scale the number of instances.

Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.