Should I parse XML on the server or provide a proxy and let the browser parse it?

Question

I need to interface with a 3rd party API. With this API I make a GET request from within the end user's browser and receive an XML response. This data is to be used in a browser based application where the user can search through it, use it to make decisions, etc. The main issue is that most browsers have locked down cross-domain XML use, so I can't simply get the XML from the API.

The overall data, though, is basically broken into two sets.

1. The first set of data is public and only needs to be updated every so often, so it can be cached for all users on the server side, lightening the traffic considerably.
2. The second set of data is private and individual to each user. This data is also updated in the API more frequently. This leads caching to be much less effective.

For scalability reasons I would like to keep the server's load a small as possible.

I see two options before me:

1. Provide a proxy that can be used to route XML requests to the 3rd party server and directly back and forth between client and 3rd party API.
2. Have the server do the conversion from XML to JSON and strip out unnecessary information. This essentially means making a new API for our server, which translates into requests from the 3rd party API

What would be the best way to provide the data to the user? (Does not have to be one of the two options)

Answer 1

The proxy option is the easiest one to implement. You don't have any custom development to do, the only thing to do is to set up a proxy. It's also straightforward: there is no additional code to maintain, and if the API changes, you have no changes to make on your side.

A proxy would be a preferred choice:

• If you need to ship working software fast. This makes it a good choice, for example, if you were about to ship a feature, but found during the implementation phase of the project that you can't just make cross-domain AJAX requests.

• Or if the current API is well designed: the architecture is good, the calls are very clear, the documentation is complete and easy to understand.

• Or if the current API is subject to change. If it changes, you just need to change the JavaScript implementation. If instead of a proxy, you are parsing the results and generating your own JSON, there is a risk that the changes to the API will require the changes in your server-side code.

On the other hand, parsing the result has a benefit to make it possible to abstract completely the API on client-side. This is a slower alternative, since it requires to design the new interface (if the original API is not well designed) and to implement the extract, transform and load features, but it may be a good long-term choice for a large project. This is a preferred choice:

• If you need additional features. You can exploit the different features which weren't available in the original API, such as caching on a level which is not supported by an ordinary proxy server, or encryption, or a different authentication model.

  For example, if the number of AJAX requests becomes an issue or if two-ways communication model makes sense, you can implement Web Sockets.

• Or if the current API is not well designed. Like a facade pattern, this approach enables you to redesign the API. If the original one is poor, having a facade makes it possible to solve the bad design choices made by the original authors of a legacy API. You can act as well on large parts, such as the overall architecture of the API, but also on details, such as the names of arguments or the error messages.

  While modifying an existent API is sometimes impossible, having a facade can make it possible to work with a piece of clean code which abstracts the drawbacks and errors in the original design.

• Or if the current API is subject to change. Indeed, you may prefer to change server-side code instead of JavaScript if the API changes over time, while keeping the public interface of your facade unaffected. It may be easier either because you're more experienced with server-side programming or because you know more tools for server-side refactoring or because it's easier in your project to deal with server-side code versioning.

You may notice that I omitted talking about JSON, performance, caching, etc. There is a reason for that:

• JSON vs. XML: it's up to you to pick the right technology. You do it by measuring objectively the overheat of XML over JSON, the time it takes to serialize data and the ease of parsing.

• Performance: benchmark different implementations, pick the fastest one, then profile it and optimize it based on the results from the profiler. Stop when you achieve the performance specified in the non-functional requirements.

  Also, understand what are you trying to achieve. There are several parts interacting with each other: the original API, the bandwidth between your server and the API's one, the performance of your server, the bandwidth between your server and the end users and the performance of their machines. If you're asked to obtain a response to a request within 30 ms., but the original API spends 40 ms. processing the request, no matter what you do, you won't be able to obtain the required performance.

• Caching: caching is one of the techniques to make your web application feeling faster, reducing bandwidth, etc.

  1. Make sure you use client caching as well (server-side caching won't reduce bandwidth usage between you and the customers), given that setting up the HTTP headers properly is often tricky.

  2. Make sure you determine correctly what to cache, for how long and when to invalidate it: if the description of the product changed 10 seconds ago, but the customers of an e-commerce website still see the old version, it's OK. If the owner changed the description, submitted it, and still sees the previous variant because of the caching, this is problematic.

  3. Don't focus only on caching. Minification, for example, is important as well. Reducing the number of requests can also be beneficial.

Answer 2

There is a third option which you may not have seen: Cross Origin Resource Sharing (CORS).

The CORS standard works by adding new HTTP headers which allow servers to serve resources to permitted origin domains. Browsers support these headers and respect the restrictions they establish.

Example: Say your site is http://my-cool-site.com and, you have a third party API at domain http://third-party-site.com, which you can access via AJAX.

And let's assume that a page you server from my-cool-site.com made a request to third-party-site.com. Normally, users browser will decline AJAX calls to any other site other than your own domain/subdomain per the Same-Origin Security Policy. But if the browser and the third party server supports CORS, the following things happen:

• Browser will send the following HTTP header to third-party-site.com

  Origin: http://my-cool-site.com
  

• If the third party server accepts requests from your domain, it will respond with the following HTTP header:

  Access-Control-Allow-Origin: http://my-cool-site.com
  

• To allow all domains, third party server can send this header:

  Access-Control-Allow-Origin: *
  

• If your site is not allowed, browser will throw an error.

If the client's have fairly modern browsers that support CORS, and your third party server supports CORS as well, you can definitely go for it with minor changes to your code.

I found a nice explanation on CORS, on which you will also find another way to do this: JSONP. But JSONP would require a fair amount of changes to your code.

To make a CORS request you simply use XMLHttpRequest in Firefox 3.5+, Safari 4+ and Chrome and XDomainRequest object in IE8+. When using XMLHttpRequest object, if the browser sees that you are trying to make a cross-domain request it will seamlessly trigger CORS behaviour.

Here is a javascript function that helps you create a cross browser CORS object.

function createCORSRequest(method, url){
    var xhr = new XMLHttpRequest();
    if ("withCredentials" in xhr){
        // XHR has 'withCredentials' property only if it supports CORS
        xhr.open(method, url, true);
    } else if (typeof XDomainRequest != "undefined"){ // if IE use XDR
        xhr = new XDomainRequest();
        xhr.open(method, url);
    } else {
        xhr = null;
    }
    return xhr;
}

Since you say that "most browsers have locked down cross-domain XML use", I guess your third party server might not support CORS. Then you have to find alternative approaches.

---

Answer 3

For scalability reasons I would like to keep the server's load a small as possible

I think this is more or less pointing to the answer. Whether or not providing preprocessed data to the client or not depends mainly on:

1. the difference with regard to traffic
2. the performance impact of the processing
3. the impact of a different data format on the client

If the XML is comparingly small or there are only a few requests, it may make sense to just forward it to the client and forget it. Same is true when the preprocessed data are still a large part of the original data, or if the client is not able to profit much from a different data format (say JSON, for example).

However, if the client struggles with processing a large XML data set, or if the client needs only a small fraction of the original XML data, then it may make sense to do some preprocessing on the server side.

At the end, it is easier to scale a server, than it is to scale a client/browser or the available bandwidth. To put it in one sentence, it depends on where the bottleneck is in the system.

Answer 4

My choice would be to cache and compress (throw away unnecessary information) and gzip results to the client browser, your option #2. Because browsers are not usually high-end CPUs and the server to browser network lines are of limited capacity. I'm talking about mobile clients. If you don't plan to support mobile clients then choose whatever is simpler, e.g. some Google:CORS proxy