From a developer point of view which platform would you consider for a large social web application? If you could provide some details on what you consider to be the strengths of which alternative it would be great.
locked by yannis Mar 13 '12 at 20:32
This question exists because it has historical significance, but it is not considered a good, on-topic question for this site, so please do not use it as evidence that you can ask similar questions here. This question and its answers are frozen and cannot be changed. More info: help center.
Read more about locked posts here.
migrated from stackoverflow.com Apr 2 '11 at 23:33
This question came from our site for professional and enthusiast programmers.
I've written the same app on GAE (Python and now Java) and Azure. I'll probably continue to use both, for different things. Here are a few thoughts that I'll keep updating:
Reasons to use GAE:
Reasons to use Azure:
So, no obvious answers. I'm defaulting to App Engine at the moment because of costs and ease of use. I might use Azure for very MS-oriented apps. I use Amazon S3 for downloads but likely won't use EC2 because I prefer leaving everything under the application level to the experts.
I'm clearly biased - I work on the App Engine team doing developer relations - but this is my take:
They're not directly comparable. There's a set of apps you could write for any of them, but you'll be writing a different thing in each case. App Engine provides a restricted runtime environment - no writing to files, no sockets, and so forth - and a non-relational DBMS. But in return, you get a runtime environment that scales indefinitely, and a reasonable degree of assurance that your app will scale as big as you want it to.
Azure, on the other hand, provides a slightly less constrained environment, which lets you write a wider array of apps, but requires you to write more - since you're implementing more of the stack yourself - and provides a much looser assurance of scalability.
Finally, AWS provides the ultimate do-it-yourself solution. They provide the hardware, and the storage, and not much else. You build your stack from the ground up, maintain it, upgrade it, and so forth. Your app scales if and only if you write it to scale, which is no small challenge. But, you get complete control over your hardware.
My advice would be: If your app fits the App Engine model - and a social network app is likely to be a pretty good example of ones that does - write your app on App Engine (Java or Python, your choice). It's cheaper, and it's much easier to write an app that scales.
If your app doesn't fit the GAE model, pick Azure or AWS, depending on if you're writing for the MS stack, and on how much control you want over the execution environment. If most of your app fits on GAE, but small parts don't, you might consider a hybrid - for example, live serving on GAE, but storage on S3, or bulk processing on EC2.
For me, lock-in is the deciding factor.
If you choose for Google, your application will work only on Google. If you find yourself being less than satisfied after some time, you're stuck.
If you choose for MS, your application will work only on Azure. Same thing.
At Amazon, you get (a) virtual server(s) that work exactly like the machines you're used to. Not satisfied? Pick up your app, install on real hardware, done.
- If you're .NET Developer - go to Azure.
- If you're on Python or Java - go to Google.
- If you're on Ruby - go to Amazon
My personal choice right now would be Google with Java (even if I'm .NET most of the time). Think about costs - their schema is difficult to compare.
Check out this article - http://www.infoq.com/news/2008/11/Comparing-EC2-App-Engine-Azure
Like Arachnid, I may be biased, being a googler. However, I'm also an Amazon stockholder, so that bias may partly offset the first;-). No Azure experience (though I also hold MSFT stock, so I hope they, too, do well -- yet another bias;-).
My very simple take on it is that App Engine easily offers you the ability to work (within its limitations) just by coding -- no system administration tasks are needed. AWS is much more flexible, but you will need substantial system administration work (and not really trivial at all) to take advantage of that flexibility. So in the end I'd second Arachnid's suggestion: if App Engine can meet your needs, absolutely go for it; if you need more flexibility, AWS seems the way to go (unless Azure's unknown-to-me capabilities should be a better match -- but I think AWS is going to be more flexible no matter what Azure can do, for example with AWS you can even pick which OS to use, if you need that).
I've just started working with Azure, and am already impressed you can do it in F#: http://code.msdn.microsoft.com/fsharpazure! So far, it's the only cloud platform allowing one to use functional programming in a managed way (of course you can do Haskell in EC2... or Algol 68 for that matter). I'm very impressed with the quality of Visual Studio integration -- you get a local "cloud" to test, DevFabric, with a storage which is a real SQL Server, so you can play before uploading. Can GAE do that? Looking at Azure, learning VS with F# (coming from Linux and OCaml), I wish I'd have switched to MS stack long ago for it. It's super easy to create SQL storage and inspect it in VS -- comes very handy. Open Source does not have a matching toolset and it's time folks give MS a fair consideration -- they've done an awesome job here. I'm surely sticking with my Mac OSX base (dual-booting into Vista), and my hunch is, with the ability of Azure to be developed locally, I'll be getting a separate Vista box for Azure development. .NET is truly overwhelming when you come from Unix pipe world -- PowerShell, SQL and LINQ, C# and F# (which is my key reason) -- but it turns out it all adds up and is worth learning in addition to, not instead of, Linux; and in all cases, Azure will widen your horizons.
As much as I love GAE, one major reason why I'm going with EC2 over GAE for my current project is that I need to be able to have the front end of my application serviced from data centers located in different parts of the world. GAE runs in one data center at a time. For example, I need users in Asia to hit servers in Asia for the fastest response times possible for my application. Add on the ability to manage dns, load balancers, database of choice, flume pushing into S3 for hadoop processing of data, etc... and EC2 becomes a really compelling solution.
Some things to consider:
Getting up to speed: how quickly can you get productive with the chosen environment what kind of docs exist and are they clear and well supported samples apparent and useful
Cost: cost is a factor, but if you are making a commercial App that will actually have customers, these are all viable choices. If you assume that Azure, with one proc on a "small" instance runs for about $90 a month for 24x7 use... how many users can you service in that time? Add a second instance for redundancy... still not that costly if your traffic warrants it. If it doesn't, why are you in the cloud instead of at a cheap hosted provider? Larger cost factors come in your time to implement this. AWS is a roll your own solution. That's a lot to handle to get a solution that will be stable and well managed. Azure and GAE have that out of the box. In my mind AWS is the most expensive due to the work you have to put into it. Do you really need control over it at that fine a level of granularity? If so, blow the wad and buy your own boxes maybe...well AWS will still be cheaper than the hardware costs.
Ability to do what you want: AWS all the way. Azure is second, GAE is third. No biggie if what you want is Java and Python. Biggie if you want to do relational DB or extensive multi-threaded data processing in C++ (not sure if any of these do this now?).
What about portability? Can you take it back to your own farm later or move it to another cloud farm? They are all portable to some extent.
Lots to think about... still learning about this myself.
If you need to start instances manually to meet the demand then it's not a cloud.
Azure and EC2 are just virtual servers with some services on the side.
EC2 and Azure do give you options to manage starting new instances automatically under load, but you still have to manage this. And you pay for instances that are up and idle.
GAE handles this automatically out-of-the-box and it only charges you for the time when your code is running during requests.
Here's some other considerations.
GAE - Sits higher on the platform as a service stack then AWS and Azure, all traffic routed through their ghs.google.com DNS, dynamically loading serving your page through one of their machines, which allows them to keep prices low. scaling is fine grained with this approach, Cons is no static ip, prone to being filtered or blocked. From the no static ip limitation, you'll not be able to setup any site specific https cert.
AWS and Azure gives you pretty much a static IP and a dedicated VM, allowing for such basic requirements like a https cert. you'll also get relational storage support. Cost is also higher to reflect this dedicated VM fact, and you'll scale per VM so in 40 dollar/month chunks. advantage is that since you get a VM to yourself, you're not constrained to the 30 second cpu processing limitation on GAE, and can run larger tasks.
So if your considering customer bases in filtered countries, or want a static IP to do your own DNS setup, or have requirements that need relational db or more than 30 second tasks. AWS, Azure would be much more friendly to work with.
Look at the solutions each cloud offering provides and go the hybrid model. Some problems require a hammer and others a screwdriver. Get to know your tools and apply it to the right problem.
I don't have enough reputation to leave a comment for one of the answers above. The suitability of any of those cloud solutions depends on many factors including your needs and skill set.
I have a social networking project that requires a nosql database. AppEngine would be a good solution if it had better support for the various frameworks. Django with nonrel adapter works on Python GAE, but I prefer Rails for many reasons. Rails3 has been out for a few months and no one in the community or on the GAE team has written a recipe yet to support it. Unless you have the skill set--knowing ruby and rails internals, jruby, and GAE internals--to write your own recipe, you're at the mercy of other people just to get on the platform.
AWS is a lot more work but at least you can get on the platform with whatever tools and deal with many issues administratively rather than as an internals developer or supplicant to higher powers.
My complaint about Heroku and EngineYard, for Ruby developers, is the mystery about how databases scale. How do they scale?
In my case, I'm opting for a NoSQL solution and Mongo seems to be a good choice. MongoMachine seems to be the recommended solution for the likes of Heroku or EY but it is mad expensive. $2.50/GB storage? Storage is only $0.10 GB/mo on GAE or EBS.
I've started experimenting with Google App Engine fairly recently, and for a web social network I believe it would serve all your needs. It's easy to get the hang of it and can be used either with Python or Java. It is true it doesn't give you access over files, but for your application, GQL (the SQL-like interface to the database they provide) will probably be more than enough (and it's quite robust).
One thing you might want to consider, is that an application on GAE can use an interface that will let users with Google Accounts, or accounts on a domain using Google Apps log in (a shortcut). You choose either of these. So if you already use a Google Apps Website, Google App Engine would be a great choice for you, as your users wouldn't have to register new accounts.
EDIT: As Arachnid pointed out, it's not that you can't code your own login system. Sorry, if I got you worried there.
As for the other two alternatives, I have only read about them and not tested them. But I believe GAE provides an easier framework, from my research, and as you mentioned great prices.
In any case, you can try GAE out using the free quota on space and bandwidth and see if it fits your needs.
Best of luck.
Azure has Windows/SQL as a server "Platform as a Service", and you are definately NOT stuck, just go back to Windows/SQL in your own datacenter (No linux, but yes they support Java, Python, PHP, Ruby, Tomcat, Apache, etc.). Like Amazon, they will be providing the fully accessible Virtual Machine option as well, so you can install/run whatever you want.
Amazon has is Virtual Machine only, so you still have to intall, patch, license, secure, etc... Kind of defeats the benefits of moving to the cloud in my opinion. You've just moved something from your datacenter to another.
Google has no relational database and you'd be STUCK. They really only cater to Python developers and some limited support for Java. They really are not a player in the cloud space from my opinion.
One thing not mentioned here, is what anyone considers of the "Windows Azure AppFabric Service Bus & ACS" besides the horrendous name...?
It appears to be a really powerful stack of integration features that would make Azure appealing from the perspective of any business with an investment in on premises infrastructure.
After experimenting Amazon EC2 for a while and hitting some delays I've started researching Google Apps while experimenting due to cost. I'd prefer Erlang as the development language but can deal with Python, so that wasn't a deciding factor. When I saw no static IP, that was. Also the whole part about it being higher up on the stack makes me a bit nervous with respect to performance.
I wish AWS was cheaper, but until Google provides static IPs and preferably additional languages such as Scala, JRuby, and Erlang, the choice is clear for me: AWS. The first two languages should be simple too, they are both JVM based. May even have been done already through work-arounds, as I seem to remember reading something about that.
Guys I think apart from just thinking about what platform it supports the comparison should be on Scalability, Ease of access, Versatile (in terms of implementation) , can accommodate different hosting platforms, equally economically viable for the business case, has multiple solutions for enterprise applications (viz. storage, delivery, bandwidth, licensing policy etc.), track record credibility of quality of service, Security audited, Transparency in billing as well as costings etc. If you look at all the above metrics I feel AWS scores much above. I am managing 10 production accounts on AWS since 2 yrs & at the same time the company / business unit were able to satisfy customer's huge scalability demands.... No doubt on AWS one needs to maintain the infrastructure, Updates (if any / if required), security etc. But you have all the tools available in the market / net freely. Existing IT resources can maintain all the infrastructure on AWS too.
Azure sure has an integrated IDE with VS 2010, but actual costing of any cloud will start after successfully deploying the application (platform for deployment). Still a long way to mature to address real time deployment / scalable production scenarios....... As everyone knows MS plays many hidden agenda's on costings.. very difficult to make out the costs incurred or going to incur (while sending out estimates).
GAE is very specific for Python / Java apps. Huge effort (both in terms of resource + cost) in getting the application rewritten (existing), tested , deployed etc.