What are best practices when it comes to designing a social network app for Android (possible iOS port later depending on uptake)?

It's my first time architecting something this scale from scratch, and I'm probably already making mistakes in the design phase.

What I already have in mind is to:

  1. Have a Postgresql backend to store customer data. I'm going to set up one 'app' user to handle all the incoming RESTful requests, which leads to my next point..

  2. Connect the database to the app through a RESTful interface (I don't think the platform is overly important but I'm looking at either Go or Puma/Sinatra or Flask as I want to minimise overhead and keep everything as efficient as possible).

  3. Authenticate users using whatever authentication plugin is suitable within the RESTful interface and then..

  4. Tunnel all traffic through a SSL socket which I'll open on the app which will connect to the interface (this is one of the areas where I don't have a great amount of knowledge/experience)

  5. Load data pertaining to other users and store it locally in an encrypted sqlite db on the Android device which I will create as part of the app start-up process. Images will be the only media type I'll allow. I'll wipe the sqlite db each time the user logs in to the app or once it reaches a certain size i.e. over 3 mb.

  6. Make the app subscription based so along with checking the user credentials each time they log on I'll check their subscription status and enable different behaviour accordingly.

  7. Deal with heavy load connection issues by putting Pgbouncer on the db.

  8. Store the user images on my server as normal image files and when the relevant users are selected I'll send them to the app over the socket.

  9. Run the entire server as a VM, possibly from CoreOS/Docker machines, which sounds like a good fit from what I've read.

Is there anything wrong with any part of my intended design? Is there anything necessary I've missed out?

closed as too broad by Robert Harvey, GlenH7, user40980, Ampt, Wayne Molina Jul 31 '14 at 19:15

Please edit the question to limit it to a specific problem with enough detail to identify an adequate answer. Avoid asking multiple distinct questions at once. See the How to Ask page for help clarifying this question. If this question can be reworded to fit the rules in the help center, please edit the question.

  • I hope you don't intend on having many users... – Telastyn Jul 29 '14 at 21:08
  • 'Many' is subjective. I'm hoping for at least a few thousand in the first few months. Why do you say that? – warsong Jul 29 '14 at 21:16
  • 5
    There seems to be no plan to address scalability (short of a basic connection pooler), and a few things that will actively hinder it. Since that's the most difficult technical challenge this sort of app faces, it's a little concerning. – Telastyn Jul 29 '14 at 21:24
  • What would hinder scalability? Also this is exactly why I'm asking. From what I've been reading, the setup (meaning postgresql and Go, even Puma) I'm planning to go with is allegedly purpose-built to scale. But as I stated, I haven't taken on something this big on my own yet and I would ideally like to not get too much wrong ;) – warsong Jul 29 '14 at 21:34
  • recommended reading: Why is asking a question on “best practice” a bad thing? – gnat Jul 31 '14 at 19:19

I am going to be overly critical. I don't mean this to be mean, but to provide the most accurate feedback I can:

What are best practices when it comes to designing a social network app for Android (possible iOS port later depending on uptake)?

Focusing on a specific platform for architecture is unwise. Mobile apps have their own foibles (is it connected to the internet or not, limited permissions/disk space/screen space/user times) but many of these foibles come in the design of your product (usability, target audience, etc) not in the design. The difficult technical challenges aren't on the client side.

That said, look at facebook. They've largely defined best practice.

Is there anything necessary I've missed out?

Yes. This is a small list based on my limited experience. I expect there to be about 5 times as many issues. Many of which I expect to me more severe/nuanced.

  • A single database. Even the beefiest database machine is only going to scale so far. That database tends to be the first chokepoint to scalability. It also will be the most severe, and the one that will last the longest in almost every arena. Disk space, processor, ram, disk latency, network latency.. all of these are issues, but the most pervasive is that this is a single touchpoint/source of truth for your system.
  • SSL. The rule of thumb I've heard is that SSL will add about 20% to your bandwidth. Worse, it impacts the latency and processing power of your web servers, meaning you need more to serve the same amount of traffic. Certain specialized hardware can help, but is costly. I question the benefit of having all social networking traffic encrypted.
  • Data size. You are absurdly underestimating the amount of data that is involved in synchronizing user data to devices. Assume you have (merely) 10 friends. These 10 friends have 8 pictures. Let's be generous and say the pictures are only 50k. That alone is 4 megs of data to push to every single one of your users.
  • CDN. Now go back to that last example and realize that it's pretty damned likely that 4 of those pictures are really the same picture - reposted 10 times by each of your friends. Sure, a relational database can reduce that duplication, but the common best practice is to use some sort of CDN to spread the wealth (read: load). Why send them via socket from your server when you can use built-in http libraries to get them from companies specialized to distribute and deliver content for you?
  • Subscriptions(!) Ignoring the challenges of getting people to pay for something new when they can get dozens of other awesome options for free... Subscriptions place a far greater burden on your design, because now you can't trust the client. You need to check to see who this client is, do they have rights, did the rights expire, did the payment go through... And they need to be dead simple to not become some role/tier based monstrosity.
  • Load Balancer. Multi-threaded web servers on a cluster are all fine and dandy, but something needs to take incoming traffic and divvy it out. CoreOS likely has something to do that, but I imagine specialized hardware will be necessary shortly thereafter.
  • Caching. There's no talk about caching in your points. Having caching (or more likely, some NoSQL implementation) will be the first vital step to scaling past the limitations of your single database server.

All in all, you're tossing out technologies as though they're going to solve your architecture problems for you. They will not. The tools are good, and the tools are necessary - but if applied in the wrong places or in the wrong ways, it's not going to work. And given the ambient force of large, highly concurrent applications things rarely "sorta" work. They either scale out nicely or they die horrible, horrible deaths under that pressure.

  • Thanks, and no I don't think you were overly critical or mean. I'm not saying this in an ungrateful way, but a lot of what you mention are problems I was hoping I wouldn't have to deal with until at least the 5000 user mark, at which point I could rethink and re-evaluate my strategy but I did mention I would like to get as much right as possible from the outset. Could I please get a little clarification if possible? A single database, so you're saying I should use clustering? SSL, so I only use it for authentication? You're right in saying I don't need all of it encrypted. – warsong Jul 30 '14 at 8:47
  • Data size, that's a good point, and I didn't think of it that way, but this is exactly why I put this out there. I suppose I could cache a lot of this on the client-side to avoid repeated hits to any server, whether that's a CDN or not. CDN, I did consider this but I'm trying to do it on as much of a shoestring budget as possible right now, it's likely I will need it at some point though. Subs, this is another area I don't know a great deal about. This is quite a niche thing so I don't think it'll necessarily be a challenge getting people in. Load balancer, reverse proxy? – warsong Jul 30 '14 at 8:53
  • Caching, I didn't mention it, but I did consider it and what would be the right way to go about doing it in my case. I don't really know NoSQL and I don't get how it would solve the issue but I'll look into it. A lot of this is just going to message-based, therefore I doubt it will be hugely heavy on the server. Lastly, are there any good references/resources out there that explain what you have already mentioned in more depth? Thanks again, I appreciate it. – warsong Jul 30 '14 at 8:58
  • Btw I do think there is value in this question, I didn't notice much either on SE or elsewhere that deals specifically with this. Hopefully it can become a little reference in itself for how to get started. – warsong Jul 30 '14 at 9:02
  • @warsong - while it is true that you'll only need to care once you hit a certain user threshold, social networking requires quite a high user threshold to be useful. A social network with 20 random people isn't very useful or profitable. And scalability is one of those things that tends to be hard to architect in later (since new users won't wait for your fixes. The site will just die). – Telastyn Jul 30 '14 at 11:38

Not the answer you're looking for? Browse other questions tagged or ask your own question.