I've started working on my web application and I'm stuck at the design stage. I cannot decide which web technologies would be the most suited to this project. Let me explain what I want to achieve:

1) I need to web scrape data from a website which provides live scores for sport games (e.g. flashscore)

2) I would like to display that data (and perform different operations on that dataset) and update it simultaneously with the original website (when something changes on flashscore, it should be updated on my website)

I am afraid that it is going to be super inefficient, correct me if I'm wrong. Was thinking about using Python (BeautifulSoup and Urllib2) to web scrape the data and then to store it somewhere in a database. If you had to build similar app, what kind of database would you use? NoSQL (perhaps MongoDB) or SQL?

Taking into account that the data has to be updated perpetually, which frontend framework would be the best choice to go with? React? Angular? How about Meteor.js - it could work on both front and back-end?

  • 2
    Tool recommendations are off topic here. – joshp Sep 1 '17 at 6:57

You're in what we call analysis paralysis. You see an overwhelming number of choices and considerations. Heck you're even talking about efficiency.

Stop it. Pick one very small useful thing and make that work in the simplest possible way. It might just be a script you have to run manually. Just so long as it does something useful.

Now that you have that what real problems do you see? Is it really to slow? Is the data really unsuited for a relational database? Now pick another very small useful thing to do.

The best way to choose a tool is to try it and learn about it. Asking us to pick for you isn't going to help you any more than flipping a coin would. The coin is actually more likely to give an answer since tool recommendations are off topic here. Identify capabilities you need. Not tools.

  • 3
    In order to get yourself out of the analysis paralysis, start first with technologies which you feel comfortable. Don't try to go from the paper to the final product in a single and straightforward iteration. Get something working first and if it doesn't meet your expectations, change it. – Laiv Sep 1 '17 at 6:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.