Timeline for Designing a big data web app
Current License: CC BY-SA 4.0
15 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Feb 20, 2019 at 2:15 | vote | accept | Minh Thai | ||
| Feb 20, 2019 at 2:15 | vote | accept | Minh Thai | ||
| Feb 20, 2019 at 2:15 | |||||
| Feb 20, 2019 at 2:15 | vote | accept | Minh Thai | ||
| Feb 20, 2019 at 2:15 | |||||
| Feb 20, 2019 at 2:14 | comment | added | Minh Thai | @RibaldEddie didn't I just basically answer my question... I think the key point is to go from the most basic approach like the terminal and keep adding improvements. Thanks for your help. If only I have enough rep to upvote your comments. | |
| Feb 20, 2019 at 1:26 | comment | added | RibaldEddie | great. Now you know what you need to build: an interface for defining queries that can be submitted to Spark. | |
| Feb 19, 2019 at 14:39 | answer | added | Jé Queue | timeline score: 1 | |
| Feb 19, 2019 at 5:35 | comment | added | Minh Thai | I would setup something that allows SQL queries and can connect with HDFS, for example Spark. I will add more servers until it cannot go faster. Then to save time on the same query later, I cache the result. | |
| Feb 19, 2019 at 4:17 | comment | added | RibaldEddie | Thanks. Now I want to ask you a question— forget the web site part. How would you make this work if the queries were submitted at the command line? | |
| Feb 19, 2019 at 4:15 | comment | added | Minh Thai | You're are right, that is an important part. I've updated the question. | |
| Feb 19, 2019 at 4:15 | history | edited | Minh Thai | CC BY-SA 4.0 |
clarify query result
|
| Feb 19, 2019 at 2:26 | comment | added | RibaldEddie | You haven’t told us much about the results of the queries, which is what matters in the context of web sites. For example, if you run a search over the 100 million users and return a single value in 100ms, then all of the engineering effort is going to be on the back end and doesn’t really have anything to do with web sites. However if you need to display a table with a million rows, that’s a different problem. So you need to be more specific. | |
| Feb 19, 2019 at 1:50 | answer | added | Nelson | timeline score: 1 | |
| Feb 18, 2019 at 23:59 | history | edited | Minh Thai | CC BY-SA 4.0 |
deleted 110 characters in body
|
| Feb 18, 2019 at 4:25 | review | Close votes | |||
| Feb 27, 2019 at 3:05 | |||||
| Feb 18, 2019 at 3:45 | history | asked | Minh Thai | CC BY-SA 4.0 |