I've been trying to design a backend architecture to a SPA that satisfies the following:
- user constantly answer yes/no questions (ex: tinder swipe cards)
- The application calculates each user's similarity to one another solely based upon their answers
- Similarity between users should update as they answer more questions
- The application periodically prompts two active users with sufficient similarity to enter private chat
- The application will have up to 10 000 questions
Which architectural component should calculate/store/handle user similarity given that there are many concurrently active users constantly/frequently updating their answer set ?
Possible Approaches I've considered:
- Create K-many tables (db). Periodically, the users are clustered (K-Means) into exactly one of the K tables. All users within a given table have sufficient similarity. The client requests two active users from a given table. sticking points: where/when is this calculation performed in the architecture and can only 1 user be re-clustered based upon their updates?
- Calculate on-the-go (client and/or server worker threads) + caching. A client background process requests a list of random users and calculates
thisuser's similarity to each. Caches matches. sticking points: If a user is answering at a high frequency, how often is this calculation performed? Furthermore does it scale?
- Bitfield/bit representation of answers in the
usermodel(server/db): Periodically the client requests a matched user. The server responds with a user by querying based upon the bitfield. sticking points: How does this scale to answering all 10 000 questions (ie: can a database store and query on such a large bit representation)?
Technology currently being used: Node/Express, PostgreSQL + ORM, Websockets.
Any insight with architectural patterns or database design or better approaches for this scenario would be greatly appreciated.