How should a matchmaking app store matches for each user?

Question

What is the most efficient way to store each users' matches in a matchmaking app/website? Given the complexity of these kind of algorithms, is it reasonable to calculate the matches all the time, on-the-fly, when a user logs-in or presses the search button and later ignore the results after they have left?

If that's not okay for a reasonably large app, then storing all the matches for each user can be a challenge also, if there are more than a few million registered users.

Is narrowing the potential matches, like max. 100 matches per user or search only between users who live in this city, is the way to go or are there better ways to get full match results and store it for every user?

Also how would one design a database structure for storing the results? A NoSQL document that stores anyone's matches in their own document or just a relational table that stores two users' match percentage in a single record and repeats it for all the matches?

Update

How matchmaking algorithm works: If we say A, B and C are all users, it would first calculates how much B's answers and preferences satisfies A, then it would calculate how much A's answers satisfies B, then gets the geometric mean of these two numbers, which is then marked as the actual match percentage between A and B. Then it would repeat this for A and C and then B and C.

Answer 1

I wouldn't store matches.

Instead, I'd determine "characteristics that matter" and have a bitfield for each one. For example, maybe you decide "male or not" matters, so you have a bitfield for that, and if the first entry is for a male you set bit0, and if the second entry is for a male you set bit1, and so on. Then you might decide "over 40" matters too, so you create another bitfield for that, and if the first entry is for someone over 40 you set bit0, and if the second entry is for someone over 40 you set bit1, etc.

Now if you want to search for "males over 40" you get the bitfield for "male" and the bitfield for "over 40" and you AND them; giving you a bitfield representing "males over 40".

If done properly; a cheap laptop should be able to handle millions of matches per second.

Answer 2

I actually built a "dating" site awhile back that has a similar matching algorithm. What I learned from building that site:

• You will want to cache your match results
• Reduce your problem set by making sure that you take into account sexual orientation. For instance it would be a waste of time for the user to be matched with a gender they aren't interested in.
• Prioritize match calculations that fit strictly within in the user's preferences (age range, religion, distance from user, ..etc.). Also make sure to check preferences both ways (UserA -> UserB and UserB <- UserA). UserA might not care about religion but UserB could care a lot about that.

Essentially you work to reduce your problem set then prioritize your work. This way you don't spend a ton of time calculating a million matches for UserA while 10 other users haven't had any matches calculated yet.

We did a kind of round robin where we would calculate a certain number of matches for each user before moving on to the next. This was really only needed during the first month when the site was bombarded with people signing up.

Hopefully this helps you out.

Answer 3

Depending on what kind of matches we're talking about, you might want to be able to log which matches have been made and by which criteria. In this case an intermediate record for these matches might come in handy.

Especially if the properties on which the matching algorithm acts can change, it might be impossible to get a reproducible result.

I would go with storing the matches. You can then also record other information, such as whether the matched users agree with the match, which will help you to improve the matching.

Answer 4

You need to carefully elaborate how matches are created before deciding on how to store them. When you will know how exactly matches are created questions about storage would be easier and more obvious to handle. Expect combination of various approaches to be used:

1. On the go. Always changing set of marketing rules (A/B testing, promotion campaigns) defining set of [non-] matches depending on user attributes, moment of time, user device, previous orders, etc.
2. Off-line. Scoring algorithms running once an hour/day/month and producing matches per user/marketing group.

Generally it is preferable to calculate on the go to enable maximum flexibility. And only when some performance issue occurs move a part of the calculation to off-line algorithm.

Keep in mind that apart from having just list of matches per user you often need some sort of priorities between them.