We have a big set of survey questions, which we distribute to users online. We collect submission from users and based on their submission, we decide the answers for those questions. Now, each user gets a set of questions, say N. For each submission given by user, he/she gets some points(cost c). Now, there is a upper limit on questions such that if a question receives more that k submissions, it will be disabled from next time assignment.

Now the problem is, sometimes, users take questions and don't submit it for a long time. So, we allocate same question to many users (say m, m > k), and after sometime everyone submits the solution, so instead of required k submissions, we get m submission. Which in the end increases the total cost attached to survey. Is there any algorithm to solve things like this.

We thought of doing a 'soft reservation' system, where we will keep track of each question's assignment count, so that if we have given k users the same question, we won't give it to other users. But, it has side-effects which includes timer check for each question whether its submission has come or not. Which is kinda dirty.

Other which looks closely related is assignment-problem, hungarian-algorithm, but its solution isn't what we are looking for. Please help.


There is another clause of random selection of questions. We want to give differently selected set of questions to each user. So now, if I have 50 users and 1000 questions and a question's max submission (k) is 5, how can I achieve result in minimum cost spent.

2 Answers 2


Choose the questions one by one as the user progresses through the survey rather than selecting them in advance.

***** edit

Assume that the survey is completed online via a website.

User logs in and opens survey

User opens first question.

system selects a question which has not been answered the required number of times yet

User fills in answer

User moves to next question

system selects a question which has not been answered the required number of times yet....


Each question is only selected as the user requests it. Thus you can apply a shorter timeout to 'user has opened a single question and left the browser open'. Vs 'user has registed to do a survey and hasn't got back to us yet'

  • 1
    Sorry, but I didn't understand. I have updated the post details. Commented Aug 28, 2016 at 8:15
  • Uodated with further explaination
    – Ewan
    Commented Aug 28, 2016 at 9:41

Have an expiration like 10 days

Have a response of virtual count (v)
m received
d days out for any question out
e expiration

v = m + sum(e-d) / e

assign from a random group with lowest v

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.