I have a website that allows admin users to input free text questions to a database that will end up being asked to non-admin users using the website. This system is replacing an older system in which the library of questions became almost useless due to the amount of duplicate questions so one of my main goals is to do my best to detect and force the reuse of existing questions.

What strategies do I have to avoid ending up with the same issue? My initial thought is to compare a user input to existing data using a search engine (CloudSearch, Elastic Search, etc) and blocking question input on a certain score threshold.

Are there other strategies? Is there any specific way I should think about storing/sanitizing my data to maximize efficiency? I'm thinking of things like removing low value words (the, of, is, etc) to boil down to the essence of the question.

While I am asking this question it seems very meta as perhaps what I am trying to create is the "Similar questions" widget here on Stack Exchange.

New contributor
user2429765 is a new contributor to this site. Take care in asking for clarification, commenting, and answering. Check out our Code of Conduct.

Your Answer

user2429765 is a new contributor. Be nice, and check out our Code of Conduct.

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.