1

I'm working on a quiz system that will allow users to enter text as an answer. The question could be something simple to start with, looking for a short phrase, or a select few words as the "correct" answer.

Having a single correct phrase however could be limiting. It may be that the user misses out a few words, writes in short hand, supplies a partial answer, or something along those lines.

I'm allowed to make this as simple, or as complex as I like. If I decide I want to keep it easy, then forcing users to be rather more explicit is a viable option. However if I have time, I would also like to think about allowing a little vagueness or spelling errors, or even doing some analysis of the answer to be smarter about how I handle it.

Lets start somewhere in the middle. I would like to analyse a (short) block of text entered by a user for spelling mistakes and key words that would make the answer a match. What theories should I be looking at to achieve this?

If its any help, I will be using PHP, although I'm certain the principles will be the same.

Many Thanks,

Simon.

3
  • There's a close vote (not from me) with "too broad" as the reason. What you describe could be as simple as a list of synonyms for keywords or as complex as a complete natural language processor. Perhaps you can edit your question and narrow the scope a bit? – Dan Pichelman Nov 28 '16 at 14:16
  • Thanks, Its difficult for me when I don't really know enough about what I'm trying to do. I know I'd like to allow maybe some kind of fuzzy logic to interpret the answers. But I'm not sure if "fuzzy logic" is the correct analogy. Would it be sufficient to ask something like "I want to interpret a users input, ignoring trivial words that can missed out, but relying on the important keywords to determine if their answer is correct, whilst allowing for a degree of typos"? – simonw16 Nov 28 '16 at 14:32
  • Maybe you should look at Weizenbaum's Eliza: en.wikipedia.org/wiki/ELIZA – user188153 Nov 28 '16 at 15:30
2

It may be that the user misses out a few words, writes in short hand, supplies a partial answer, or something along those lines.

This leads to a natural language processor. Since it looks like you never worked with one, it may be useful to highlight that NLPs are a broad and rather complex subject, which means that you shouldn't expect to implement one in a small project in a matter of days.

You can simplify your original problem a bit, for instance by:

  • Passing user's input through a spell checker,
  • Computing how many words match the list of expected words, and how many don't,
  • Determining whether the threshold is reached.

Of course, this would lead to both false positives and false negatives.

Example:

For a question like yours in a form of a quiz, you could expect to find the keywords “natural”, “language”, “processor”, “processing”, “nlp”.

  • If the user misspells the terms by writing “i'd use a natrual langage procesor for taht task”, the spell checker will fix it to “I'd use a natural language processor for that task”, which would give the match/mismatch index of 3/7, which, in turn, would indicate that the user responded correctly.

  • However, the user entering an answer such as “To respond to this issue, a few alternatives could be used, including the techniques from natural language understanding field.” would have an index of 2/17 and will probably fail, despite the fact that the answer is correct. This makes this approach error prone.

Another issue with such approach is that if you continue working on the project, you'll be inclined to add more and more rules over time (for instance a dictionary of irrelevant words, such as “I”, “the”, “and”, etc.) This could lead you to virtually reinvent the wheel, and maybe reinvent what NLP solutions give you from scratch.

4
  • Thanks, this is useful information. I certainly have never worked with one, and If the scope is that large than I shan't be implementing something so extreme. What you list in the simplified solution is a good idea, but as you say, false positives and negatives will lead to answers being incorrect, but threshold still being met. – simonw16 Nov 28 '16 at 14:34
  • Could you use Levenshtein for example to allow for some simple typos, and pair that with matching a list of keywords to get something viable? – simonw16 Nov 28 '16 at 14:36
  • @simonw16: Levenshtein distance may help, although I've never used this technique and don't have enough knowledge of its benefits, drawbacks and constraints. – Arseni Mourzenko Nov 28 '16 at 14:42
  • This is enough information to get by with for now. I won't be introducing an NLP for sure, but I will take in to account your example and see if there's a way to use it, or if the margin for error is too large and not worth the resources. Many Thanks – simonw16 Nov 29 '16 at 17:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.