4

For a small hobby project I thought about giving users the ability to edit their submissions, but, since their submissions will be voted on by other users, restrict editing to correct (minor) spelling mistakes only, preferably natural-language-agnostic.

During a tentative investigation into the subject I thought I might utilize edit distance and related subjects such as (Damerau–)Levenshtein distance, for this.

So far I pondered about putting a certain edit distance threshold on either the entire original submission or on their individual words (for languages where this makes sense). Now, I certainly don't want to create a complex natural language processing algorithm, but I have a feeling this is a way too naive approach, particularly in light of languages such as Japanese and Chinese, etc, where replacing a single symbol could change the entire meaning of a sentence.

So, before delving deeper into these edit distance subjects, I thought I'd ask the community whether it is even feasible to merely use edit distance, in a fairly simple fashion like described above as a reasonably reliable measure for something like this. Is it? Or is this too naive and am I almost certainly bound to creating a complex natural language processing algorithm and/or use entire dictionaries for any reasonably reliable approach (in which case I'll probably ditch the entire idea altogether ;-))?

Come to think of it: since I can't remember ever coming across this kind of functionality on major sites with voting systems, I can probably guess the answer myself already, but I thought I'd ask you anyway, just to be sure.

Improve this question
2
  • @gbjbaanb Thank you for creating and adding the tags.
    – Codifier
    Commented Sep 15, 2015 at 12:16
  • 1
    Consider: "I was a bit hoarse" to "I was a bit horse". Edit distance of 1. Both spelled correctly. Identical soundex. Completely different meaning.
    – user40980
    Commented Oct 22, 2015 at 17:01

1 Answer 1

Reset to default
1

If you only want to edit spelling changes, then you'll be doing this on a word-by-word basis, so checking levenstein against the entire text is probably not going to work (especially as IIRC a small change up front can have a big knock-on effect in a large document).

Its possibly better to encode every word with a soundex (or better, a double-metaphone) code and allow changes there - but only if the code doesn't change (ie the word may change but the sound of it doesn't, so you can be reasonably sure its not been changed from its original meaning, just the spelling).

The ASpell (spell checker) project uses metaphone to offer replacement suggestions, so its very better suited for spelling alterations only.

Improve this answer
1
  • Word-by-word with double-metaphones looks like the best approach indeed, but if someone forgot a word boundary delimiter I may already be in trouble with this approach, since the original text doesn't accurately represent word boundaries anymore then. Would you agree? I'd also have to investigate how well implementations of double-metaphone in php (the language I'm using) handle non-Latin/Cyrillic text. At first glance they all appear to require transliteration (or is that a given anyway, with double-metaphones?). I'll look into ASpell as well. In any case, thank you for the pointers.
    – Codifier
    Commented Sep 15, 2015 at 17:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.