I'm thinking about a way of finding similar parts in Strings. I have a set of strings of variing length i.e:

• The quick brown fox jumps
• fox force five
• the bunny is much quicker than the fox
• is

First i thought of just tokenizing the strings and count the tokens but in case of "quick" here i also need to match "quicker".

So the output should be something like this (a mapping of tokens to the count, if the count is 1 it is ommited):

{
  "the": 3,
  "fox": 3,
  "quick": 2,
  "is": 2
}

The use case is as follows:

The user groups together Strings to categories, the goal is to make suggestions for a new string to which category it might belong. So the idea was to search all current strings in a category for keywords like this.

I'm thinking about a way of finding similar parts in Strings. I have a set of strings of varying length i.e:

• The quick brown fox jumps
• fox force five
• the bunny is much quicker than the fox
• is

First, i thought of just tokenizing the strings and count the tokens but in the case of "quick" here I also need to match "quicker".

So the output should be something like this (a mapping of tokens to the count, if the count is 1 it is omitted):

{
  "the": 3,
  "fox": 3,
  "quick": 2,
  "is": 2
}

The use case is as follows:

The user groups together Strings to categories, the goal is to make suggestions for a new string to which category it might belong. So the idea was to search all current strings in a category for keywords like this.