What's a good way to store/load a word list with thousands of entries?

Question

I'm working on a project that requires a simple dictionary for checking spelling; it will be large, but will lack definitions. The solution we're currently working on has the base word list in a text file, and customizations (extra words, excluded words) in the DB.

The entire list needs to be loaded (best cached I think) for paginated display and customization using an existing JS widget, so what's the best way to store and load the base word list?

My initial thought was to have a text file with one word per line and write my own loader and cache object, but my boss suggested using a ResourceBundle to leverage the built-in caching. I'm thinking that could be done by using the words as keys with empty values and using the keyset of the bundle.

What would be the best solution to this?

Answer 1

In general, the data structure your are looking for is called TRIE.

If you are looking to implement this in javascript, be sure to read John Resig's blog on that subject. He evaluated several techniques for a similar problem, highly recommended.

Also, take a look at this stackoverflow.com question to find java implementations.

Answer 2

Is this for a language that does not already have dictionary files out there? I'm not 100% certain, but I was pretty sure that the Aspell open sourced dictionaries were available to developers...