1

I have a bunch of plain-text like this:

1 MILE, PACE, PURSE $1,100.
FILLIES & MARES N/W $541 L5 STARTS AE N/A $301 L5 & N/A $60 PS
IN 2015-16 DRAW INSIDE
                                                                                         Last
Horse                       HV PP    1/4     1/2     3/4     Stretch  Finish     Time    1/4  Driver           Odds   Trainer
7   Im A Debutant               7    7/9H    7/5T    5/2T    5/2H     1/3        2:03    31   C Macpherson     7.45   R Gass
3   M D Caseys Charm            3    2@/1H   3/1T    3/1H    3/1Q     2/3        2:03.3  32   Ma Campbell      3.20   S Ford
5   Lucksgottachange            5    1/1H    1/T     1/1Q    1/1      3/3        2:03.3  32.1 J Hughes         1.55*  J Hughes
2   Gascoigne Dickie            2    4/4T    5/3Q    4@@/1T  2/1      4/3H       2:03.3  31.4 K Sorrie        30.10   K Sorrie
8   Avid Yankee                 8    8/12    8/8     8/5     8/4Q     5/5        2:04    31.3 K Murphy         5.25   A Ramsay
1   Honor Roll                  1    3/3     2@/T    2@/1Q   4/2Q     6/6        2:04.1  32.3 B Andrew         9.90   B Andrew
4   Julep Hanover               4    5/6Q    6@/4H   7@@/3T  6/3T     7/6        2:04.1  32   W Myers         19.05   W Myers
6   Putnams Snap                6    6/7T    4@/3    6@/3H   7/4      8/10       2:05    33   M Mcguigan       2.75   G Dunn
Time: 29.2, 1:00.3, 1:31.2, 2:03 (Temperature: -2, Condition: GD, Variant: 1)

taken from http://www.standardbredcanada.ca/racing/results/data/r0130chrtnn.dat

But in my case it's human-written and may contains extra/missing spaces, dots, etc

And I need to parse it into a data structure. What would be different program languages approach to that? Good libraries?

I'm mostly a python programmer, but looking towards learning new languages.

Also, I'd really love to see how strong-typed languages deal with this.

|improve this question
  • 1
    I think the keyword you are looking for is "natural language processing" or NLP. Here is an example of NLP extracting dates and places: loadfive.com/os/knwl/demo You presumably want something similar in python and you want to make a filter for each of your columns. – Max Murphy Feb 8 '16 at 11:44
  • 6
    I don't see how are programming languages relevant here. Barring some relatively uncommon approaches (like parser combinators), parsing is going to be mostly the same in every language. – svick Feb 8 '16 at 14:46
  • 1
    Machine learning researchers tend to have a preference for languages such as R, Sala and Python, so if someone is looking for a good or the best implementation of a natural language processing algorithm they are more likely to find it in one of those than any other. There are exceptions, of course, but that is the general trend. So I can see how language can be relevant, although perhaps not quite in the way the OP expected. I doubt that being strongly typed has much of an effect on the algorithm. – Max Murphy Feb 9 '16 at 10:35
1

One way to approach it: Have a dictionary stored as a trie, and look things up in it, analogous to a spelling corrector. Missing spaces between words can be treated as just another misspelling.

|improve this answer
  • 1
    How would that help with parsing e.g. Im␣␣A␣Debutant␣␣␣␣␣␣␣␣␣␣␣␣␣7 (one space too many in the name, one space too little between columns)? – svick Feb 8 '16 at 14:49
  • @svick: The nice thing about a trie is it can cycle back on itself, or if you prefer, the recursive walk procedure can cycle back to the top of the trie as a way of handling more than one word in the string, and it can still handle all kinds of misspellings, such as spaces inserted or deleted. – Mike Dunlavey Feb 8 '16 at 15:03
1

While I rarely offer this advice on public forums, a regular expression should be enough. The key is that there seems to be very distinct pattern groups in the data. These groupings means you don't have to rely just on whitespace.

For example, the table is a number, some whitespace, a bunch of letters and numbers, some whitespace, a number, five columns made up of two pairs of characters with a /, followed by a timestamp. That's an easy thing for regular expressions.

|improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.