9

This question could go here or on S.O. perhaps...

Suppose that your training dataset contains both categorical and continuous data such as this setup: 

 Animal, breed,  sex, age, weight, blood_pressure, annual_cost
 cat,    calico, M,   10,  15    ,   100         , 100 
 cat,    tabby,  F,   5,   10    ,   80          , 200
 dog,    beagle, M,   3,   30    ,   90          , 200
 dog,    lab,    F,   8,   75    ,   80          , 100

And the dependent variable to be predicted is the annual vet cost. I'm a bit confused as to the specific techniques available to deal with such a dataset. What are the methods commonly used to deal with datasets that are a mixture of both continuous and categorical data?

Improve this question

2 Answers 2

Reset to default
8

For each class (breed, gender...) of categorical attributes, you can add a number of components to your feature vector equal to the number of possible values in that class. Then, if a data point has the ith value, you set the ith one of those components to 1, and the rest for that attribute to 0.

In your example, for gender, you would add two new components to your feature vector. If the animal is male, you would set the first one to 1 and the second to 0, and vice versa if the animal is female. For animal, if your possibilities were cat, dog, and fish, then you would do the same with three components.

These would coexist side by side with the continuous attributes. You might want to adjust the magnitude of the "indicator value" (the value that you use when an attribute is "on") so that it's comparable to the magnitudes of the continuous values you're using, though.

If you chose this way of going about your problem, the next step would be to pick and algorithm such as a support vector machine and feed it your feature vector. Of course, some approaches like Decision Trees would not need the step I mentioned above to begin with.

Improve this answer
2
  • 1
    Cool, great tips! What about the possible situation in the test data where the breed never came up in the training data? For instance, if there was a poodle in the test data but not the training data? Would the best way to go just have zeros for all components of the breed vector?
    – reptilicus
    Jan 15, 2013 at 22:11
  • 2
    In fact, the opposite approach (known as data discretization) is usually the best choice for hierarchical methods (such as decision tree algorithms). Data discretization is also a data pre-processing technique (as mentioned in the other answer).
    – rvcoutinho
    Jan 16, 2013 at 0:06
2

You should take a look at data pre-processing. It is prior to any machine learning technique. Here is a good introduction (found at Google).

Regarding the techniques, there are lots of different approaches. You can probably use most of them after pre-processing your data. You should try them and pick the one that best fits your needs.

Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.