Vector input for Artificial Neural Network?

Question

My problem is fairly complex, so here is a similar, simplified example.

Let's say you wanted to predict the Miles Per Gallon (fuel consumption) of a car, based on the following information: you have the weight of the vehicle in pounds, the number of cylinders in the engine, and a number which corresponds to a type of vehicle (0 = cube car, 1 = standard car, 2 = van, et. cetera), and the amount of rain within the past hour (in gallons).

Say you take all of this information, and compile it into a 4 Dimensional vector, [Weight, Cylinders, Vehicle Class, Rain]. Because rain from 4 hours ago is still important, lets say we took multiple of these vectors from the past 5 hours, and feed them into a network, so we have something like this:

[Weight 5 hours ago, Cylinders 5 hours ago, Vehicle Class 5 hours ago, Rain 5 hours ago]
[Weight 4 hours ago, Cylinders 4 hours ago, Vehicle Class 4 hours ago, Rain 4 hours ago]
...
[Weight in past hour, Cylinders in past hour, Vehicle Class in past hour, Rain in past hour]

Being feed in, resulting in in "current rain amount"

Question: How would one format this problem for use in Artificial Neural Network? Can networks take vectors as inputs? Would it be better to allot each node to a single value (so each vector takes 4 nodes, one for each dimension, resulting in 20 nodes for the set)?

Because of the time changing, I've considered using a HMM/MM however, I am not dealing in exclusively categorical data. This isn't for a class or anything, I'm just learning out of pure interest... Thanks!

Answer 1

A sliding window approach (NN is trained to use the last k values of a series) is the way to go for a feed forward neural network.

Redundant input values should be removed because they can negatively affect the neural network learning ability (another benefit to removing redundant variables is faster training times):

[Weight Cylinders VehicleClass Rain5 Rain4 Rain3 Rain2 Rain1]

is a first change (consider that the "window size" does have an important effect on the quality of a neural network based forecaster so you should check with different number of Rain[n] inputs).

Also instead of using one number for the type of vehicle (0 = cube car, 1 = standard car, 2 = van...) you should use a 1-hot encoding:

                  <-- Vehicle Class -->
[Weight Cylinders (0.0 ... 1.0 ... 0.0) Rain5 Rain4 Rain3 Rain2 Rain1]    

This will work much better, although your feature vector will get much bigger (and, depending on how much data you have, you might suffer from overfitting).

Recurrent Neural Networks (check the Long-Short Term Memory variant) can take into account many instances in the previous time stamps (rain value) and you don't have to define a sliding window.