I am a experimental physicist. In our research, we have our experimental data in a 400*400*400 matrix(x, y, z axis of a 3d space), each entry is associated with a value("brightness"). We expect the brightest entries will form a closed path in the 3d space. But some portion of the path is always too dark to identify, also there are some noise in the space(random bright lines).

Our current algorithm generates random seed points into the space, they are attracted by the brightness of the data. After certain amount of time, they will be trapped in the path. If we record the seed points position we can get the path. Our current algorithm does a fairly good job while I still need to manually add those relatively dark points to the path.

I am not familiar with machine learning, but I am wondering if it can be used on this case? For example, is it possible that I tell the program which data points are on the path, the program will know how to choose the data points for the path in the future. Or if there is some other method we can use? Thanks!

  • 1
    No, not really unless you would provide more information than just the location of a point. What is hindering you to simply fit a curve to the data? Does your data set have too much noise? Could you provide some visualization? You might be able to use clustering techniques to filter the data. – amon Jan 22 '14 at 12:43
  • 2
    This sounds more like an image/signal processing problem to me – jk. Jan 22 '14 at 12:43
  • I think the standard methods to predict future values based on current values can be taken as a basis and modified for your need. Regarding machine learning, why do want to do this? IMO your problem is more maths than anything else. – superM Jan 22 '14 at 12:48
  • 1
    This sounds like a target tracking problem to me. If your sensor gives you time information on your voxels (volume pixels) (it lit up at time T, died at time T + deltaT), then what you have is a straightforward radar data association/multiple target tracking problem. – John R. Strohm Jan 22 '14 at 13:20

Isn't this just a regression on (x, y, z)? But it sounds like a regression rather than a classification because you're trying to determine a path -- in other words, (continuous) values of x, y, z close to the measured path, based on initial measurements of the brightness.

  • Alternately, you could train a neural network to identify the path, and produce a "fitness" estimate based on (x, y, z) inputs. You might have to have a different neural network for each data set, though, if the shape of the path is different each time. (I think this would just be a more generalized version of doing a mathematical regression, though.)

  • If you could express the path as a parameterized formula that scores an (x, y, z) tuple based on closeness to the path, you could do a Genetic Algorithm to search the space and guess and select good values for the parameters. In other words, the Genetic Algorithm could search for parameters that optimize the formula for the path.

From your description, it sounds like you're almost doing a Genetic Algorithm already -- selecting a random point and then doing incremental optimization. That's the basic idea, the difference would be in the optimization method.

Each of these techniques optimizes the answer to minimize the effect of noise.


Saying "regression" is still pretty broad, because there are lots of variations on regression (linear regression, logistic regression, etc.). Here's a roadmap from Python's scikit-learn site that might give you roadmap of techniques that might work for you:


(Note that something happened a while back, and the whole field of statistical analysis kind of crashed into the Arificial Intelligence realm and created the Machine Learning buzzword. So a lot of things that just would have been "statistics" or "analysis" are now "machine learning" because they meet the criteria of "improving the quality of the answer given more time and data")


Yes, sounds like what you want to do is basic machine learning. Sorry, I can't comment yet.

Essentially, you need to be able to (in the future with real data) do a good job of discriminating between dark points not actually on a path and dark points on a path. So, you need to train a classifier that discriminates between path and not path.

Amon, each datum is (x, y, z, brightness). Furthermore, you have points around each other. So a classifier or clustering algorithm [by definition, machine learning] would look at not only the brightness of a given point, but what the other points are doing around it.

Support vector machines may be a good classifier to start with. You may need to search around or ask in your CS dept for introductory resource material. Your problem difficulty is such that I feel you are more than capable of having a classifier up and running within a week or two of working on this. I don't know any online tutorials offhand, I typically refer to my intro to machine learning notes whenever I need a "simple" refresher.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.