I need help in identifying a better algorithm.

I have developed a script using pythons scipy package to analyse a rather large model that I wish to solve. The model contains over 12GB of data including over 500 parameters.

The problem is that when I run small simulations of about 0.5GB of data with 20 parameters it can take my computer a decent amount of time if I allow a reasonable number of iterations through the random forest classifier.

Currently my script is only using one core, so I guess that making the script multi-threaded would be the first step. But I do not believe this will be enough given how complex the model is. I am willing to explore the use of a cluster based HPC solution but I am not sure how to go about this.

Are there better algorithms I can use, or is there a cluster based algorithm that would be more appropriate?


It is a bit unclear in your question whether you ask about how to parallelize the Random Forests™ 1 algorithm, or you ask which other algorithm would perform better.

On the first issue, it seems that the Random Forests™ algorithm is embarrassingly parallel, and that there is a readily parallel implementation of the algorithm within scikit-learn, using the so-called Ensemble methods. (Do not miss the section on parallelization on that document).

On the second issue, the established opinion seems to be that Random Forests™ are

one of the most successful ensemble methods which exhibits performance on the level of boosting and support vector machines. The method is fast, robust to noise, does not overfit and offers possibilities for explanation and visualization of its output.

as quoted from Robnik-Šikonja, M. (2004). Improving random forests. In Machine Learning: ECML 2004 (pp. 359-370). Springer Berlin Heidelberg.

Which, to me, sounds like you should stick with RF™, then try Adaboost, then try SVM. You can find them on scikit, too: svm and adaboost.

1: Random Forests™ is a trademark of Leo Breiman and Adele Cutler.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.