I've been going over some tutorials on machine learning using Python and libraries including SciKit Learn and Tensor Flow. (Basic tutorials like creating an algorithm to predict a price given input values).
I've found that these tools are extremely powerful, but they also appear to be quite slow, and require a lot of tweaking due to overfitting or underfitting the data.
This wouldn't be a problem if you were dealing with a batch of data (such as you receive a new set of data once a day that you want to calculate predictions on), but I see this being very difficult to run something in real time. I know that there are many sites and applications that are executing machine learning algorithms in real time, so I'm trying to understand how they realistically accomplish it. A few possibilities I can think of are:
They use these ML tools (such as Python, SciKit Learn, Tensor Flow) in real time to predict real time values and continually learn, but require a huge amount of computing power and optimization to keep performance at acceptable levels.
They use these ML tools for real time predictions, but only update the algorithm periodically in some form of offline or batch process. This still requires a lot of computing power and optimization for the real time predictions, but not nearly as much as option 1.
They use these ML tools offline to figure out the algorithm, and then translate that algorithm into some other language better suited for their need (i.e. Java, C++, JavaScript, etc.). This would seem very tedious to do.
Or is there a 4th option that I'm missing?