What advantages would the Twitter streaming API provide?

Question

I'm in the very early stages of writing an application to mine through Twitter data looking for certain types of information. I've been using Twitterizer, which I have found effective in downloading the information (it provides interfaces to the "old" REST based API, and the newer streaming one).

This would only be for personal use, so I don't think I would max out the number of API calls per hour on the old API, and it seems as though the streaming API is slightly slower. On the surface, it seems like the REST would suit me better, but I'm wondering if I'm missing any hidden advantages to using the streaming API that could benefit my application.

Answer 1

If your goal is to get information as it comes into the system, use the API designed for that - with the streaming API you have a persistent connection and the message will appear in your stream immediately with minimal overhead.

You can get the same information running single queries, but you would need to deal with polling intervals and duplicate messages.

If running searches on recent rather than new data provides the result you need, the streaming API may well seem slower - while data is added to twitter very quickly, it is not as fast as retrieving data that is already in place.

Answer 2

Mining real-time trending topics would be an idea that takes advantage of the Twitter Streaming API. I had tried to develop a tool like that sometime ago. I don't know about now, but back then only 1% of the tweets was available in the streaming API, which made it quite difficult to construct valid and meaningful trending topics. The low volume of available tweets and their geographical sparsity were the reasons I dropped the project.