I hope this doesnt get perceived as ambiguous or too vague, but i'm attempting to make a service that monitors twitter and archives tweets. Apart from PHP cron jobs i cant think of any way to detect if there has been a new tweet posted?

I'm not too fussed about what programming language i have to write it in, or what framework i have to work in. Also whether its desktop or web based.

What route would everyone go down for this?


2 Answers 2


Twitter provides an API for you to query a particular timeline. Take a look at the dev portal for details, but the gist of it is that you can create a Twitter client, use an appropriate authentication method, and poll the site at some time interval (keeping the API limits in mind).

There are libraries for various languages that will do a lot of the legwork around authentication and queries for you.


How important is timeliness? What about the possibility of missing tweets? By monitors twitter do you mean all of twitter or just some users?

You basically have two options, depending on your needs. The polling Search API (or some other feed) or the use a streaming API such as the firehose.

Which I'd use depends on what I want to archive. A simple query over the public tweets - Search API. A complex query over all the public tweets - firehose2 (if I could get it) orstatuses/filter. Private tweets - Poll statatus/home_timeline and set up a user as required.

As you can see, in most cases you will need to poll, and a scheduled task/cron job is a valid way to handle that. But you can get a live feed with the firehose and statuses/filter APIs, in these cases you will need an app running all the time that response to new bits of XML being thrown at it.

  • Yes, i would get users to sign up, so it would only be certain users. I'd ideally be polling live and i'd like to detect if that user tweets Commented Jan 20, 2011 at 14:14
  • 1
    @benhowdle89 I am not sure if you can poll Twitter "live". You will hit the API calls upper limit pretty quickly then.
    – Mayank
    Commented Jan 20, 2011 at 15:42
  • oh ok, no problem. My only thought is that, if i only poll once a day its going to be a big mysql insert of all the signed up users tweets. Can you think of a more optimal solution? Commented Jan 20, 2011 at 15:50
  • I'd check out statuses/filter, you might be able to get what you want with that (this is live - you are NOT polling).
    – mlk
    Commented Jan 20, 2011 at 16:13

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.