Sites like webpagetest allow users to enter a website url and a test location, to run a speed test on the site from multiple locations using real browsers. Can anyone give me a basic idea of how sites like this work? You also have plugin's like Aptimize latency simulator or charles web debugging proxy app, that simulate the delay while accessing a site from different locations. I am assuming since these are plugin's these function in a different way. How do these plugin's work ?
Webpagetest uses Dummynet for traffic shaping the connection (only the last mile though). Dummynet was originally built for testing networking protocols and does an excellent job at constructing a virtual network that behaves like the real world and is pretty much the gold standard for traffic shaping (which includes latency injection, bandwidth tuning and pacet loss). It was originally part of BSD but has since been ported to Linux and Windows and is also used by Apple for the Mac traffic shaping tool in XCode.
Dummynet works at the packet level and lets you construct arbitrary "pipes" for packets to go through (and you can even chain them together). You assign each of those pipes a latency, bandwidth, packet loss and buffer size. Packets are added to the pipes based on rules that you define (ipfw packet matching rules) and if a pipe's buffer is full the packet will get dropped (like with real-world routers). At a regular interval (depends on kernel Hz setting but best if it is 1ms or faster), Dummynet checks each pipe to see is packets should come out (if they have been waiting long enough and if letting the packet out will stay under the configured bandwidth limit).
In the case of Webpagetest, 2 pipes are configured. One for the outbound packets and one for the inbound packets with the appropriate bandwidth on each and the latency is split between them (25ms in each direction for the default 50ms DSL configuration).
It is important to only simulate the last-mile of the link because the network gets REALLY complicated beyond that unless all of your content comes from a single server (CDN's and 3rd-party content usually come from different paths). Theoretically you could build a complete simulation in dummynet but it would be quite a hairy mess.
Plugins like Charles get as close as they can from the application-level. They will generally use similar logic but apply it when writing the data to the TCP connection. Depending on how your browser makes connections to the plugin and how that plugin makes connections to the outside world the results may differ from what you would see from applying it at the packet level. It's a reasonable approximation but I highly recommend using a packet-level simulator when possible.