CDNs are used mostly for really static content: content which is the same for every or most users, and which will probably stay unchanged for days, weeks or months. CDNs are ideal for CSS files, JavaScript or images: they don't change much, they are usually cached for a year on client side anyway, and when they change, propagating the change to CDNs and clients is painless (using a revision number in the URI, such as global.css?r=514 is one of the ways to do it).
CDNs can also be used for content which is dynamic, but:
- Either is identical for every user,
- Or stays unchanged for days, weeks or months.
The more the content is dynamic, the less beneficial is the CDN. For example, you can use a CDN for a blog where content changes rarely (say a new blog post per week). While being dynamic, the content is the same for every user and doesn't need to be refreshed too often (for example, caching all blog posts for a hour is OK, because not being able to see the most recent post for an hour is acceptable in this case).
If the content is more dynamic, it is processed directly by the application servers, not the CDNs. By the way, crawling the website would be practically impossible, because the content will change during the crawling process or vary from user to user. For example, there is no way for a CDN to cache my Amazon purchasing page: not only do I need the up-to-date version of the webpage, containing my last purchase I've done five seconds ago, but there is no benefit of having a CDN at this level.
Note that since the primary goal of a CDN is to be close to the users (i.e. a user from New Delhi shouldn't be waiting for a server in Nevada to process his request), the same technique is used for large sites as well: dynamic requests are often processed by servers which are close enough to the users, when the company can afford building several data centers around the world.
