User Generated Content and Hotlink Protection

Question

I have a website that allows for users to paste content (like snippets of code, etc) for sharing. Like Pastebin and Github, I also have a "raw" link that will display the raw contents of those posts.

However, some users are posting up code and then using our service as a host for distributing content that violates our TOS (for example, javascript code and then linking to that code from external sites).

Running on NGINX and PHP, what is the best way to manage this?

I have a feature that when reported, I can disable the raw version of a particular post. However, it is not feasible to monitor each and every post (and then be sure that I understand what is good / what is bad).

Is my only solution to disable raw functionality across the board? Should I block the raw versions from sites like facebook (using referrer maybe)? I played around with hotlink protection, but in all truth, it doesn't really appear to work all that well (or it could be just my configuration of it).

Answer 1

Output raw content in plain text and tell (modern) browsers to honor it.

header("Content-Type: text/plain");
header('X-Content-Type-Options: nosniff');

You can also do a referrer check, but that will disable all external linking to the raw content.

Answer 2

Maybe every time you get a request for a raw version of a file with a Referer header that's not your site, log it. Then it's pretty easy to build a list of "top suspicious content in the last X hours that admins should check". You can't check every bit of content, but maybe keeping on top of that would be ok? You could even weight the chart so it's by bandwidth (requests x size), that way you know you are dealing with the most troublesome ones first.