How does Facebook strip html/apostrophes for XSS but also display it?

Question

I'm not quite sure if this is a question for programmers.se rather than stackoverflow, but here goes. So Facebook [or any other large company] when given something like an apostrophe or html, can strip it of its malicious intent, but still display it properly. My current sanitizing function in PHP just strips those characters/makes them harmless via htmlentities() and such. So if I wrote an HTML tag, I would want it to be sanitized but also displayed on the website. How do I do this?

Answer 1

In general anything entered by a user (or untrusted machine or software using an API) needs to be escaped before embedding it within code (HTML, JavaScript, etc.) that is interpreted. Escaping is I think what you mean by "making it harmless". Most libraries have APIs (like htmlentities()) to facilitate this.

If you don't escape it, storing it is basically the only safe thing you can do. Analysis can be OK, as long as the analyzer cannot be commandeered by its input (i.e. it is robust and defensive and has no exploit).

Modifying input (e.g. stripping dangerous characters) can also be effective, but it is hard to do it in such a way that legitimate characters are suppressed (false positives). For example, if someone's name is John O'Malley-O'Hara, you don't want the system to remove the apostrophes (or the text between them), even though they look like single-quote delimiters common in code. In other words, it is so hard to make sure that input modification is done right that is it perhaps better to not do it at all.

I think the best approach is to treat all input carefully and escape it when displaying it. Some languages and frameworks can assist you with this (see "taint mode").

Answer 2

First, you need to figure out what kind of text your input represents. One possibility is raw text: a letter means a letter, an angle bracket means an angle bracket, a backslash means a backslash, and so on. Or maybe it's text with backslash escapes: a letter means a letter, an angle bracket means an angle bracket, but a backslash starts an escape sequence. Another possibility is HTML markup: a left angle bracket starts an HTML element, an ampersand starts an entity, and so on. There are all sorts of other markup languages: XML, Markdown, bbcode, etc.

Validation: You might get an input that doesn't fit your interpretation, like malformed HTML markup. You should either reject it or automatically correct it. Maybe you only want to allow certain tags in your HTML markup. You should do something with the other tags: delete them, treat them as literal text (ie, display them as a left angle bracket, letters, right angle bracket), reject the input, or whatever. You will probably need to parse the input according to your interpretation in order to validate it; if you try to do it with regular expressions, you will probably get it wrong.

Now that you know what you have (interpretation), and you know that you have it (validation and/or parsing), it should be easy to translate it into whatever you want to render it as. When rendering raw text to HTML, you need to pay attention to the characters that are special to HTML. If you're rendering some wiki markup, you need to map the wiki markup to HTML markup---and again, if the wiki markup can contains characters special to HTML, they need to be escaped. If you're rendering a subset of HTML and you've already validated the input so it contains only the subset you allow, then you're done.

That's the whole process. Now you can look at various libraries and decide whether they do the translation you need. For example, IIUC, htmlentities() renders a string representing raw text to HTML markup. Maybe. Depending on how you wanted to render newlines. Probably the output should be though of as HTML-where-newlines-are-still-significant.

It's easy to deceive yourself into thinking that shortcuts are okay (eg "I'll just remove script tags with a regexp"). Beware.