Is it fair/legal to have a sort of licenses/credit page linked in the footer of my page, that simply lists out every licensed thing that I use?
Yes, I think so. It is very common for a copyright notice to be included at the bottom of every page of a given site. If the page contains small icons that you don't own the copyright to, this is the best place to note it. If there are many such, with many distinct attributions, then you can indicate as much, and provide a link to the details, as you suggest.
By the way, it's better to err on the side of attribution that is too visible than attribution that is not visible enough. When you're adding a copyright notice, it may seem glaringly unsubtle to you (because you're focusing on it), so you might feel tempted to make it ridiculously tiny and/or ridiculously faint. Resist that temptation. It's OK (and common) for it to be somewhat smaller and optionally fainter than the regular page font, but it should still be easy to read. Since it's in the footer, and it's very obvious what it is, people who aren't interested in it will ignore it (without even noticing that they're ignoring it).
If I do, do I need to render the icon on that page too so that people know immediately know to what I am referring, or can I just link back to the source pages?
If the icon has a title, then technically that's all that's needed to identify it (and you have to provide the title anyway as part of the attribution requirement); but rendering the icon itself sounds like a good idea, IMHO.
<!-- comment -->
in my HTML is acceptable, or if I need a "Credits" page, or if I can bury it 20 links deep on my website, or what.