6

I came across the term identity mapping in some papers about neural networks but am not sure what it is supposed to mean in that context. I'm guessing it means to map a sort of input to an output?

I came across this term in the papers Deep Residual Learning for Image Recognition (2015) and Identity Mappings in Deep Residual Networks (2016), both by He et al.

1

2 Answers 2

6

I'm not an neural network expert but I understand that identity mapping ensures that the output of some multilayer neural net is ensured to be equal to its input. Such a net is also called a replicator.

I have understood that such identity/replication facilitates unsupervised training, and that the hidden layers of such nets can be used for feature detection and data compression.

Further reading:

2
  • Aren't they called "autoencoders" instead of "replicators"?
    – chris
    Jan 26, 2018 at 16:15
  • 1
    @ChrisAnderson aren't autoencoders are more general concept compared to the replicator ? I think that the replicator always has to replicate 1 to 1 its input at a certain layer of the net (i.e. identity) while autoencoders map themselves to the input, but not necessarily with a 1 to 1 layer. There's a link to an SE contribution about this difference.
    – Christophe
    Jan 26, 2018 at 16:58
5

It's the same as it is in Algebra.

An identity map or identity function gives out exactly what it got.

When they say:

h(xl) = xl

They mean h is an identity mapping / function. If you give it xl it will give you back xl.

h might be something else but once they say it's h(xl) = xl then it's an identity map / function.

I don't see anything here to suggest the meaning changes once you're in a neural network context.

It might seem pointless to have a map or function that gives you back what you gave it, and it kind of is. The need for them arises from using a system (an architecture, framework, or whatever) that expects a map/function in a step that actually isn't needed in this case. Rather than build a new system for this case we use an identity map/function that lets us reuse the old system.

It's the same kind of thinking that invented 0, the null object pattern, the option type, and some of the reasons I pass in empty collections. The system expects something but you need that something to be nothing. These constructs give the nothing a form that wont break the system.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.