But what does it really know? By passing in a delegate, such as a callback, event handler, factory method, etc, all that is known by the code that gets this delegate reference is that it has a reference to something it can or should call at a specific time, or for a specific purpose. It doesn't have to know exactly what that call will do (outside of any return value expected), where the actual code that will run lives, how it was chosen to be used as a delegate, etc. It needs no knowledge about this delegate other than it has it, and knows how to call it, which are the minimum requirements (in C# at least).
The delegate definition is therefore little more than a mini-interface; you define a delegate type, possibly named as to its ostensible purpose in consuming code, that expects a certain number of named parameters and is expected in turn to return a particular value. This is simplified even further with .NET 3.5's built-in generic delegate types like Action, Func, Predicate, Comparison etc; you don't even have to define your own strongly-typed delegate, just ask for a Func<string, string>
(the downside being that general-purpose generics are, well, general-purpose and generic; you can't use the delegate type name or parameter names to self-document what the delegate is expected to do for the calling code, or what types of things you should give it).
It's similar, then, to a full interface definition, which defines a list of such methods, by name, with fully-defined signatures, all of which must be available in an implementing class. In Java, this is more or less how the same functionality you get with .NET delegates is achieved; an interface is defined with a single method representing the delegate method, classes exposing this method for use as a delegate are marked as implementing the interface, and a reference to the class is given, on which the method can be called. Delegates simply remove the middleman; you no longer care that the class containing the method implements any defined interface, you care that the actual method in question matches the required signature (here, signature is just parameter list and return type, not including method name). The class containing that method could then have a dozen methods with the same parameters and return type, but different names, and some machination your code doesn't have to care about can choose a specific one to give to you as a delegate.
Understand that while C# is considered an object-oriented language, it is also a multi-paradigm language, incorporating procedural and functional aspects of other languages into itself on request or demonstrated need. Delegates were a lesson learned by observing Java programmers and their travails of setting up simple event-driven, multi-tier applications in a strictly-enforced object-oriented paradigm, requiring hundreds of highly specific single-method interfaces. Java eventually worked around this problem by allowing anonymous interface implementations, so code that needed to inject an interface could define a one-off implementation inline, but it can still be a bit verbose. The rumor mill is also churning that Oracle has given in to popular demand, and lambda statements and other delegates will be part of the upcoming Java 8 specification (along with a lot of other syntax sugar goodness that Java devs are salivating over after looking across the fence to the C# side).
C++ avoided that by retaining the highly pointer-based flexibility of the C lineage, but its limitation was that, unlike interfaces, function pointers couldn't self-document; all you could know, without external documentation or traversing source code, is that you need to pass the address of a function, but you couldn't know anything about what parameters the consumer would try to pass, what it might expect as a return value, what your function might be expected to do and when it might be called, etc. As the ability to write self-documenting code was a key tenet of C#'s language spec, the designers combined the two, by allowing function pointers in a sense, but requiring a strong definition of the method's I/O signature as a type, allowing compiler-checked, self-describing function pointers without needing full interfaces.