I do not quite understand your question. And I think there are some misconceptions:
I) A factory is a design pattern used to separate object creation from object consumption. There are two possible ways to deal with that:
1) You delegate instance creation to a separate object
2) A static method is used to create instances
The advantage: Dependencies are no longer created. The object in need asks for its dependencies. Factories allow dependency injection. But instead of managing a dozen factories and all the wiring up, you leave that for an abstraction called (DI-) container.
II) An Interface is the C#/Java way of dealing with multiple inheritance. Instead of letting object inherit implementations from multiple objects, you only inherit an API so to say. An interface is a contract that defines an API and leaves the implementation to the object. Objects define families of somehow related objects via an is-a-relationship, you could say, interfaces define a relationship of different objects sharing common behaviour.
What does that mean for (I)?
If you have a factory for different objects who follow the same API, the return-type of the factory is the common interface ( think of ILogging with FileLogger, DatabaseLogger, EventlogLogger and so on).
The consuming object only gets something that behaves like a Logger. It doesn't need to know which kind of logger it is using - even if it is a NullLogger.
III) A Proxy could be any object mimicing another object. A common usecase is mock objects. You wrap one functionality with another.
Which one you want to implement depends on your intention:
If you are designing an object which needs somehow to use a common API for
UdpStackExchangeClient you would imlement it via IStackExchangeClient and a factory
UdpStackExchangeClient have a different API and you want one behave like the other you would use a proxy wrapping up the other.
If you have IStackExchangeClient as an interface and
UdpStackExchangeClient implement it as well, using a proxy makes no sense, since the consuming object does not know which one it is going to use.
I've thought hard about it and I can't choose between one style or the other, but clearly some parts of the .NET framework use factories, and some parts don't, even when there are multiple underlying implementations of its interface.
Note, that the framework has a history and was not always designed thoroughly (I would bet), which might be the reason for some inconsistencies.